2024-09-11T06:35:58.163 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-11T06:35:58.181 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T06:35:58.259 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900631 branch: squid description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_nbd} email: ceph-qa@ceph.com first_in_suite: false job_id: '7900631' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi no_nested_subset: false openstack: - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: squid ceph: conf: global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluefs allocator: hybrid bluestore allocator: hybrid bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: extra_packages: - rbd-nbd flavor: default sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: daaf78de334647fc0c800f4ac495cc9475a26bae owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - osd.0 - osd.1 - osd.2 - - mon.b - mgr.x - osd.3 - osd.4 - osd.5 - - client.0 seed: 6175 sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc sleep_before_teardown: 0 subset: 88/128 suite: rbd suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_daaf78de334647fc0c800f4ac495cc9475a26bae/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: daaf78de334647fc0c800f4ac495cc9475a26bae targets: smithi071.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAsPkUEtaeK9fdViOAbo1tNJi31yoOvkLCCbzicWSl6C3EKI0N53fBaj0cY+pT7jRddjNtDimhk3ZvXnK8/21Hw= smithi089.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL+OXkIXcTcVfMSPnVHzyHnYRQsMzpHhB9KsmVGdh1RuVfEkyD3P2hIIvvJU9BIpx29hVx069Ns93Iu/wq1dH6o= smithi176.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL8YgroLZ6X4Agx78ypmwC9s8cDMc0pbauE0dsOzTE48O2rhwQpTvU5VlPREuI5UfCWL+PhgtZyv3Iozp+aYRuk= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - workunit: clients: client.0: - rbd/rbd-nbd.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_21:16:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-11T06:35:58.259 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_daaf78de334647fc0c800f4ac495cc9475a26bae/qa; will attempt to use it 2024-09-11T06:35:58.260 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_daaf78de334647fc0c800f4ac495cc9475a26bae/qa/tasks 2024-09-11T06:35:58.260 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-11T06:35:58.262 INFO:teuthology.task.internal:Checking packages... 2024-09-11T06:35:58.285 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc' 2024-09-11T06:35:58.285 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-11T06:35:58.285 INFO:teuthology.packaging:ref: None 2024-09-11T06:35:58.285 INFO:teuthology.packaging:tag: None 2024-09-11T06:35:58.285 INFO:teuthology.packaging:branch: squid 2024-09-11T06:35:58.285 INFO:teuthology.packaging:sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T06:35:58.285 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=squid 2024-09-11T06:35:58.494 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-298-gb53b6d3e-1jammy 2024-09-11T06:35:58.496 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-11T06:35:58.507 INFO:teuthology.task.internal:no buildpackages task found 2024-09-11T06:35:58.507 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-11T06:35:58.520 INFO:teuthology.task.internal:Saving configuration 2024-09-11T06:35:58.532 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-11T06:35:58.541 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-11T06:35:58.564 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900631', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 06:30:17.667859', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICJr7+E0dl8KiQLM25w+lAeGaXKfEfGf5RPBNQL46e82'} 2024-09-11T06:35:58.585 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi089.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900631', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 06:30:17.670487', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINLJUc6c2Ztugy48YB4+op5GqXJdyiKHKCldfON7s/II'} 2024-09-11T06:35:58.606 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi176.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900631', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 06:30:17.669215', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJwatNrIUv1Ekd8/feAJJ7CQIw73TJ68t2ieGvKEjcJR'} 2024-09-11T06:35:58.606 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-11T06:35:58.617 INFO:teuthology.task.internal:roles: ubuntu@smithi071.front.sepia.ceph.com - ['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'] 2024-09-11T06:35:58.617 INFO:teuthology.task.internal:roles: ubuntu@smithi089.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'] 2024-09-11T06:35:58.617 INFO:teuthology.task.internal:roles: ubuntu@smithi176.front.sepia.ceph.com - ['client.0'] 2024-09-11T06:35:58.617 INFO:teuthology.run_tasks:Running task console_log... 2024-09-11T06:35:58.741 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f95b6be6f80>, signals=[15]) 2024-09-11T06:35:58.741 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-11T06:35:58.752 INFO:teuthology.task.internal:Opening connections... 2024-09-11T06:35:58.752 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi071.front.sepia.ceph.com 2024-09-11T06:35:58.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T06:35:58.835 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi089.front.sepia.ceph.com 2024-09-11T06:35:58.835 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T06:35:58.913 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi176.front.sepia.ceph.com 2024-09-11T06:35:58.914 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T06:35:58.992 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-11T06:35:58.999 DEBUG:teuthology.orchestra.run.smithi071:> uname -m 2024-09-11T06:35:59.005 INFO:teuthology.orchestra.run.smithi071.stdout:x86_64 2024-09-11T06:35:59.005 DEBUG:teuthology.orchestra.run.smithi071:> cat /etc/os-release 2024-09-11T06:35:59.050 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-11T06:35:59.050 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="Ubuntu" 2024-09-11T06:35:59.050 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="22.04" 2024-09-11T06:35:59.050 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-11T06:35:59.050 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_CODENAME=jammy 2024-09-11T06:35:59.050 INFO:teuthology.orchestra.run.smithi071.stdout:ID=ubuntu 2024-09-11T06:35:59.051 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE=debian 2024-09-11T06:35:59.051 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-11T06:35:59.051 INFO:teuthology.orchestra.run.smithi071.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-11T06:35:59.051 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-11T06:35:59.051 INFO:teuthology.orchestra.run.smithi071.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-11T06:35:59.051 INFO:teuthology.orchestra.run.smithi071.stdout:UBUNTU_CODENAME=jammy 2024-09-11T06:35:59.052 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2024-09-11T06:35:59.087 DEBUG:teuthology.orchestra.run.smithi089:> uname -m 2024-09-11T06:35:59.093 INFO:teuthology.orchestra.run.smithi089.stdout:x86_64 2024-09-11T06:35:59.093 DEBUG:teuthology.orchestra.run.smithi089:> cat /etc/os-release 2024-09-11T06:35:59.140 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-11T06:35:59.140 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="Ubuntu" 2024-09-11T06:35:59.140 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="22.04" 2024-09-11T06:35:59.140 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-11T06:35:59.140 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_CODENAME=jammy 2024-09-11T06:35:59.140 INFO:teuthology.orchestra.run.smithi089.stdout:ID=ubuntu 2024-09-11T06:35:59.141 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE=debian 2024-09-11T06:35:59.141 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-11T06:35:59.141 INFO:teuthology.orchestra.run.smithi089.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-11T06:35:59.141 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-11T06:35:59.141 INFO:teuthology.orchestra.run.smithi089.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-11T06:35:59.141 INFO:teuthology.orchestra.run.smithi089.stdout:UBUNTU_CODENAME=jammy 2024-09-11T06:35:59.141 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2024-09-11T06:35:59.169 DEBUG:teuthology.orchestra.run.smithi176:> uname -m 2024-09-11T06:35:59.174 INFO:teuthology.orchestra.run.smithi176.stdout:x86_64 2024-09-11T06:35:59.175 DEBUG:teuthology.orchestra.run.smithi176:> cat /etc/os-release 2024-09-11T06:35:59.221 INFO:teuthology.orchestra.run.smithi176.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-11T06:35:59.221 INFO:teuthology.orchestra.run.smithi176.stdout:NAME="Ubuntu" 2024-09-11T06:35:59.222 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION_ID="22.04" 2024-09-11T06:35:59.222 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-11T06:35:59.222 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION_CODENAME=jammy 2024-09-11T06:35:59.222 INFO:teuthology.orchestra.run.smithi176.stdout:ID=ubuntu 2024-09-11T06:35:59.222 INFO:teuthology.orchestra.run.smithi176.stdout:ID_LIKE=debian 2024-09-11T06:35:59.222 INFO:teuthology.orchestra.run.smithi176.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-11T06:35:59.222 INFO:teuthology.orchestra.run.smithi176.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-11T06:35:59.222 INFO:teuthology.orchestra.run.smithi176.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-11T06:35:59.222 INFO:teuthology.orchestra.run.smithi176.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-11T06:35:59.222 INFO:teuthology.orchestra.run.smithi176.stdout:UBUNTU_CODENAME=jammy 2024-09-11T06:35:59.222 INFO:teuthology.lock.ops:Updating smithi176.front.sepia.ceph.com on lock server 2024-09-11T06:35:59.244 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-11T06:35:59.252 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-11T06:35:59.265 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-11T06:35:59.266 DEBUG:teuthology.orchestra.run.smithi071:> test '!' -e /home/ubuntu/cephtest 2024-09-11T06:35:59.268 DEBUG:teuthology.orchestra.run.smithi089:> test '!' -e /home/ubuntu/cephtest 2024-09-11T06:35:59.271 DEBUG:teuthology.orchestra.run.smithi176:> test '!' -e /home/ubuntu/cephtest 2024-09-11T06:35:59.275 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-11T06:35:59.284 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-11T06:35:59.284 DEBUG:teuthology.orchestra.run.smithi071:> test -z $(ls -A /var/lib/ceph) 2024-09-11T06:35:59.315 DEBUG:teuthology.orchestra.run.smithi089:> test -z $(ls -A /var/lib/ceph) 2024-09-11T06:35:59.318 DEBUG:teuthology.orchestra.run.smithi176:> test -z $(ls -A /var/lib/ceph) 2024-09-11T06:35:59.393 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-11T06:35:59.459 INFO:teuthology.run_tasks:Running task kernel... 2024-09-11T06:35:59.474 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-11T06:35:59.474 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'client.0': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-11T06:35:59.474 DEBUG:teuthology.orchestra.run.smithi071:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T06:35:59.475 DEBUG:teuthology.orchestra.run.smithi089:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T06:35:59.475 DEBUG:teuthology.orchestra.run.smithi176:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T06:35:59.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T06:35:59.480 DEBUG:teuthology.orchestra.run.smithi089:> uname -r 2024-09-11T06:35:59.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T06:35:59.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T06:35:59.482 DEBUG:teuthology.orchestra.run.smithi176:> uname -r 2024-09-11T06:35:59.482 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2024-09-11T06:35:59.528 INFO:teuthology.orchestra.run.smithi089.stdout:5.15.0-56-generic 2024-09-11T06:35:59.528 INFO:teuthology.task.kernel:Running kernel on smithi089: 5.15.0-56-generic 2024-09-11T06:35:59.528 DEBUG:teuthology.orchestra.run.smithi089:> sudo apt-get clean 2024-09-11T06:35:59.529 INFO:teuthology.orchestra.run.smithi176.stdout:5.15.0-56-generic 2024-09-11T06:35:59.530 INFO:teuthology.task.kernel:Running kernel on smithi176: 5.15.0-56-generic 2024-09-11T06:35:59.530 DEBUG:teuthology.orchestra.run.smithi176:> sudo apt-get clean 2024-09-11T06:35:59.531 INFO:teuthology.orchestra.run.smithi071.stdout:5.15.0-56-generic 2024-09-11T06:35:59.531 INFO:teuthology.task.kernel:Running kernel on smithi071: 5.15.0-56-generic 2024-09-11T06:35:59.531 DEBUG:teuthology.orchestra.run.smithi071:> sudo apt-get clean 2024-09-11T06:35:59.649 DEBUG:teuthology.orchestra.run.smithi089:> sudo apt-get update 2024-09-11T06:35:59.660 DEBUG:teuthology.orchestra.run.smithi176:> sudo apt-get update 2024-09-11T06:35:59.661 DEBUG:teuthology.orchestra.run.smithi071:> sudo apt-get update 2024-09-11T06:35:59.795 INFO:teuthology.orchestra.run.smithi176.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-11T06:35:59.800 INFO:teuthology.orchestra.run.smithi089.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-11T06:35:59.808 INFO:teuthology.orchestra.run.smithi071.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-11T06:35:59.818 INFO:teuthology.orchestra.run.smithi176.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-11T06:35:59.891 INFO:teuthology.orchestra.run.smithi176.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-11T06:35:59.898 INFO:teuthology.orchestra.run.smithi089.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-11T06:35:59.910 INFO:teuthology.orchestra.run.smithi071.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-11T06:35:59.917 INFO:teuthology.orchestra.run.smithi176.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-11T06:35:59.980 INFO:teuthology.orchestra.run.smithi089.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-11T06:35:59.992 INFO:teuthology.orchestra.run.smithi071.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-11T06:35:59.994 INFO:teuthology.orchestra.run.smithi176.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [690 kB] 2024-09-11T06:36:00.015 INFO:teuthology.orchestra.run.smithi089.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,771 kB] 2024-09-11T06:36:00.039 INFO:teuthology.orchestra.run.smithi071.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,771 kB] 2024-09-11T06:36:00.066 INFO:teuthology.orchestra.run.smithi176.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,988 kB] 2024-09-11T06:36:00.101 INFO:teuthology.orchestra.run.smithi176.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [349 kB] 2024-09-11T06:36:00.103 INFO:teuthology.orchestra.run.smithi176.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-11T06:36:00.104 INFO:teuthology.orchestra.run.smithi176.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-11T06:36:00.104 INFO:teuthology.orchestra.run.smithi176.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,386 kB] 2024-09-11T06:36:00.118 INFO:teuthology.orchestra.run.smithi089.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [533 kB] 2024-09-11T06:36:00.120 INFO:teuthology.orchestra.run.smithi176.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [410 kB] 2024-09-11T06:36:00.123 INFO:teuthology.orchestra.run.smithi176.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-11T06:36:00.123 INFO:teuthology.orchestra.run.smithi176.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,123 kB] 2024-09-11T06:36:00.130 INFO:teuthology.orchestra.run.smithi176.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [731 kB] 2024-09-11T06:36:00.134 INFO:teuthology.orchestra.run.smithi089.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [291 kB] 2024-09-11T06:36:00.135 INFO:teuthology.orchestra.run.smithi176.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-11T06:36:00.137 INFO:teuthology.orchestra.run.smithi176.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.2 kB] 2024-09-11T06:36:00.138 INFO:teuthology.orchestra.run.smithi176.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-11T06:36:00.139 INFO:teuthology.orchestra.run.smithi089.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-11T06:36:00.139 INFO:teuthology.orchestra.run.smithi089.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-11T06:36:00.139 INFO:teuthology.orchestra.run.smithi089.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,327 kB] 2024-09-11T06:36:00.142 INFO:teuthology.orchestra.run.smithi071.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [533 kB] 2024-09-11T06:36:00.146 INFO:teuthology.orchestra.run.smithi071.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [291 kB] 2024-09-11T06:36:00.149 INFO:teuthology.orchestra.run.smithi071.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-11T06:36:00.149 INFO:teuthology.orchestra.run.smithi071.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-11T06:36:00.149 INFO:teuthology.orchestra.run.smithi071.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,327 kB] 2024-09-11T06:36:00.155 INFO:teuthology.orchestra.run.smithi089.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [400 kB] 2024-09-11T06:36:00.158 INFO:teuthology.orchestra.run.smithi089.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-11T06:36:00.158 INFO:teuthology.orchestra.run.smithi089.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [901 kB] 2024-09-11T06:36:00.159 INFO:teuthology.orchestra.run.smithi176.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-11T06:36:00.159 INFO:teuthology.orchestra.run.smithi176.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-11T06:36:00.159 INFO:teuthology.orchestra.run.smithi176.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-11T06:36:00.160 INFO:teuthology.orchestra.run.smithi176.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-11T06:36:00.160 INFO:teuthology.orchestra.run.smithi176.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-11T06:36:00.160 INFO:teuthology.orchestra.run.smithi176.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-11T06:36:00.160 INFO:teuthology.orchestra.run.smithi176.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-11T06:36:00.161 INFO:teuthology.orchestra.run.smithi176.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-11T06:36:00.161 INFO:teuthology.orchestra.run.smithi176.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-11T06:36:00.164 INFO:teuthology.orchestra.run.smithi089.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [626 kB] 2024-09-11T06:36:00.168 INFO:teuthology.orchestra.run.smithi089.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-11T06:36:00.169 INFO:teuthology.orchestra.run.smithi089.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-11T06:36:00.169 INFO:teuthology.orchestra.run.smithi071.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [400 kB] 2024-09-11T06:36:00.170 INFO:teuthology.orchestra.run.smithi089.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-11T06:36:00.170 INFO:teuthology.orchestra.run.smithi089.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-11T06:36:00.173 INFO:teuthology.orchestra.run.smithi071.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-11T06:36:00.173 INFO:teuthology.orchestra.run.smithi071.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [626 kB] 2024-09-11T06:36:00.177 INFO:teuthology.orchestra.run.smithi176.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-11T06:36:00.177 INFO:teuthology.orchestra.run.smithi176.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-11T06:36:00.177 INFO:teuthology.orchestra.run.smithi071.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [901 kB] 2024-09-11T06:36:00.183 INFO:teuthology.orchestra.run.smithi089.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-11T06:36:00.184 INFO:teuthology.orchestra.run.smithi071.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-11T06:36:00.185 INFO:teuthology.orchestra.run.smithi071.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-11T06:36:00.192 INFO:teuthology.orchestra.run.smithi071.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-11T06:36:00.192 INFO:teuthology.orchestra.run.smithi071.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-11T06:36:00.192 INFO:teuthology.orchestra.run.smithi071.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-11T06:36:00.211 INFO:teuthology.orchestra.run.smithi089.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-11T06:36:00.228 INFO:teuthology.orchestra.run.smithi071.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-11T06:36:00.335 INFO:teuthology.orchestra.run.smithi089.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-11T06:36:00.347 INFO:teuthology.orchestra.run.smithi071.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-11T06:36:00.365 INFO:teuthology.orchestra.run.smithi176.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,771 kB] 2024-09-11T06:36:00.475 INFO:teuthology.orchestra.run.smithi089.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [690 kB] 2024-09-11T06:36:00.486 INFO:teuthology.orchestra.run.smithi071.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [690 kB] 2024-09-11T06:36:00.668 INFO:teuthology.orchestra.run.smithi089.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,988 kB] 2024-09-11T06:36:00.680 INFO:teuthology.orchestra.run.smithi071.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,988 kB] 2024-09-11T06:36:00.782 INFO:teuthology.orchestra.run.smithi176.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [533 kB] 2024-09-11T06:36:00.799 INFO:teuthology.orchestra.run.smithi176.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [291 kB] 2024-09-11T06:36:00.807 INFO:teuthology.orchestra.run.smithi176.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-11T06:36:00.808 INFO:teuthology.orchestra.run.smithi176.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-11T06:36:00.809 INFO:teuthology.orchestra.run.smithi176.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,327 kB] 2024-09-11T06:36:00.826 INFO:teuthology.orchestra.run.smithi089.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [349 kB] 2024-09-11T06:36:00.837 INFO:teuthology.orchestra.run.smithi071.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [349 kB] 2024-09-11T06:36:00.840 INFO:teuthology.orchestra.run.smithi089.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-11T06:36:00.841 INFO:teuthology.orchestra.run.smithi089.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-11T06:36:00.842 INFO:teuthology.orchestra.run.smithi089.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,386 kB] 2024-09-11T06:36:00.851 INFO:teuthology.orchestra.run.smithi071.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-11T06:36:00.852 INFO:teuthology.orchestra.run.smithi071.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,386 kB] 2024-09-11T06:36:00.905 INFO:teuthology.orchestra.run.smithi176.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [400 kB] 2024-09-11T06:36:00.910 INFO:teuthology.orchestra.run.smithi089.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [410 kB] 2024-09-11T06:36:00.916 INFO:teuthology.orchestra.run.smithi176.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-11T06:36:00.916 INFO:teuthology.orchestra.run.smithi176.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [626 kB] 2024-09-11T06:36:00.919 INFO:teuthology.orchestra.run.smithi071.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-11T06:36:00.921 INFO:teuthology.orchestra.run.smithi071.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [410 kB] 2024-09-11T06:36:00.923 INFO:teuthology.orchestra.run.smithi089.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-11T06:36:00.923 INFO:teuthology.orchestra.run.smithi089.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [731 kB] 2024-09-11T06:36:00.932 INFO:teuthology.orchestra.run.smithi176.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [901 kB] 2024-09-11T06:36:00.937 INFO:teuthology.orchestra.run.smithi071.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-11T06:36:00.937 INFO:teuthology.orchestra.run.smithi071.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,123 kB] 2024-09-11T06:36:00.946 INFO:teuthology.orchestra.run.smithi089.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,123 kB] 2024-09-11T06:36:00.958 INFO:teuthology.orchestra.run.smithi176.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-11T06:36:00.964 INFO:teuthology.orchestra.run.smithi176.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-11T06:36:00.964 INFO:teuthology.orchestra.run.smithi176.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-11T06:36:00.966 INFO:teuthology.orchestra.run.smithi176.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-11T06:36:00.966 INFO:teuthology.orchestra.run.smithi176.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-11T06:36:00.971 INFO:teuthology.orchestra.run.smithi071.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [731 kB] 2024-09-11T06:36:00.978 INFO:teuthology.orchestra.run.smithi089.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-11T06:36:00.983 INFO:teuthology.orchestra.run.smithi089.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.2 kB] 2024-09-11T06:36:00.989 INFO:teuthology.orchestra.run.smithi071.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-11T06:36:00.990 INFO:teuthology.orchestra.run.smithi089.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-11T06:36:00.992 INFO:teuthology.orchestra.run.smithi089.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-11T06:36:00.992 INFO:teuthology.orchestra.run.smithi089.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-11T06:36:00.995 INFO:teuthology.orchestra.run.smithi071.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.2 kB] 2024-09-11T06:36:01.002 INFO:teuthology.orchestra.run.smithi071.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-11T06:36:01.003 INFO:teuthology.orchestra.run.smithi071.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-11T06:36:01.003 INFO:teuthology.orchestra.run.smithi071.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-11T06:36:01.045 INFO:teuthology.orchestra.run.smithi176.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-11T06:36:01.065 INFO:teuthology.orchestra.run.smithi089.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-11T06:36:01.066 INFO:teuthology.orchestra.run.smithi089.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-11T06:36:01.067 INFO:teuthology.orchestra.run.smithi089.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-11T06:36:01.069 INFO:teuthology.orchestra.run.smithi089.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-11T06:36:01.069 INFO:teuthology.orchestra.run.smithi089.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-11T06:36:01.072 INFO:teuthology.orchestra.run.smithi089.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-11T06:36:01.073 INFO:teuthology.orchestra.run.smithi089.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-11T06:36:01.079 INFO:teuthology.orchestra.run.smithi071.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-11T06:36:01.079 INFO:teuthology.orchestra.run.smithi071.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-11T06:36:01.080 INFO:teuthology.orchestra.run.smithi071.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-11T06:36:01.082 INFO:teuthology.orchestra.run.smithi071.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-11T06:36:01.082 INFO:teuthology.orchestra.run.smithi071.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-11T06:36:01.083 INFO:teuthology.orchestra.run.smithi071.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-11T06:36:01.084 INFO:teuthology.orchestra.run.smithi071.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-11T06:36:01.154 INFO:teuthology.orchestra.run.smithi089.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-11T06:36:01.154 INFO:teuthology.orchestra.run.smithi089.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-11T06:36:01.165 INFO:teuthology.orchestra.run.smithi071.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-11T06:36:01.165 INFO:teuthology.orchestra.run.smithi071.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-11T06:36:06.312 INFO:teuthology.orchestra.run.smithi176.stdout:Fetched 15.8 MB in 2s (7,727 kB/s) 2024-09-11T06:36:06.435 INFO:teuthology.orchestra.run.smithi089.stdout:Fetched 15.8 MB in 2s (7,387 kB/s) 2024-09-11T06:36:06.535 INFO:teuthology.orchestra.run.smithi071.stdout:Fetched 15.8 MB in 2s (7,355 kB/s) 2024-09-11T06:36:07.714 INFO:teuthology.orchestra.run.smithi176.stdout:Reading package lists... 2024-09-11T06:36:07.731 DEBUG:teuthology.orchestra.run.smithi176:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-11T06:36:07.794 INFO:teuthology.orchestra.run.smithi176.stdout:Reading package lists... 2024-09-11T06:36:07.821 INFO:teuthology.orchestra.run.smithi089.stdout:Reading package lists... 2024-09-11T06:36:07.838 DEBUG:teuthology.orchestra.run.smithi089:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-11T06:36:07.900 INFO:teuthology.orchestra.run.smithi089.stdout:Reading package lists... 2024-09-11T06:36:08.002 INFO:teuthology.orchestra.run.smithi176.stdout:Building dependency tree... 2024-09-11T06:36:08.003 INFO:teuthology.orchestra.run.smithi176.stdout:Reading state information... 2024-09-11T06:36:08.055 INFO:teuthology.orchestra.run.smithi071.stdout:Reading package lists... 2024-09-11T06:36:08.073 DEBUG:teuthology.orchestra.run.smithi071:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-11T06:36:08.109 INFO:teuthology.orchestra.run.smithi089.stdout:Building dependency tree... 2024-09-11T06:36:08.110 INFO:teuthology.orchestra.run.smithi089.stdout:Reading state information... 2024-09-11T06:36:08.128 INFO:teuthology.orchestra.run.smithi071.stdout:Reading package lists... 2024-09-11T06:36:08.177 INFO:teuthology.orchestra.run.smithi176.stdout:The following packages were automatically installed and are no longer required: 2024-09-11T06:36:08.177 INFO:teuthology.orchestra.run.smithi176.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-11T06:36:08.177 INFO:teuthology.orchestra.run.smithi176.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-11T06:36:08.178 INFO:teuthology.orchestra.run.smithi176.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-11T06:36:08.178 INFO:teuthology.orchestra.run.smithi176.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-11T06:36:08.178 INFO:teuthology.orchestra.run.smithi176.stdout: python2.7-minimal 2024-09-11T06:36:08.178 INFO:teuthology.orchestra.run.smithi176.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-11T06:36:08.179 INFO:teuthology.orchestra.run.smithi176.stdout:The following additional packages will be installed: 2024-09-11T06:36:08.180 INFO:teuthology.orchestra.run.smithi176.stdout: linux-generic linux-headers-5.15.0-119 linux-headers-5.15.0-119-generic 2024-09-11T06:36:08.180 INFO:teuthology.orchestra.run.smithi176.stdout: linux-headers-generic linux-image-5.15.0-119-generic 2024-09-11T06:36:08.180 INFO:teuthology.orchestra.run.smithi176.stdout: linux-modules-5.15.0-119-generic linux-modules-extra-5.15.0-119-generic 2024-09-11T06:36:08.181 INFO:teuthology.orchestra.run.smithi176.stdout:Suggested packages: 2024-09-11T06:36:08.181 INFO:teuthology.orchestra.run.smithi176.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-11T06:36:08.181 INFO:teuthology.orchestra.run.smithi176.stdout:Recommended packages: 2024-09-11T06:36:08.181 INFO:teuthology.orchestra.run.smithi176.stdout: thermald 2024-09-11T06:36:08.227 INFO:teuthology.orchestra.run.smithi176.stdout:The following NEW packages will be installed: 2024-09-11T06:36:08.227 INFO:teuthology.orchestra.run.smithi176.stdout: linux-headers-5.15.0-119 linux-headers-5.15.0-119-generic 2024-09-11T06:36:08.227 INFO:teuthology.orchestra.run.smithi176.stdout: linux-image-5.15.0-119-generic linux-modules-5.15.0-119-generic 2024-09-11T06:36:08.227 INFO:teuthology.orchestra.run.smithi176.stdout: linux-modules-extra-5.15.0-119-generic 2024-09-11T06:36:08.228 INFO:teuthology.orchestra.run.smithi176.stdout:The following packages will be upgraded: 2024-09-11T06:36:08.229 INFO:teuthology.orchestra.run.smithi176.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-11T06:36:08.280 INFO:teuthology.orchestra.run.smithi176.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-11T06:36:08.280 INFO:teuthology.orchestra.run.smithi176.stdout:Need to get 113 MB of archives. 2024-09-11T06:36:08.280 INFO:teuthology.orchestra.run.smithi176.stdout:After this operation, 583 MB of additional disk space will be used. 2024-09-11T06:36:08.280 INFO:teuthology.orchestra.run.smithi176.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-119-generic amd64 5.15.0-119.129 [22.6 MB] 2024-09-11T06:36:08.281 INFO:teuthology.orchestra.run.smithi089.stdout:The following packages were automatically installed and are no longer required: 2024-09-11T06:36:08.281 INFO:teuthology.orchestra.run.smithi089.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-11T06:36:08.281 INFO:teuthology.orchestra.run.smithi089.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-11T06:36:08.282 INFO:teuthology.orchestra.run.smithi089.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-11T06:36:08.282 INFO:teuthology.orchestra.run.smithi089.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-11T06:36:08.282 INFO:teuthology.orchestra.run.smithi089.stdout: python2.7-minimal 2024-09-11T06:36:08.282 INFO:teuthology.orchestra.run.smithi089.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-11T06:36:08.283 INFO:teuthology.orchestra.run.smithi089.stdout:The following additional packages will be installed: 2024-09-11T06:36:08.283 INFO:teuthology.orchestra.run.smithi089.stdout: linux-generic linux-headers-5.15.0-119 linux-headers-5.15.0-119-generic 2024-09-11T06:36:08.283 INFO:teuthology.orchestra.run.smithi089.stdout: linux-headers-generic linux-image-5.15.0-119-generic 2024-09-11T06:36:08.284 INFO:teuthology.orchestra.run.smithi089.stdout: linux-modules-5.15.0-119-generic linux-modules-extra-5.15.0-119-generic 2024-09-11T06:36:08.285 INFO:teuthology.orchestra.run.smithi089.stdout:Suggested packages: 2024-09-11T06:36:08.285 INFO:teuthology.orchestra.run.smithi089.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-11T06:36:08.285 INFO:teuthology.orchestra.run.smithi089.stdout:Recommended packages: 2024-09-11T06:36:08.285 INFO:teuthology.orchestra.run.smithi089.stdout: thermald 2024-09-11T06:36:08.312 INFO:teuthology.orchestra.run.smithi071.stdout:Building dependency tree... 2024-09-11T06:36:08.313 INFO:teuthology.orchestra.run.smithi071.stdout:Reading state information... 2024-09-11T06:36:08.341 INFO:teuthology.orchestra.run.smithi089.stdout:The following NEW packages will be installed: 2024-09-11T06:36:08.341 INFO:teuthology.orchestra.run.smithi089.stdout: linux-headers-5.15.0-119 linux-headers-5.15.0-119-generic 2024-09-11T06:36:08.341 INFO:teuthology.orchestra.run.smithi089.stdout: linux-image-5.15.0-119-generic linux-modules-5.15.0-119-generic 2024-09-11T06:36:08.341 INFO:teuthology.orchestra.run.smithi089.stdout: linux-modules-extra-5.15.0-119-generic 2024-09-11T06:36:08.342 INFO:teuthology.orchestra.run.smithi089.stdout:The following packages will be upgraded: 2024-09-11T06:36:08.343 INFO:teuthology.orchestra.run.smithi089.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-11T06:36:08.395 INFO:teuthology.orchestra.run.smithi089.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-11T06:36:08.395 INFO:teuthology.orchestra.run.smithi089.stdout:Need to get 113 MB of archives. 2024-09-11T06:36:08.395 INFO:teuthology.orchestra.run.smithi089.stdout:After this operation, 583 MB of additional disk space will be used. 2024-09-11T06:36:08.395 INFO:teuthology.orchestra.run.smithi089.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-119-generic amd64 5.15.0-119.129 [22.6 MB] 2024-09-11T06:36:08.478 INFO:teuthology.orchestra.run.smithi071.stdout:The following packages were automatically installed and are no longer required: 2024-09-11T06:36:08.478 INFO:teuthology.orchestra.run.smithi071.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-11T06:36:08.478 INFO:teuthology.orchestra.run.smithi071.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-11T06:36:08.478 INFO:teuthology.orchestra.run.smithi071.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-11T06:36:08.478 INFO:teuthology.orchestra.run.smithi071.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-11T06:36:08.479 INFO:teuthology.orchestra.run.smithi071.stdout: python2.7-minimal 2024-09-11T06:36:08.479 INFO:teuthology.orchestra.run.smithi071.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-11T06:36:08.480 INFO:teuthology.orchestra.run.smithi071.stdout:The following additional packages will be installed: 2024-09-11T06:36:08.480 INFO:teuthology.orchestra.run.smithi071.stdout: linux-generic linux-headers-5.15.0-119 linux-headers-5.15.0-119-generic 2024-09-11T06:36:08.480 INFO:teuthology.orchestra.run.smithi071.stdout: linux-headers-generic linux-image-5.15.0-119-generic 2024-09-11T06:36:08.480 INFO:teuthology.orchestra.run.smithi071.stdout: linux-modules-5.15.0-119-generic linux-modules-extra-5.15.0-119-generic 2024-09-11T06:36:08.481 INFO:teuthology.orchestra.run.smithi071.stdout:Suggested packages: 2024-09-11T06:36:08.482 INFO:teuthology.orchestra.run.smithi071.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-11T06:36:08.482 INFO:teuthology.orchestra.run.smithi071.stdout:Recommended packages: 2024-09-11T06:36:08.482 INFO:teuthology.orchestra.run.smithi071.stdout: thermald 2024-09-11T06:36:08.533 INFO:teuthology.orchestra.run.smithi071.stdout:The following NEW packages will be installed: 2024-09-11T06:36:08.533 INFO:teuthology.orchestra.run.smithi071.stdout: linux-headers-5.15.0-119 linux-headers-5.15.0-119-generic 2024-09-11T06:36:08.533 INFO:teuthology.orchestra.run.smithi071.stdout: linux-image-5.15.0-119-generic linux-modules-5.15.0-119-generic 2024-09-11T06:36:08.534 INFO:teuthology.orchestra.run.smithi071.stdout: linux-modules-extra-5.15.0-119-generic 2024-09-11T06:36:08.535 INFO:teuthology.orchestra.run.smithi071.stdout:The following packages will be upgraded: 2024-09-11T06:36:08.535 INFO:teuthology.orchestra.run.smithi071.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-11T06:36:08.716 INFO:teuthology.orchestra.run.smithi071.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-11T06:36:08.716 INFO:teuthology.orchestra.run.smithi071.stdout:Need to get 113 MB of archives. 2024-09-11T06:36:08.716 INFO:teuthology.orchestra.run.smithi071.stdout:After this operation, 583 MB of additional disk space will be used. 2024-09-11T06:36:08.716 INFO:teuthology.orchestra.run.smithi071.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-119-generic amd64 5.15.0-119.129 [22.6 MB] 2024-09-11T06:36:08.726 INFO:teuthology.orchestra.run.smithi176.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-119-generic amd64 5.15.0-119.129 [11.5 MB] 2024-09-11T06:36:08.830 INFO:teuthology.orchestra.run.smithi176.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-119-generic amd64 5.15.0-119.129 [63.9 MB] 2024-09-11T06:36:08.833 INFO:teuthology.orchestra.run.smithi089.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-119-generic amd64 5.15.0-119.129 [11.5 MB] 2024-09-11T06:36:08.924 INFO:teuthology.orchestra.run.smithi089.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-119-generic amd64 5.15.0-119.129 [63.9 MB] 2024-09-11T06:36:09.663 INFO:teuthology.orchestra.run.smithi176.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.119.119 [1,694 B] 2024-09-11T06:36:09.663 INFO:teuthology.orchestra.run.smithi176.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.119.119 [2,502 B] 2024-09-11T06:36:09.663 INFO:teuthology.orchestra.run.smithi176.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-119 all 5.15.0-119.129 [12.3 MB] 2024-09-11T06:36:09.726 INFO:teuthology.orchestra.run.smithi089.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.119.119 [1,694 B] 2024-09-11T06:36:09.726 INFO:teuthology.orchestra.run.smithi089.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.119.119 [2,502 B] 2024-09-11T06:36:09.726 INFO:teuthology.orchestra.run.smithi089.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-119 all 5.15.0-119.129 [12.3 MB] 2024-09-11T06:36:09.787 INFO:teuthology.orchestra.run.smithi176.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-119-generic amd64 5.15.0-119.129 [2,879 kB] 2024-09-11T06:36:09.810 INFO:teuthology.orchestra.run.smithi176.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.119.119 [2,350 B] 2024-09-11T06:36:09.846 INFO:teuthology.orchestra.run.smithi089.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-119-generic amd64 5.15.0-119.129 [2,879 kB] 2024-09-11T06:36:09.941 INFO:teuthology.orchestra.run.smithi089.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.119.119 [2,350 B] 2024-09-11T06:36:10.118 INFO:teuthology.orchestra.run.smithi071.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-119-generic amd64 5.15.0-119.129 [11.5 MB] 2024-09-11T06:36:10.244 INFO:teuthology.orchestra.run.smithi176.stdout:Fetched 113 MB in 2s (72.0 MB/s) 2024-09-11T06:36:10.375 INFO:teuthology.orchestra.run.smithi089.stdout:Fetched 113 MB in 2s (71.2 MB/s) 2024-09-11T06:36:10.432 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package linux-modules-5.15.0-119-generic. 2024-09-11T06:36:10.470 INFO:teuthology.orchestra.run.smithi071.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-119-generic amd64 5.15.0-119.129 [63.9 MB] 2024-09-11T06:36:10.554 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package linux-modules-5.15.0-119-generic. 2024-09-11T06:36:12.536 INFO:teuthology.orchestra.run.smithi176.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-11T06:36:12.540 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../0-linux-modules-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-11T06:36:12.597 INFO:teuthology.orchestra.run.smithi089.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-11T06:36:12.602 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../0-linux-modules-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-11T06:36:12.670 INFO:teuthology.orchestra.run.smithi071.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.119.119 [1,694 B] 2024-09-11T06:36:12.670 INFO:teuthology.orchestra.run.smithi071.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.119.119 [2,502 B] 2024-09-11T06:36:12.670 INFO:teuthology.orchestra.run.smithi071.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-119 all 5.15.0-119.129 [12.3 MB] 2024-09-11T06:36:12.760 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking linux-modules-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T06:36:12.787 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking linux-modules-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T06:36:13.025 INFO:teuthology.orchestra.run.smithi071.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-119-generic amd64 5.15.0-119.129 [2,879 kB] 2024-09-11T06:36:13.108 INFO:teuthology.orchestra.run.smithi071.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.119.119 [2,350 B] 2024-09-11T06:36:13.512 INFO:teuthology.orchestra.run.smithi071.stdout:Fetched 113 MB in 5s (24.8 MB/s) 2024-09-11T06:36:13.706 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package linux-modules-5.15.0-119-generic. 2024-09-11T06:36:14.804 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package linux-image-5.15.0-119-generic. 2024-09-11T06:36:14.825 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../1-linux-image-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-11T06:36:14.912 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package linux-image-5.15.0-119-generic. 2024-09-11T06:36:14.917 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T06:36:14.933 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../1-linux-image-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-11T06:36:15.016 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T06:36:15.390 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-119-generic. 2024-09-11T06:36:15.411 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-11T06:36:15.448 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking linux-modules-extra-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T06:36:15.515 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-119-generic. 2024-09-11T06:36:15.536 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-11T06:36:15.573 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking linux-modules-extra-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T06:36:15.732 INFO:teuthology.orchestra.run.smithi071.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-11T06:36:15.737 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../0-linux-modules-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-11T06:36:15.852 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking linux-modules-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T06:36:18.078 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package linux-image-5.15.0-119-generic. 2024-09-11T06:36:18.099 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../1-linux-image-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-11T06:36:18.228 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T06:36:18.673 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-119-generic. 2024-09-11T06:36:18.693 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-11T06:36:18.744 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking linux-modules-extra-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T06:36:22.114 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../3-linux-generic_5.15.0.119.119_amd64.deb ... 2024-09-11T06:36:22.222 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking linux-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-11T06:36:22.468 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../3-linux-generic_5.15.0.119.119_amd64.deb ... 2024-09-11T06:36:22.545 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.119.119_amd64.deb ... 2024-09-11T06:36:22.568 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking linux-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-11T06:36:22.667 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking linux-image-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-11T06:36:22.892 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.119.119_amd64.deb ... 2024-09-11T06:36:22.986 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package linux-headers-5.15.0-119. 2024-09-11T06:36:23.009 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../5-linux-headers-5.15.0-119_5.15.0-119.129_all.deb ... 2024-09-11T06:36:23.013 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking linux-image-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-11T06:36:23.044 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking linux-headers-5.15.0-119 (5.15.0-119.129) ... 2024-09-11T06:36:23.316 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package linux-headers-5.15.0-119. 2024-09-11T06:36:23.339 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../5-linux-headers-5.15.0-119_5.15.0-119.129_all.deb ... 2024-09-11T06:36:23.383 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking linux-headers-5.15.0-119 (5.15.0-119.129) ... 2024-09-11T06:36:25.813 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../3-linux-generic_5.15.0.119.119_amd64.deb ... 2024-09-11T06:36:25.924 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking linux-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-11T06:36:26.249 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.119.119_amd64.deb ... 2024-09-11T06:36:26.402 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking linux-image-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-11T06:36:26.722 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package linux-headers-5.15.0-119. 2024-09-11T06:36:26.744 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../5-linux-headers-5.15.0-119_5.15.0-119.129_all.deb ... 2024-09-11T06:36:26.788 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking linux-headers-5.15.0-119 (5.15.0-119.129) ... 2024-09-11T06:36:28.264 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package linux-headers-5.15.0-119-generic. 2024-09-11T06:36:28.301 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../6-linux-headers-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-11T06:36:28.338 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking linux-headers-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T06:36:29.123 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package linux-headers-5.15.0-119-generic. 2024-09-11T06:36:29.160 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../6-linux-headers-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-11T06:36:29.198 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking linux-headers-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T06:36:30.956 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.119.119_amd64.deb ... 2024-09-11T06:36:31.082 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking linux-headers-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-11T06:36:31.415 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up linux-headers-5.15.0-119 (5.15.0-119.129) ... 2024-09-11T06:36:31.545 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up linux-headers-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T06:36:31.671 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up linux-headers-generic (5.15.0.119.119) ... 2024-09-11T06:36:31.783 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T06:36:31.862 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.119.119_amd64.deb ... 2024-09-11T06:36:31.988 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking linux-headers-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-11T06:36:32.329 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up linux-headers-5.15.0-119 (5.15.0-119.129) ... 2024-09-11T06:36:32.459 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up linux-headers-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T06:36:32.577 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up linux-headers-generic (5.15.0.119.119) ... 2024-09-11T06:36:32.603 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package linux-headers-5.15.0-119-generic. 2024-09-11T06:36:32.640 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../6-linux-headers-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-11T06:36:32.677 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking linux-headers-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T06:36:32.681 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T06:36:33.127 INFO:teuthology.orchestra.run.smithi176.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-11T06:36:33.128 INFO:teuthology.orchestra.run.smithi176.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-11T06:36:33.128 INFO:teuthology.orchestra.run.smithi176.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-119-generic 2024-09-11T06:36:33.128 INFO:teuthology.orchestra.run.smithi176.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-119-generic 2024-09-11T06:36:33.284 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up linux-modules-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T06:36:33.993 INFO:teuthology.orchestra.run.smithi089.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-11T06:36:33.993 INFO:teuthology.orchestra.run.smithi089.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-11T06:36:33.993 INFO:teuthology.orchestra.run.smithi089.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-119-generic 2024-09-11T06:36:33.993 INFO:teuthology.orchestra.run.smithi089.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-119-generic 2024-09-11T06:36:34.184 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up linux-modules-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T06:36:34.677 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up linux-modules-extra-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T06:36:36.111 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up linux-modules-extra-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T06:36:36.412 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up linux-image-generic (5.15.0.119.119) ... 2024-09-11T06:36:36.529 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up linux-generic (5.15.0.119.119) ... 2024-09-11T06:36:36.631 INFO:teuthology.orchestra.run.smithi176.stdout:Processing triggers for linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T06:36:36.679 INFO:teuthology.orchestra.run.smithi176.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-11T06:36:36.680 INFO:teuthology.orchestra.run.smithi176.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-119-generic 2024-09-11T06:36:37.087 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.119.119_amd64.deb ... 2024-09-11T06:36:37.237 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking linux-headers-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-11T06:36:37.462 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up linux-image-generic (5.15.0.119.119) ... 2024-09-11T06:36:37.571 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up linux-generic (5.15.0.119.119) ... 2024-09-11T06:36:37.603 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up linux-headers-5.15.0-119 (5.15.0-119.129) ... 2024-09-11T06:36:37.698 INFO:teuthology.orchestra.run.smithi089.stdout:Processing triggers for linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T06:36:37.725 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up linux-headers-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T06:36:37.768 INFO:teuthology.orchestra.run.smithi089.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-11T06:36:37.768 INFO:teuthology.orchestra.run.smithi089.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-119-generic 2024-09-11T06:36:37.876 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up linux-headers-generic (5.15.0.119.119) ... 2024-09-11T06:36:38.014 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T06:36:39.358 INFO:teuthology.orchestra.run.smithi071.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-11T06:36:39.358 INFO:teuthology.orchestra.run.smithi071.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-11T06:36:39.358 INFO:teuthology.orchestra.run.smithi071.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-119-generic 2024-09-11T06:36:39.358 INFO:teuthology.orchestra.run.smithi071.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-119-generic 2024-09-11T06:36:39.548 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up linux-modules-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T06:36:40.991 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up linux-modules-extra-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T06:36:42.600 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up linux-image-generic (5.15.0.119.119) ... 2024-09-11T06:36:42.718 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up linux-generic (5.15.0.119.119) ... 2024-09-11T06:36:42.894 INFO:teuthology.orchestra.run.smithi071.stdout:Processing triggers for linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T06:36:42.959 INFO:teuthology.orchestra.run.smithi071.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-11T06:36:42.959 INFO:teuthology.orchestra.run.smithi071.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-119-generic 2024-09-11T06:36:51.068 INFO:teuthology.orchestra.run.smithi176.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-11T06:36:51.068 INFO:teuthology.orchestra.run.smithi176.stdout:Sourcing file `/etc/default/grub' 2024-09-11T06:36:51.081 INFO:teuthology.orchestra.run.smithi176.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-11T06:36:51.098 INFO:teuthology.orchestra.run.smithi176.stdout:Generating grub configuration file ... 2024-09-11T06:36:51.361 INFO:teuthology.orchestra.run.smithi176.stdout:Found linux image: /boot/vmlinuz-5.15.0-119-generic 2024-09-11T06:36:51.383 INFO:teuthology.orchestra.run.smithi176.stdout:Found initrd image: /boot/initrd.img-5.15.0-119-generic 2024-09-11T06:36:51.669 INFO:teuthology.orchestra.run.smithi176.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-11T06:36:51.674 INFO:teuthology.orchestra.run.smithi176.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-11T06:36:51.740 INFO:teuthology.orchestra.run.smithi176.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-11T06:36:51.745 INFO:teuthology.orchestra.run.smithi176.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-11T06:36:51.904 INFO:teuthology.orchestra.run.smithi176.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-11T06:36:51.904 INFO:teuthology.orchestra.run.smithi176.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-11T06:36:51.904 INFO:teuthology.orchestra.run.smithi176.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-11T06:36:51.952 INFO:teuthology.orchestra.run.smithi176.stdout:done 2024-09-11T06:36:52.385 INFO:teuthology.orchestra.run.smithi089.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-11T06:36:52.385 INFO:teuthology.orchestra.run.smithi089.stdout:Sourcing file `/etc/default/grub' 2024-09-11T06:36:52.401 INFO:teuthology.orchestra.run.smithi089.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-11T06:36:52.430 DEBUG:teuthology.orchestra.run.smithi176:> dpkg -s linux-image-generic 2024-09-11T06:36:52.431 INFO:teuthology.orchestra.run.smithi089.stdout:Generating grub configuration file ... 2024-09-11T06:36:52.455 INFO:teuthology.orchestra.run.smithi176.stdout:Package: linux-image-generic 2024-09-11T06:36:52.456 INFO:teuthology.orchestra.run.smithi176.stdout:Status: install ok installed 2024-09-11T06:36:52.456 INFO:teuthology.orchestra.run.smithi176.stdout:Priority: optional 2024-09-11T06:36:52.456 INFO:teuthology.orchestra.run.smithi176.stdout:Section: kernel 2024-09-11T06:36:52.456 INFO:teuthology.orchestra.run.smithi176.stdout:Installed-Size: 21 2024-09-11T06:36:52.456 INFO:teuthology.orchestra.run.smithi176.stdout:Maintainer: Ubuntu Kernel Team 2024-09-11T06:36:52.456 INFO:teuthology.orchestra.run.smithi176.stdout:Architecture: amd64 2024-09-11T06:36:52.456 INFO:teuthology.orchestra.run.smithi176.stdout:Source: linux-meta 2024-09-11T06:36:52.456 INFO:teuthology.orchestra.run.smithi176.stdout:Version: 5.15.0.119.119 2024-09-11T06:36:52.456 INFO:teuthology.orchestra.run.smithi176.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-119), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-11T06:36:52.457 INFO:teuthology.orchestra.run.smithi176.stdout:Depends: linux-image-5.15.0-119-generic, linux-modules-extra-5.15.0-119-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-11T06:36:52.457 INFO:teuthology.orchestra.run.smithi176.stdout:Recommends: thermald 2024-09-11T06:36:52.457 INFO:teuthology.orchestra.run.smithi176.stdout:Description: Generic Linux kernel image 2024-09-11T06:36:52.457 INFO:teuthology.orchestra.run.smithi176.stdout: This package will always depend on the latest generic kernel image 2024-09-11T06:36:52.457 INFO:teuthology.orchestra.run.smithi176.stdout: available. 2024-09-11T06:36:52.457 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-119-generic 2024-09-11T06:36:52.457 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T06:36:52.457 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T06:36:52.458 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-11T06:36:52.458 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi176.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T06:36:52.458 DEBUG:teuthology.orchestra.run.smithi176:> sudo apt-get clean 2024-09-11T06:36:52.639 DEBUG:teuthology.orchestra.run.smithi176:> sudo apt-get update 2024-09-11T06:36:52.696 INFO:teuthology.orchestra.run.smithi089.stdout:Found linux image: /boot/vmlinuz-5.15.0-119-generic 2024-09-11T06:36:52.716 INFO:teuthology.orchestra.run.smithi089.stdout:Found initrd image: /boot/initrd.img-5.15.0-119-generic 2024-09-11T06:36:52.761 INFO:teuthology.orchestra.run.smithi176.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-11T06:36:52.765 INFO:teuthology.orchestra.run.smithi176.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-11T06:36:52.782 INFO:teuthology.orchestra.run.smithi176.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-11T06:36:52.877 INFO:teuthology.orchestra.run.smithi176.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-11T06:36:53.018 INFO:teuthology.orchestra.run.smithi089.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-11T06:36:53.023 INFO:teuthology.orchestra.run.smithi089.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-11T06:36:53.094 INFO:teuthology.orchestra.run.smithi089.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-11T06:36:53.099 INFO:teuthology.orchestra.run.smithi089.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-11T06:36:53.256 INFO:teuthology.orchestra.run.smithi089.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-11T06:36:53.256 INFO:teuthology.orchestra.run.smithi089.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-11T06:36:53.256 INFO:teuthology.orchestra.run.smithi089.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-11T06:36:53.297 INFO:teuthology.orchestra.run.smithi089.stdout:done 2024-09-11T06:36:53.810 DEBUG:teuthology.orchestra.run.smithi089:> dpkg -s linux-image-generic 2024-09-11T06:36:53.832 INFO:teuthology.orchestra.run.smithi089.stdout:Package: linux-image-generic 2024-09-11T06:36:53.832 INFO:teuthology.orchestra.run.smithi089.stdout:Status: install ok installed 2024-09-11T06:36:53.832 INFO:teuthology.orchestra.run.smithi089.stdout:Priority: optional 2024-09-11T06:36:53.832 INFO:teuthology.orchestra.run.smithi089.stdout:Section: kernel 2024-09-11T06:36:53.832 INFO:teuthology.orchestra.run.smithi089.stdout:Installed-Size: 21 2024-09-11T06:36:53.832 INFO:teuthology.orchestra.run.smithi089.stdout:Maintainer: Ubuntu Kernel Team 2024-09-11T06:36:53.833 INFO:teuthology.orchestra.run.smithi089.stdout:Architecture: amd64 2024-09-11T06:36:53.833 INFO:teuthology.orchestra.run.smithi089.stdout:Source: linux-meta 2024-09-11T06:36:53.833 INFO:teuthology.orchestra.run.smithi089.stdout:Version: 5.15.0.119.119 2024-09-11T06:36:53.833 INFO:teuthology.orchestra.run.smithi089.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-119), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-11T06:36:53.833 INFO:teuthology.orchestra.run.smithi089.stdout:Depends: linux-image-5.15.0-119-generic, linux-modules-extra-5.15.0-119-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-11T06:36:53.833 INFO:teuthology.orchestra.run.smithi089.stdout:Recommends: thermald 2024-09-11T06:36:53.833 INFO:teuthology.orchestra.run.smithi089.stdout:Description: Generic Linux kernel image 2024-09-11T06:36:53.833 INFO:teuthology.orchestra.run.smithi089.stdout: This package will always depend on the latest generic kernel image 2024-09-11T06:36:53.833 INFO:teuthology.orchestra.run.smithi089.stdout: available. 2024-09-11T06:36:53.833 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-119-generic 2024-09-11T06:36:53.833 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T06:36:53.833 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T06:36:53.834 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-11T06:36:53.834 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi089.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T06:36:53.834 DEBUG:teuthology.orchestra.run.smithi089:> sudo apt-get clean 2024-09-11T06:36:53.965 DEBUG:teuthology.orchestra.run.smithi089:> sudo apt-get update 2024-09-11T06:36:54.103 INFO:teuthology.orchestra.run.smithi176.stdout:Reading package lists... 2024-09-11T06:36:54.121 DEBUG:teuthology.orchestra.run.smithi176:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-11T06:36:54.174 INFO:teuthology.orchestra.run.smithi176.stdout:Reading package lists... 2024-09-11T06:36:54.211 INFO:teuthology.orchestra.run.smithi089.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-11T06:36:54.211 INFO:teuthology.orchestra.run.smithi089.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-11T06:36:54.276 INFO:teuthology.orchestra.run.smithi089.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-11T06:36:54.357 INFO:teuthology.orchestra.run.smithi089.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-11T06:36:54.359 INFO:teuthology.orchestra.run.smithi176.stdout:Building dependency tree... 2024-09-11T06:36:54.360 INFO:teuthology.orchestra.run.smithi176.stdout:Reading state information... 2024-09-11T06:36:54.524 INFO:teuthology.orchestra.run.smithi176.stdout:linux-image-generic is already the newest version (5.15.0.119.119). 2024-09-11T06:36:54.524 INFO:teuthology.orchestra.run.smithi176.stdout:The following packages were automatically installed and are no longer required: 2024-09-11T06:36:54.524 INFO:teuthology.orchestra.run.smithi176.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-11T06:36:54.525 INFO:teuthology.orchestra.run.smithi176.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-11T06:36:54.525 INFO:teuthology.orchestra.run.smithi176.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-11T06:36:54.525 INFO:teuthology.orchestra.run.smithi176.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-11T06:36:54.525 INFO:teuthology.orchestra.run.smithi176.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-11T06:36:54.525 INFO:teuthology.orchestra.run.smithi176.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-11T06:36:54.566 INFO:teuthology.orchestra.run.smithi176.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-11T06:36:54.567 DEBUG:teuthology.orchestra.run.smithi176:> dpkg -s linux-image-generic 2024-09-11T06:36:54.629 INFO:teuthology.orchestra.run.smithi176.stdout:Package: linux-image-generic 2024-09-11T06:36:54.629 INFO:teuthology.orchestra.run.smithi176.stdout:Status: install ok installed 2024-09-11T06:36:54.629 INFO:teuthology.orchestra.run.smithi176.stdout:Priority: optional 2024-09-11T06:36:54.629 INFO:teuthology.orchestra.run.smithi176.stdout:Section: kernel 2024-09-11T06:36:54.629 INFO:teuthology.orchestra.run.smithi176.stdout:Installed-Size: 21 2024-09-11T06:36:54.629 INFO:teuthology.orchestra.run.smithi176.stdout:Maintainer: Ubuntu Kernel Team 2024-09-11T06:36:54.629 INFO:teuthology.orchestra.run.smithi176.stdout:Architecture: amd64 2024-09-11T06:36:54.629 INFO:teuthology.orchestra.run.smithi176.stdout:Source: linux-meta 2024-09-11T06:36:54.629 INFO:teuthology.orchestra.run.smithi176.stdout:Version: 5.15.0.119.119 2024-09-11T06:36:54.630 INFO:teuthology.orchestra.run.smithi176.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-119), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-11T06:36:54.630 INFO:teuthology.orchestra.run.smithi176.stdout:Depends: linux-image-5.15.0-119-generic, linux-modules-extra-5.15.0-119-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-11T06:36:54.630 INFO:teuthology.orchestra.run.smithi176.stdout:Recommends: thermald 2024-09-11T06:36:54.630 INFO:teuthology.orchestra.run.smithi176.stdout:Description: Generic Linux kernel image 2024-09-11T06:36:54.630 INFO:teuthology.orchestra.run.smithi176.stdout: This package will always depend on the latest generic kernel image 2024-09-11T06:36:54.630 INFO:teuthology.orchestra.run.smithi176.stdout: available. 2024-09-11T06:36:54.630 DEBUG:teuthology.orchestra.run.smithi176:> mktemp 2024-09-11T06:36:54.677 INFO:teuthology.orchestra.run.smithi176.stdout:/tmp/tmp.HRLGrQQhHf 2024-09-11T06:36:54.677 DEBUG:teuthology.orchestra.run.smithi176:> sudo cp /boot/grub/grub.cfg /tmp/tmp.HRLGrQQhHf 2024-09-11T06:36:54.733 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod 0666 /tmp/tmp.HRLGrQQhHf 2024-09-11T06:36:54.864 DEBUG:teuthology.orchestra.remote:smithi176:/tmp/tmp.HRLGrQQhHf is 10KB 2024-09-11T06:36:54.877 DEBUG:teuthology.orchestra.run.smithi176:> rm -fr /tmp/tmp.HRLGrQQhHf 2024-09-11T06:36:54.883 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-11T06:36:54.937 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-11T06:36:54.937 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-11T06:36:54.938 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-11T06:36:55.004 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-119-generic 2024-09-11T06:36:55.005 DEBUG:teuthology.orchestra.run.smithi176:> sudo update-grub 2024-09-11T06:36:55.683 INFO:teuthology.orchestra.run.smithi089.stdout:Reading package lists... 2024-09-11T06:36:55.701 DEBUG:teuthology.orchestra.run.smithi089:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-11T06:36:55.763 INFO:teuthology.orchestra.run.smithi089.stdout:Reading package lists... 2024-09-11T06:36:55.968 INFO:teuthology.orchestra.run.smithi089.stdout:Building dependency tree... 2024-09-11T06:36:55.968 INFO:teuthology.orchestra.run.smithi089.stdout:Reading state information... 2024-09-11T06:36:56.157 INFO:teuthology.orchestra.run.smithi089.stdout:linux-image-generic is already the newest version (5.15.0.119.119). 2024-09-11T06:36:56.157 INFO:teuthology.orchestra.run.smithi089.stdout:The following packages were automatically installed and are no longer required: 2024-09-11T06:36:56.157 INFO:teuthology.orchestra.run.smithi089.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-11T06:36:56.157 INFO:teuthology.orchestra.run.smithi089.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-11T06:36:56.157 INFO:teuthology.orchestra.run.smithi089.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-11T06:36:56.158 INFO:teuthology.orchestra.run.smithi089.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-11T06:36:56.158 INFO:teuthology.orchestra.run.smithi089.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-11T06:36:56.158 INFO:teuthology.orchestra.run.smithi089.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-11T06:36:56.162 INFO:teuthology.orchestra.run.smithi176.stderr:Sourcing file `/etc/default/grub' 2024-09-11T06:36:56.163 INFO:teuthology.orchestra.run.smithi176.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-11T06:36:56.165 INFO:teuthology.orchestra.run.smithi176.stderr:Generating grub configuration file ... 2024-09-11T06:36:56.200 INFO:teuthology.orchestra.run.smithi089.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-11T06:36:56.202 DEBUG:teuthology.orchestra.run.smithi089:> dpkg -s linux-image-generic 2024-09-11T06:36:56.217 INFO:teuthology.orchestra.run.smithi089.stdout:Package: linux-image-generic 2024-09-11T06:36:56.218 INFO:teuthology.orchestra.run.smithi089.stdout:Status: install ok installed 2024-09-11T06:36:56.218 INFO:teuthology.orchestra.run.smithi089.stdout:Priority: optional 2024-09-11T06:36:56.218 INFO:teuthology.orchestra.run.smithi089.stdout:Section: kernel 2024-09-11T06:36:56.218 INFO:teuthology.orchestra.run.smithi089.stdout:Installed-Size: 21 2024-09-11T06:36:56.218 INFO:teuthology.orchestra.run.smithi089.stdout:Maintainer: Ubuntu Kernel Team 2024-09-11T06:36:56.218 INFO:teuthology.orchestra.run.smithi089.stdout:Architecture: amd64 2024-09-11T06:36:56.218 INFO:teuthology.orchestra.run.smithi089.stdout:Source: linux-meta 2024-09-11T06:36:56.218 INFO:teuthology.orchestra.run.smithi089.stdout:Version: 5.15.0.119.119 2024-09-11T06:36:56.218 INFO:teuthology.orchestra.run.smithi089.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-119), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-11T06:36:56.218 INFO:teuthology.orchestra.run.smithi089.stdout:Depends: linux-image-5.15.0-119-generic, linux-modules-extra-5.15.0-119-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-11T06:36:56.218 INFO:teuthology.orchestra.run.smithi089.stdout:Recommends: thermald 2024-09-11T06:36:56.219 INFO:teuthology.orchestra.run.smithi089.stdout:Description: Generic Linux kernel image 2024-09-11T06:36:56.219 INFO:teuthology.orchestra.run.smithi089.stdout: This package will always depend on the latest generic kernel image 2024-09-11T06:36:56.219 INFO:teuthology.orchestra.run.smithi089.stdout: available. 2024-09-11T06:36:56.219 DEBUG:teuthology.orchestra.run.smithi089:> mktemp 2024-09-11T06:36:56.263 INFO:teuthology.orchestra.run.smithi089.stdout:/tmp/tmp.6a7eA34xFw 2024-09-11T06:36:56.263 DEBUG:teuthology.orchestra.run.smithi089:> sudo cp /boot/grub/grub.cfg /tmp/tmp.6a7eA34xFw 2024-09-11T06:36:56.319 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0666 /tmp/tmp.6a7eA34xFw 2024-09-11T06:36:56.332 INFO:teuthology.orchestra.run.smithi176.stderr:Found linux image: /boot/vmlinuz-5.15.0-119-generic 2024-09-11T06:36:56.339 INFO:teuthology.orchestra.run.smithi176.stderr:Found initrd image: /boot/initrd.img-5.15.0-119-generic 2024-09-11T06:36:56.447 DEBUG:teuthology.orchestra.remote:smithi089:/tmp/tmp.6a7eA34xFw is 10KB 2024-09-11T06:36:56.459 DEBUG:teuthology.orchestra.run.smithi089:> rm -fr /tmp/tmp.6a7eA34xFw 2024-09-11T06:36:56.465 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-11T06:36:56.519 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-11T06:36:56.519 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-11T06:36:56.519 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-11T06:36:56.551 INFO:teuthology.orchestra.run.smithi176.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-11T06:36:56.556 INFO:teuthology.orchestra.run.smithi176.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-11T06:36:56.595 INFO:teuthology.orchestra.run.smithi176.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-11T06:36:56.600 INFO:teuthology.orchestra.run.smithi176.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-11T06:36:56.601 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-119-generic 2024-09-11T06:36:56.601 DEBUG:teuthology.orchestra.run.smithi089:> sudo update-grub 2024-09-11T06:36:56.705 INFO:teuthology.orchestra.run.smithi176.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-11T06:36:56.706 INFO:teuthology.orchestra.run.smithi176.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-11T06:36:56.706 INFO:teuthology.orchestra.run.smithi176.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-11T06:36:56.725 INFO:teuthology.orchestra.run.smithi176.stderr:done 2024-09-11T06:36:56.734 DEBUG:teuthology.orchestra.run.smithi176:> sudo shutdown -r now 2024-09-11T06:36:57.329 INFO:teuthology.orchestra.run.smithi071.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-11T06:36:57.329 INFO:teuthology.orchestra.run.smithi071.stdout:Sourcing file `/etc/default/grub' 2024-09-11T06:36:57.364 INFO:teuthology.orchestra.run.smithi071.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-11T06:36:57.391 INFO:teuthology.orchestra.run.smithi071.stdout:Generating grub configuration file ... 2024-09-11T06:36:57.673 INFO:teuthology.orchestra.run.smithi071.stdout:Found linux image: /boot/vmlinuz-5.15.0-119-generic 2024-09-11T06:36:57.695 INFO:teuthology.orchestra.run.smithi071.stdout:Found initrd image: /boot/initrd.img-5.15.0-119-generic 2024-09-11T06:36:57.806 INFO:teuthology.orchestra.run.smithi089.stderr:Sourcing file `/etc/default/grub' 2024-09-11T06:36:57.807 INFO:teuthology.orchestra.run.smithi089.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-11T06:36:57.810 INFO:teuthology.orchestra.run.smithi089.stderr:Generating grub configuration file ... 2024-09-11T06:36:57.994 INFO:teuthology.orchestra.run.smithi089.stderr:Found linux image: /boot/vmlinuz-5.15.0-119-generic 2024-09-11T06:36:57.997 INFO:teuthology.orchestra.run.smithi071.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-11T06:36:58.000 INFO:teuthology.orchestra.run.smithi089.stderr:Found initrd image: /boot/initrd.img-5.15.0-119-generic 2024-09-11T06:36:58.002 INFO:teuthology.orchestra.run.smithi071.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-11T06:36:58.071 INFO:teuthology.orchestra.run.smithi071.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-11T06:36:58.076 INFO:teuthology.orchestra.run.smithi071.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-11T06:36:58.223 INFO:teuthology.orchestra.run.smithi089.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-11T06:36:58.228 INFO:teuthology.orchestra.run.smithi089.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-11T06:36:58.258 INFO:teuthology.orchestra.run.smithi071.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-11T06:36:58.258 INFO:teuthology.orchestra.run.smithi071.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-11T06:36:58.258 INFO:teuthology.orchestra.run.smithi071.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-11T06:36:58.269 INFO:teuthology.orchestra.run.smithi089.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-11T06:36:58.274 INFO:teuthology.orchestra.run.smithi089.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-11T06:36:58.294 INFO:teuthology.orchestra.run.smithi071.stdout:done 2024-09-11T06:36:58.403 INFO:teuthology.orchestra.run.smithi089.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-11T06:36:58.403 INFO:teuthology.orchestra.run.smithi089.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-11T06:36:58.403 INFO:teuthology.orchestra.run.smithi089.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-11T06:36:58.416 INFO:teuthology.orchestra.run.smithi089.stderr:done 2024-09-11T06:36:58.425 DEBUG:teuthology.orchestra.run.smithi089:> sudo shutdown -r now 2024-09-11T06:36:58.802 DEBUG:teuthology.orchestra.run.smithi071:> dpkg -s linux-image-generic 2024-09-11T06:36:58.835 INFO:teuthology.orchestra.run.smithi071.stdout:Package: linux-image-generic 2024-09-11T06:36:58.835 INFO:teuthology.orchestra.run.smithi071.stdout:Status: install ok installed 2024-09-11T06:36:58.835 INFO:teuthology.orchestra.run.smithi071.stdout:Priority: optional 2024-09-11T06:36:58.835 INFO:teuthology.orchestra.run.smithi071.stdout:Section: kernel 2024-09-11T06:36:58.835 INFO:teuthology.orchestra.run.smithi071.stdout:Installed-Size: 21 2024-09-11T06:36:58.835 INFO:teuthology.orchestra.run.smithi071.stdout:Maintainer: Ubuntu Kernel Team 2024-09-11T06:36:58.835 INFO:teuthology.orchestra.run.smithi071.stdout:Architecture: amd64 2024-09-11T06:36:58.836 INFO:teuthology.orchestra.run.smithi071.stdout:Source: linux-meta 2024-09-11T06:36:58.836 INFO:teuthology.orchestra.run.smithi071.stdout:Version: 5.15.0.119.119 2024-09-11T06:36:58.836 INFO:teuthology.orchestra.run.smithi071.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-119), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-11T06:36:58.836 INFO:teuthology.orchestra.run.smithi071.stdout:Depends: linux-image-5.15.0-119-generic, linux-modules-extra-5.15.0-119-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-11T06:36:58.836 INFO:teuthology.orchestra.run.smithi071.stdout:Recommends: thermald 2024-09-11T06:36:58.836 INFO:teuthology.orchestra.run.smithi071.stdout:Description: Generic Linux kernel image 2024-09-11T06:36:58.836 INFO:teuthology.orchestra.run.smithi071.stdout: This package will always depend on the latest generic kernel image 2024-09-11T06:36:58.836 INFO:teuthology.orchestra.run.smithi071.stdout: available. 2024-09-11T06:36:58.836 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-119-generic 2024-09-11T06:36:58.837 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T06:36:58.837 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T06:36:58.837 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-11T06:36:58.837 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi071.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T06:36:58.837 DEBUG:teuthology.orchestra.run.smithi071:> sudo apt-get clean 2024-09-11T06:36:59.081 DEBUG:teuthology.orchestra.run.smithi071:> sudo apt-get update 2024-09-11T06:36:59.201 INFO:teuthology.orchestra.run.smithi071.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-11T06:36:59.202 INFO:teuthology.orchestra.run.smithi071.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-11T06:36:59.220 INFO:teuthology.orchestra.run.smithi071.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-11T06:36:59.237 INFO:teuthology.orchestra.run.smithi071.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-11T06:37:00.616 INFO:teuthology.orchestra.run.smithi071.stdout:Reading package lists... 2024-09-11T06:37:00.633 DEBUG:teuthology.orchestra.run.smithi071:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-11T06:37:00.685 INFO:teuthology.orchestra.run.smithi071.stdout:Reading package lists... 2024-09-11T06:37:00.873 INFO:teuthology.orchestra.run.smithi071.stdout:Building dependency tree... 2024-09-11T06:37:00.874 INFO:teuthology.orchestra.run.smithi071.stdout:Reading state information... 2024-09-11T06:37:01.038 INFO:teuthology.orchestra.run.smithi071.stdout:linux-image-generic is already the newest version (5.15.0.119.119). 2024-09-11T06:37:01.039 INFO:teuthology.orchestra.run.smithi071.stdout:The following packages were automatically installed and are no longer required: 2024-09-11T06:37:01.039 INFO:teuthology.orchestra.run.smithi071.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-11T06:37:01.039 INFO:teuthology.orchestra.run.smithi071.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-11T06:37:01.039 INFO:teuthology.orchestra.run.smithi071.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-11T06:37:01.039 INFO:teuthology.orchestra.run.smithi071.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-11T06:37:01.039 INFO:teuthology.orchestra.run.smithi071.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-11T06:37:01.039 INFO:teuthology.orchestra.run.smithi071.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-11T06:37:01.079 INFO:teuthology.orchestra.run.smithi071.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-11T06:37:01.081 DEBUG:teuthology.orchestra.run.smithi071:> dpkg -s linux-image-generic 2024-09-11T06:37:01.140 INFO:teuthology.orchestra.run.smithi071.stdout:Package: linux-image-generic 2024-09-11T06:37:01.140 INFO:teuthology.orchestra.run.smithi071.stdout:Status: install ok installed 2024-09-11T06:37:01.140 INFO:teuthology.orchestra.run.smithi071.stdout:Priority: optional 2024-09-11T06:37:01.140 INFO:teuthology.orchestra.run.smithi071.stdout:Section: kernel 2024-09-11T06:37:01.140 INFO:teuthology.orchestra.run.smithi071.stdout:Installed-Size: 21 2024-09-11T06:37:01.140 INFO:teuthology.orchestra.run.smithi071.stdout:Maintainer: Ubuntu Kernel Team 2024-09-11T06:37:01.140 INFO:teuthology.orchestra.run.smithi071.stdout:Architecture: amd64 2024-09-11T06:37:01.140 INFO:teuthology.orchestra.run.smithi071.stdout:Source: linux-meta 2024-09-11T06:37:01.140 INFO:teuthology.orchestra.run.smithi071.stdout:Version: 5.15.0.119.119 2024-09-11T06:37:01.140 INFO:teuthology.orchestra.run.smithi071.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-119), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-11T06:37:01.141 INFO:teuthology.orchestra.run.smithi071.stdout:Depends: linux-image-5.15.0-119-generic, linux-modules-extra-5.15.0-119-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-11T06:37:01.141 INFO:teuthology.orchestra.run.smithi071.stdout:Recommends: thermald 2024-09-11T06:37:01.141 INFO:teuthology.orchestra.run.smithi071.stdout:Description: Generic Linux kernel image 2024-09-11T06:37:01.141 INFO:teuthology.orchestra.run.smithi071.stdout: This package will always depend on the latest generic kernel image 2024-09-11T06:37:01.141 INFO:teuthology.orchestra.run.smithi071.stdout: available. 2024-09-11T06:37:01.141 DEBUG:teuthology.orchestra.run.smithi071:> mktemp 2024-09-11T06:37:01.189 INFO:teuthology.orchestra.run.smithi071.stdout:/tmp/tmp.TTfnBOmAJU 2024-09-11T06:37:01.190 DEBUG:teuthology.orchestra.run.smithi071:> sudo cp /boot/grub/grub.cfg /tmp/tmp.TTfnBOmAJU 2024-09-11T06:37:01.257 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0666 /tmp/tmp.TTfnBOmAJU 2024-09-11T06:37:01.365 DEBUG:teuthology.orchestra.remote:smithi071:/tmp/tmp.TTfnBOmAJU is 10KB 2024-09-11T06:37:01.379 DEBUG:teuthology.orchestra.run.smithi071:> rm -fr /tmp/tmp.TTfnBOmAJU 2024-09-11T06:37:01.384 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-11T06:37:01.438 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T06:37:01.438 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-11T06:37:01.438 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-11T06:37:01.514 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-119-generic 2024-09-11T06:37:01.515 DEBUG:teuthology.orchestra.run.smithi071:> sudo update-grub 2024-09-11T06:37:02.913 INFO:teuthology.orchestra.run.smithi071.stderr:Sourcing file `/etc/default/grub' 2024-09-11T06:37:02.914 INFO:teuthology.orchestra.run.smithi071.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-11T06:37:02.918 INFO:teuthology.orchestra.run.smithi071.stderr:Generating grub configuration file ... 2024-09-11T06:37:03.094 INFO:teuthology.orchestra.run.smithi071.stderr:Found linux image: /boot/vmlinuz-5.15.0-119-generic 2024-09-11T06:37:03.100 INFO:teuthology.orchestra.run.smithi071.stderr:Found initrd image: /boot/initrd.img-5.15.0-119-generic 2024-09-11T06:37:03.351 INFO:teuthology.orchestra.run.smithi071.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-11T06:37:03.356 INFO:teuthology.orchestra.run.smithi071.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-11T06:37:03.395 INFO:teuthology.orchestra.run.smithi071.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-11T06:37:03.399 INFO:teuthology.orchestra.run.smithi071.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-11T06:37:03.559 INFO:teuthology.orchestra.run.smithi071.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-11T06:37:03.559 INFO:teuthology.orchestra.run.smithi071.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-11T06:37:03.559 INFO:teuthology.orchestra.run.smithi071.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-11T06:37:03.570 INFO:teuthology.orchestra.run.smithi071.stderr:done 2024-09-11T06:37:03.573 DEBUG:teuthology.orchestra.run.smithi071:> sudo shutdown -r now 2024-09-11T06:37:26.757 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-11T06:37:26.757 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-09-11T06:37:26.758 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T06:37:28.429 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-11T06:37:28.429 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-09-11T06:37:28.429 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T06:37:33.580 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-11T06:37:33.580 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-11T06:37:33.581 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T06:37:45.149 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.176 2024-09-11T06:37:46.941 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.89 2024-09-11T06:37:52.065 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.71 2024-09-11T06:37:54.153 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-09-11T06:37:54.154 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T06:37:55.944 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-09-11T06:37:55.945 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T06:37:57.213 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.176 2024-09-11T06:37:59.009 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.89 2024-09-11T06:38:01.068 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-11T06:38:01.069 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T06:38:04.129 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.71 2024-09-11T06:38:09.219 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-09-11T06:38:09.220 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T06:38:11.011 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-09-11T06:38:11.012 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T06:38:16.133 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-11T06:38:16.134 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T06:38:27.649 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.176 2024-09-11T06:38:29.437 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.89 2024-09-11T06:38:34.557 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.71 2024-09-11T06:38:42.657 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-09-11T06:38:42.658 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T06:38:42.954 DEBUG:teuthology.orchestra.run.smithi176:> true 2024-09-11T06:38:43.584 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-09-11T06:38:43.585 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.15.0-119-generic"... 2024-09-11T06:38:43.585 DEBUG:teuthology.orchestra.run.smithi176:> uname -r 2024-09-11T06:38:43.630 INFO:teuthology.orchestra.run.smithi176.stdout:5.15.0-119-generic 2024-09-11T06:38:43.630 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-119-generic vs 5.15.0-119-generic 2024-09-11T06:38:43.631 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T06:38:43.631 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-11T06:38:44.437 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-09-11T06:38:44.438 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T06:38:44.631 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2024-09-11T06:38:44.631 DEBUG:teuthology.orchestra.run.smithi176:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T06:38:44.695 INFO:teuthology.orchestra.run.smithi176.stdout:ttyS1 2024-09-11T06:38:44.703 DEBUG:teuthology.orchestra.run.smithi089:> true 2024-09-11T06:38:44.737 DEBUG:teuthology.parallel:result is None 2024-09-11T06:38:45.755 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-09-11T06:38:45.755 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-119-generic"... 2024-09-11T06:38:45.756 DEBUG:teuthology.orchestra.run.smithi089:> uname -r 2024-09-11T06:38:45.803 INFO:teuthology.orchestra.run.smithi089.stdout:5.15.0-119-generic 2024-09-11T06:38:45.803 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-119-generic vs 5.15.0-119-generic 2024-09-11T06:38:45.803 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T06:38:45.804 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-11T06:38:46.805 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-11T06:38:46.805 DEBUG:teuthology.orchestra.run.smithi089:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T06:38:46.876 INFO:teuthology.orchestra.run.smithi089.stdout:ttyS1 2024-09-11T06:38:46.907 DEBUG:teuthology.parallel:result is None 2024-09-11T06:38:49.558 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-11T06:38:49.559 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T06:38:50.250 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-09-11T06:38:51.257 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-11T06:38:51.258 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-119-generic"... 2024-09-11T06:38:51.258 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2024-09-11T06:38:51.303 INFO:teuthology.orchestra.run.smithi071.stdout:5.15.0-119-generic 2024-09-11T06:38:51.303 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-119-generic vs 5.15.0-119-generic 2024-09-11T06:38:51.303 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T06:38:51.303 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-11T06:38:52.304 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-11T06:38:52.304 DEBUG:teuthology.orchestra.run.smithi071:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T06:38:52.644 INFO:teuthology.orchestra.run.smithi071.stdout:ttyS1 2024-09-11T06:38:52.669 DEBUG:teuthology.parallel:result is None 2024-09-11T06:38:52.669 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-11T06:38:52.676 INFO:teuthology.task.internal:Creating test directory... 2024-09-11T06:38:52.676 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T06:38:52.679 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T06:38:52.681 DEBUG:teuthology.orchestra.run.smithi176:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T06:38:52.687 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-11T06:38:52.695 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-11T06:38:52.712 INFO:teuthology.task.internal:Creating archive directory... 2024-09-11T06:38:52.712 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T06:38:52.728 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T06:38:52.734 DEBUG:teuthology.orchestra.run.smithi176:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T06:38:52.862 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-11T06:38:52.871 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-11T06:38:52.871 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T06:38:52.874 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T06:38:52.876 DEBUG:teuthology.orchestra.run.smithi176:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T06:38:52.901 INFO:teuthology.orchestra.run.smithi176.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T06:38:52.903 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T06:38:52.911 INFO:teuthology.orchestra.run.smithi176.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T06:38:52.911 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T06:38:52.922 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T06:38:52.931 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T06:38:52.933 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-11T06:38:52.944 INFO:teuthology.task.internal:Configuring sudo... 2024-09-11T06:38:52.945 DEBUG:teuthology.orchestra.run.smithi071:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T06:38:52.947 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T06:38:52.959 DEBUG:teuthology.orchestra.run.smithi176:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T06:38:52.972 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-11T06:38:52.982 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-11T06:38:52.982 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T06:38:53.000 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T06:38:53.015 DEBUG:teuthology.orchestra.run.smithi176:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T06:38:53.020 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T06:38:53.051 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T06:38:53.099 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T06:38:53.099 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T06:38:53.174 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T06:38:53.179 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T06:38:53.226 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-11T06:38:53.226 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T06:38:53.290 DEBUG:teuthology.orchestra.run.smithi176:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T06:38:53.296 DEBUG:teuthology.orchestra.run.smithi176:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T06:38:53.345 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-11T06:38:53.346 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T06:38:53.427 DEBUG:teuthology.orchestra.run.smithi071:> sudo service rsyslog restart 2024-09-11T06:38:53.429 DEBUG:teuthology.orchestra.run.smithi089:> sudo service rsyslog restart 2024-09-11T06:38:53.432 DEBUG:teuthology.orchestra.run.smithi176:> sudo service rsyslog restart 2024-09-11T06:38:53.516 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-11T06:38:53.524 INFO:teuthology.task.internal:Starting timer... 2024-09-11T06:38:53.524 INFO:teuthology.run_tasks:Running task pcp... 2024-09-11T06:38:53.537 INFO:teuthology.run_tasks:Running task selinux... 2024-09-11T06:38:53.550 DEBUG:teuthology.task.selinux:Excluding smithi071: OS 'ubuntu' does not support SELinux 2024-09-11T06:38:53.550 DEBUG:teuthology.task.selinux:Excluding smithi089: OS 'ubuntu' does not support SELinux 2024-09-11T06:38:53.550 DEBUG:teuthology.task.selinux:Excluding smithi176: OS 'ubuntu' does not support SELinux 2024-09-11T06:38:53.550 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-11T06:38:53.550 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-11T06:38:53.550 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-11T06:38:53.550 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-11T06:38:53.559 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-11T06:38:53.562 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-11T06:38:53.687 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-11T06:38:53.699 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-11T06:38:53.700 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi071.front.sepia.ceph.com,smithi089.front.sepia.ceph.com,smithi176.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-11T06:46:20.978 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi071.front.sepia.ceph.com'), Remote(name='ubuntu@smithi089.front.sepia.ceph.com'), Remote(name='ubuntu@smithi176.front.sepia.ceph.com')] 2024-09-11T06:46:20.979 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-11T06:46:20.981 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T06:46:21.065 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-09-11T06:46:21.145 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-11T06:46:21.146 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-09-11T06:46:21.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T06:46:21.227 DEBUG:teuthology.orchestra.run.smithi089:> true 2024-09-11T06:46:21.301 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-09-11T06:46:21.301 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-09-11T06:46:21.302 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T06:46:21.381 DEBUG:teuthology.orchestra.run.smithi176:> true 2024-09-11T06:46:21.457 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-09-11T06:46:21.458 INFO:teuthology.run_tasks:Running task clock... 2024-09-11T06:46:21.467 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-11T06:46:21.468 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T06:46:21.468 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T06:46:21.471 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T06:46:21.471 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T06:46:21.473 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T06:46:21.473 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T06:46:21.498 INFO:teuthology.orchestra.run.smithi089.stdout:11 Sep 06:46:21 ntpd[18423]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-11T06:46:21.498 INFO:teuthology.orchestra.run.smithi089.stdout:11 Sep 06:46:21 ntpd[18423]: Command line: ntpd -gq 2024-09-11T06:46:21.498 INFO:teuthology.orchestra.run.smithi089.stdout:11 Sep 06:46:21 ntpd[18423]: ---------------------------------------------------- 2024-09-11T06:46:21.499 INFO:teuthology.orchestra.run.smithi089.stdout:11 Sep 06:46:21 ntpd[18423]: ntp-4 is maintained by Network Time Foundation, 2024-09-11T06:46:21.499 INFO:teuthology.orchestra.run.smithi089.stdout:11 Sep 06:46:21 ntpd[18423]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-11T06:46:21.499 INFO:teuthology.orchestra.run.smithi089.stdout:11 Sep 06:46:21 ntpd[18423]: corporation. Support and training for ntp-4 are 2024-09-11T06:46:21.499 INFO:teuthology.orchestra.run.smithi089.stdout:11 Sep 06:46:21 ntpd[18423]: available at https://www.nwtime.org/support 2024-09-11T06:46:21.499 INFO:teuthology.orchestra.run.smithi089.stdout:11 Sep 06:46:21 ntpd[18423]: ---------------------------------------------------- 2024-09-11T06:46:21.500 INFO:teuthology.orchestra.run.smithi089.stdout:11 Sep 06:46:21 ntpd[18423]: proto: precision = 0.062 usec (-24) 2024-09-11T06:46:21.500 INFO:teuthology.orchestra.run.smithi089.stdout:11 Sep 06:46:21 ntpd[18423]: basedate set to 2022-02-04 2024-09-11T06:46:21.501 INFO:teuthology.orchestra.run.smithi089.stdout:11 Sep 06:46:21 ntpd[18423]: gps base set to 2022-02-06 (week 2196) 2024-09-11T06:46:21.501 INFO:teuthology.orchestra.run.smithi089.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-11T06:46:21.501 INFO:teuthology.orchestra.run.smithi071.stdout:11 Sep 06:46:21 ntpd[18458]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-11T06:46:21.501 INFO:teuthology.orchestra.run.smithi071.stdout:11 Sep 06:46:21 ntpd[18458]: Command line: ntpd -gq 2024-09-11T06:46:21.501 INFO:teuthology.orchestra.run.smithi071.stdout:11 Sep 06:46:21 ntpd[18458]: ---------------------------------------------------- 2024-09-11T06:46:21.502 INFO:teuthology.orchestra.run.smithi071.stdout:11 Sep 06:46:21 ntpd[18458]: ntp-4 is maintained by Network Time Foundation, 2024-09-11T06:46:21.502 INFO:teuthology.orchestra.run.smithi071.stdout:11 Sep 06:46:21 ntpd[18458]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-11T06:46:21.502 INFO:teuthology.orchestra.run.smithi071.stdout:11 Sep 06:46:21 ntpd[18458]: corporation. Support and training for ntp-4 are 2024-09-11T06:46:21.502 INFO:teuthology.orchestra.run.smithi071.stdout:11 Sep 06:46:21 ntpd[18458]: available at https://www.nwtime.org/support 2024-09-11T06:46:21.502 INFO:teuthology.orchestra.run.smithi071.stdout:11 Sep 06:46:21 ntpd[18458]: ---------------------------------------------------- 2024-09-11T06:46:21.502 INFO:teuthology.orchestra.run.smithi071.stdout:11 Sep 06:46:21 ntpd[18458]: proto: precision = 0.043 usec (-24) 2024-09-11T06:46:21.502 INFO:teuthology.orchestra.run.smithi071.stdout:11 Sep 06:46:21 ntpd[18458]: basedate set to 2022-02-04 2024-09-11T06:46:21.502 INFO:teuthology.orchestra.run.smithi071.stdout:11 Sep 06:46:21 ntpd[18458]: gps base set to 2022-02-06 (week 2196) 2024-09-11T06:46:21.502 INFO:teuthology.orchestra.run.smithi071.stdout:11 Sep 06:46:21 ntpd[18458]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-11T06:46:21.502 INFO:teuthology.orchestra.run.smithi071.stdout:11 Sep 06:46:21 ntpd[18458]: restrict ::: KOD does nothing without LIMITED. 2024-09-11T06:46:21.503 INFO:teuthology.orchestra.run.smithi071.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-11T06:46:21.503 INFO:teuthology.orchestra.run.smithi071.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-11T06:46:21.503 INFO:teuthology.orchestra.run.smithi089.stdout:11 Sep 06:46:21 ntpd[18423]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-11T06:46:21.503 INFO:teuthology.orchestra.run.smithi089.stdout:11 Sep 06:46:21 ntpd[18423]: restrict ::: KOD does nothing without LIMITED. 2024-09-11T06:46:21.503 INFO:teuthology.orchestra.run.smithi089.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-11T06:46:21.505 INFO:teuthology.orchestra.run.smithi071.stdout:11 Sep 06:46:21 ntpd[18458]: Listen and drop on 0 v6wildcard [::]:123 2024-09-11T06:46:21.505 INFO:teuthology.orchestra.run.smithi071.stdout:11 Sep 06:46:21 ntpd[18458]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-11T06:46:21.505 INFO:teuthology.orchestra.run.smithi071.stdout:11 Sep 06:46:21 ntpd[18458]: Listen normally on 2 lo 127.0.0.1:123 2024-09-11T06:46:21.505 INFO:teuthology.orchestra.run.smithi071.stdout:11 Sep 06:46:21 ntpd[18458]: Listen normally on 3 enp3s0f1 172.21.15.71:123 2024-09-11T06:46:21.505 INFO:teuthology.orchestra.run.smithi071.stdout:11 Sep 06:46:21 ntpd[18458]: Listen normally on 4 lo [::1]:123 2024-09-11T06:46:21.505 INFO:teuthology.orchestra.run.smithi071.stdout:11 Sep 06:46:21 ntpd[18458]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d765%5]:123 2024-09-11T06:46:21.505 INFO:teuthology.orchestra.run.smithi071.stdout:11 Sep 06:46:21 ntpd[18458]: Listening on routing socket on fd #22 for interface updates 2024-09-11T06:46:21.505 INFO:teuthology.orchestra.run.smithi089.stdout:11 Sep 06:46:21 ntpd[18423]: Listen and drop on 0 v6wildcard [::]:123 2024-09-11T06:46:21.506 INFO:teuthology.orchestra.run.smithi089.stdout:11 Sep 06:46:21 ntpd[18423]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-11T06:46:21.506 INFO:teuthology.orchestra.run.smithi089.stdout:11 Sep 06:46:21 ntpd[18423]: Listen normally on 2 lo 127.0.0.1:123 2024-09-11T06:46:21.506 INFO:teuthology.orchestra.run.smithi089.stdout:11 Sep 06:46:21 ntpd[18423]: Listen normally on 3 enp3s0f1 172.21.15.89:123 2024-09-11T06:46:21.506 INFO:teuthology.orchestra.run.smithi089.stdout:11 Sep 06:46:21 ntpd[18423]: Listen normally on 4 lo [::1]:123 2024-09-11T06:46:21.506 INFO:teuthology.orchestra.run.smithi089.stdout:11 Sep 06:46:21 ntpd[18423]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d501%5]:123 2024-09-11T06:46:21.506 INFO:teuthology.orchestra.run.smithi089.stdout:11 Sep 06:46:21 ntpd[18423]: Listening on routing socket on fd #22 for interface updates 2024-09-11T06:46:21.521 INFO:teuthology.orchestra.run.smithi176.stdout:11 Sep 06:46:21 ntpd[18496]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-11T06:46:21.521 INFO:teuthology.orchestra.run.smithi176.stdout:11 Sep 06:46:21 ntpd[18496]: Command line: ntpd -gq 2024-09-11T06:46:21.521 INFO:teuthology.orchestra.run.smithi176.stdout:11 Sep 06:46:21 ntpd[18496]: ---------------------------------------------------- 2024-09-11T06:46:21.521 INFO:teuthology.orchestra.run.smithi176.stdout:11 Sep 06:46:21 ntpd[18496]: ntp-4 is maintained by Network Time Foundation, 2024-09-11T06:46:21.521 INFO:teuthology.orchestra.run.smithi176.stdout:11 Sep 06:46:21 ntpd[18496]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-11T06:46:21.521 INFO:teuthology.orchestra.run.smithi176.stdout:11 Sep 06:46:21 ntpd[18496]: corporation. Support and training for ntp-4 are 2024-09-11T06:46:21.521 INFO:teuthology.orchestra.run.smithi176.stdout:11 Sep 06:46:21 ntpd[18496]: available at https://www.nwtime.org/support 2024-09-11T06:46:21.521 INFO:teuthology.orchestra.run.smithi176.stdout:11 Sep 06:46:21 ntpd[18496]: ---------------------------------------------------- 2024-09-11T06:46:21.522 INFO:teuthology.orchestra.run.smithi176.stdout:11 Sep 06:46:21 ntpd[18496]: proto: precision = 0.045 usec (-24) 2024-09-11T06:46:21.522 INFO:teuthology.orchestra.run.smithi176.stdout:11 Sep 06:46:21 ntpd[18496]: basedate set to 2022-02-04 2024-09-11T06:46:21.522 INFO:teuthology.orchestra.run.smithi176.stdout:11 Sep 06:46:21 ntpd[18496]: gps base set to 2022-02-06 (week 2196) 2024-09-11T06:46:21.522 INFO:teuthology.orchestra.run.smithi176.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-11T06:46:21.524 INFO:teuthology.orchestra.run.smithi176.stdout:11 Sep 06:46:21 ntpd[18496]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-11T06:46:21.524 INFO:teuthology.orchestra.run.smithi176.stdout:11 Sep 06:46:21 ntpd[18496]: restrict ::: KOD does nothing without LIMITED. 2024-09-11T06:46:21.524 INFO:teuthology.orchestra.run.smithi176.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-11T06:46:21.525 INFO:teuthology.orchestra.run.smithi176.stdout:11 Sep 06:46:21 ntpd[18496]: Listen and drop on 0 v6wildcard [::]:123 2024-09-11T06:46:21.525 INFO:teuthology.orchestra.run.smithi176.stdout:11 Sep 06:46:21 ntpd[18496]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-11T06:46:21.525 INFO:teuthology.orchestra.run.smithi176.stdout:11 Sep 06:46:21 ntpd[18496]: Listen normally on 2 lo 127.0.0.1:123 2024-09-11T06:46:21.525 INFO:teuthology.orchestra.run.smithi176.stdout:11 Sep 06:46:21 ntpd[18496]: Listen normally on 3 enp3s0f1 172.21.15.176:123 2024-09-11T06:46:21.525 INFO:teuthology.orchestra.run.smithi176.stdout:11 Sep 06:46:21 ntpd[18496]: Listen normally on 4 lo [::1]:123 2024-09-11T06:46:21.525 INFO:teuthology.orchestra.run.smithi176.stdout:11 Sep 06:46:21 ntpd[18496]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6f61%5]:123 2024-09-11T06:46:21.525 INFO:teuthology.orchestra.run.smithi176.stdout:11 Sep 06:46:21 ntpd[18496]: Listening on routing socket on fd #22 for interface updates 2024-09-11T06:46:22.500 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:22 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:22.500 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:22 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:22.500 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:22 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:22.500 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:22 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:22.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:22 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:22.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:22 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:22.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:22 ntpd[18458]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T06:46:22.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:22 ntpd[18458]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T06:46:22.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:22 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:22.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:22 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:22.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:22 ntpd[18458]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T06:46:22.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:22 ntpd[18458]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T06:46:23.499 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:23 ntpd[18423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:23.499 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:23 ntpd[18423]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:23.499 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:23 ntpd[18423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:23.499 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:23 ntpd[18423]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:23.499 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:23 ntpd[18423]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T06:46:23.500 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:23 ntpd[18423]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T06:46:23.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:23 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:23.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:23 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:23.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:23 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:23.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:23 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:23.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:23 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:23.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:23 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:23.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:23 ntpd[18458]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T06:46:23.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:23 ntpd[18458]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T06:46:23.521 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:23 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:23.521 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:23 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:23.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:23 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:23.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:23 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:23.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:23 ntpd[18496]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T06:46:23.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:23 ntpd[18496]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T06:46:24.500 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:24 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:24.500 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:24 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:24.500 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:24 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:24.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:24 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:24.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:24 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:24.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:24 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:24.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:24 ntpd[18458]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T06:46:24.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:24 ntpd[18458]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T06:46:24.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:24 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:24.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:24 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:24.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:24 ntpd[18458]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T06:46:24.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:24 ntpd[18458]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T06:46:24.521 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:24 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:24.521 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:24 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:24.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:24 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:24.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:24 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:24.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:24 ntpd[18496]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T06:46:24.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:24 ntpd[18496]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T06:46:25.499 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:25 ntpd[18423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:25.499 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:25 ntpd[18423]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:25.499 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:25 ntpd[18423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:25.499 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:25 ntpd[18423]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:25.499 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:25 ntpd[18423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:25.500 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:25 ntpd[18423]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:25.500 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:25 ntpd[18423]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T06:46:25.500 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:25 ntpd[18423]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T06:46:25.500 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:25 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:25.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:25 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:25.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:25 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:25.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:25 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:25.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:25 ntpd[18458]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T06:46:25.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:25 ntpd[18458]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T06:46:25.521 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:25 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:25.521 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:25 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:25.521 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:25 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:25.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:25 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:25.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:25 ntpd[18496]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T06:46:25.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:25 ntpd[18496]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T06:46:26.499 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:26 ntpd[18423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:26.500 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:26 ntpd[18423]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:26.500 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:26 ntpd[18423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:26.500 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:26 ntpd[18423]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:26.500 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:26 ntpd[18423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:26.500 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:26 ntpd[18423]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:26.500 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:26 ntpd[18423]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T06:46:26.500 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:26 ntpd[18423]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T06:46:26.500 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:26 ntpd[18423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:26.500 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:26 ntpd[18423]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:26.500 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:26 ntpd[18423]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T06:46:26.500 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:26 ntpd[18423]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T06:46:26.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:26 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:26.502 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:26 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:26.502 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:26 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:26.502 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:26 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:26.502 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:26 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:26.502 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:26 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:26.502 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:26 ntpd[18458]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T06:46:26.502 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:26 ntpd[18458]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T06:46:26.502 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:26 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:26.502 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:26 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:26.502 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:26 ntpd[18458]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T06:46:26.502 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:26 ntpd[18458]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T06:46:26.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:26 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:26.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:26 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:26.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:26 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:26.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:26 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:26.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:26 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:26.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:26 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:26.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:26 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:26.523 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:26 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:26.523 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:26 ntpd[18496]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T06:46:26.523 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:26 ntpd[18496]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T06:46:26.523 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:26 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:26.523 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:26 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:26.523 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:26 ntpd[18496]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T06:46:26.523 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:26 ntpd[18496]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T06:46:27.499 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:27 ntpd[18423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:27.499 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:27 ntpd[18423]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:27.499 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:27 ntpd[18423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:27.499 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:27 ntpd[18423]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:27.499 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:27 ntpd[18423]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T06:46:27.499 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:27 ntpd[18423]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T06:46:27.500 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:27 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:27.500 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:27 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:27.500 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:27 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:27.500 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:27 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:27.500 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:27 ntpd[18458]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T06:46:27.500 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:27 ntpd[18458]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T06:46:27.521 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:27 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:27.521 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:27 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:27.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:27 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:27.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:27 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:27.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:27 ntpd[18496]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T06:46:27.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:27 ntpd[18496]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T06:46:28.499 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:28 ntpd[18423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:28.499 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:28 ntpd[18423]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:28.499 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:28 ntpd[18423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:28 ntpd[18423]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:28 ntpd[18423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:28 ntpd[18423]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:28 ntpd[18423]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T06:46:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:28 ntpd[18423]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T06:46:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:28 ntpd[18423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:28 ntpd[18423]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:28 ntpd[18423]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T06:46:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:28 ntpd[18423]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T06:46:28.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:28 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:28.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:28 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:28.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:28 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:28.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:28 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:28.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:28 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:28.501 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:28 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:28.502 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:28 ntpd[18458]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T06:46:28.502 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:28 ntpd[18458]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T06:46:28.502 INFO:teuthology.orchestra.run.smithi071.stdout:11 Sep 06:46:28 ntpd[18458]: ntpd: time slew -0.000877 s 2024-09-11T06:46:28.502 INFO:teuthology.orchestra.run.smithi071.stdout:ntpd: time slew -0.000877s 2024-09-11T06:46:28.502 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:28 ntpd[18458]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-11T06:46:28.502 INFO:teuthology.orchestra.run.smithi071.stderr:11 Sep 06:46:28 ntpd[18458]: can't open /var/log/ntpstats/loopstats.20240911: Permission denied 2024-09-11T06:46:28.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:28 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:28.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:28 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:28.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:28 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:28.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:28 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:28.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:28 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:28.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:28 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:28.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:28 ntpd[18496]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T06:46:28.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:28 ntpd[18496]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T06:46:28.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:28 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:28.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:28 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:28.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:28 ntpd[18496]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T06:46:28.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:28 ntpd[18496]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T06:46:28.615 INFO:teuthology.orchestra.run.smithi071.stdout: remote refid st t when poll reach delay offset jitter 2024-09-11T06:46:28.615 INFO:teuthology.orchestra.run.smithi071.stdout:============================================================================== 2024-09-11T06:46:28.615 INFO:teuthology.orchestra.run.smithi071.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T06:46:28.615 INFO:teuthology.orchestra.run.smithi071.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T06:46:28.616 INFO:teuthology.orchestra.run.smithi071.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T06:46:28.616 INFO:teuthology.orchestra.run.smithi071.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T06:46:29.499 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:29 ntpd[18423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:29.499 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:29 ntpd[18423]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:29.499 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:29 ntpd[18423]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:29.499 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:29 ntpd[18423]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:29.499 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:29 ntpd[18423]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T06:46:29.499 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:29 ntpd[18423]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T06:46:29.500 INFO:teuthology.orchestra.run.smithi089.stdout:11 Sep 06:46:29 ntpd[18423]: ntpd: time slew -0.001345 s 2024-09-11T06:46:29.500 INFO:teuthology.orchestra.run.smithi089.stdout:ntpd: time slew -0.001345s 2024-09-11T06:46:29.500 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:29 ntpd[18423]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-11T06:46:29.500 INFO:teuthology.orchestra.run.smithi089.stderr:11 Sep 06:46:29 ntpd[18423]: can't open /var/log/ntpstats/loopstats.20240911: Permission denied 2024-09-11T06:46:29.521 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:29 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:29.521 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:29 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:29.521 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:29 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T06:46:29.521 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:29 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T06:46:29.521 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:29 ntpd[18496]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T06:46:29.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:29 ntpd[18496]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T06:46:29.522 INFO:teuthology.orchestra.run.smithi176.stdout:11 Sep 06:46:29 ntpd[18496]: ntpd: time slew -0.000243 s 2024-09-11T06:46:29.522 INFO:teuthology.orchestra.run.smithi176.stdout:ntpd: time slew -0.000243s 2024-09-11T06:46:29.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:29 ntpd[18496]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-11T06:46:29.522 INFO:teuthology.orchestra.run.smithi176.stderr:11 Sep 06:46:29 ntpd[18496]: can't open /var/log/ntpstats/loopstats.20240911: Permission denied 2024-09-11T06:46:29.547 INFO:teuthology.orchestra.run.smithi089.stdout: remote refid st t when poll reach delay offset jitter 2024-09-11T06:46:29.547 INFO:teuthology.orchestra.run.smithi089.stdout:============================================================================== 2024-09-11T06:46:29.548 INFO:teuthology.orchestra.run.smithi089.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T06:46:29.548 INFO:teuthology.orchestra.run.smithi089.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T06:46:29.548 INFO:teuthology.orchestra.run.smithi089.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T06:46:29.548 INFO:teuthology.orchestra.run.smithi089.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T06:46:29.573 INFO:teuthology.orchestra.run.smithi176.stdout: remote refid st t when poll reach delay offset jitter 2024-09-11T06:46:29.573 INFO:teuthology.orchestra.run.smithi176.stdout:============================================================================== 2024-09-11T06:46:29.573 INFO:teuthology.orchestra.run.smithi176.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T06:46:29.573 INFO:teuthology.orchestra.run.smithi176.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T06:46:29.573 INFO:teuthology.orchestra.run.smithi176.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T06:46:29.573 INFO:teuthology.orchestra.run.smithi176.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T06:46:29.574 INFO:teuthology.run_tasks:Running task install... 2024-09-11T06:46:29.581 DEBUG:teuthology.task.install:project ceph 2024-09-11T06:46:29.582 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}} 2024-09-11T06:46:29.582 DEBUG:teuthology.task.install:config {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'} 2024-09-11T06:46:29.582 INFO:teuthology.task.install:Using flavor: default 2024-09-11T06:46:29.587 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-11T06:46:29.587 INFO:teuthology.task.install:extra packages: ['rbd-nbd'] 2024-09-11T06:46:29.588 DEBUG:teuthology.orchestra.run.smithi071:> sudo apt-key list | grep Ceph 2024-09-11T06:46:29.588 DEBUG:teuthology.orchestra.run.smithi089:> sudo apt-key list | grep Ceph 2024-09-11T06:46:29.589 DEBUG:teuthology.orchestra.run.smithi176:> sudo apt-key list | grep Ceph 2024-09-11T06:46:29.649 INFO:teuthology.orchestra.run.smithi089.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-11T06:46:29.651 INFO:teuthology.orchestra.run.smithi071.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-11T06:46:29.667 INFO:teuthology.orchestra.run.smithi176.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-11T06:46:29.676 INFO:teuthology.orchestra.run.smithi089.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-11T06:46:29.676 INFO:teuthology.orchestra.run.smithi089.stdout:uid [ unknown] Ceph.com (release key) 2024-09-11T06:46:29.677 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, rbd-nbd on remote deb x86_64 2024-09-11T06:46:29.678 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T06:46:29.683 INFO:teuthology.orchestra.run.smithi071.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-11T06:46:29.683 INFO:teuthology.orchestra.run.smithi071.stdout:uid [ unknown] Ceph.com (release key) 2024-09-11T06:46:29.684 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, rbd-nbd on remote deb x86_64 2024-09-11T06:46:29.684 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T06:46:29.694 INFO:teuthology.orchestra.run.smithi176.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-11T06:46:29.694 INFO:teuthology.orchestra.run.smithi176.stdout:uid [ unknown] Ceph.com (release key) 2024-09-11T06:46:29.694 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, rbd-nbd on remote deb x86_64 2024-09-11T06:46:29.694 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T06:46:29.830 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default/ 2024-09-11T06:46:29.830 INFO:teuthology.task.install.deb:Package version is 19.1.1-298-gb53b6d3e-1jammy 2024-09-11T06:46:29.834 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default/ 2024-09-11T06:46:29.834 INFO:teuthology.task.install.deb:Package version is 19.1.1-298-gb53b6d3e-1jammy 2024-09-11T06:46:29.843 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default/ 2024-09-11T06:46:29.843 INFO:teuthology.task.install.deb:Package version is 19.1.1-298-gb53b6d3e-1jammy 2024-09-11T06:46:29.946 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T06:46:29.946 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-11T06:46:29.956 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-11T06:46:29.956 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-11T06:46:29.961 DEBUG:teuthology.orchestra.run.smithi071:> sudo apt-get update 2024-09-11T06:46:29.964 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-11T06:46:29.964 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-11T06:46:29.970 DEBUG:teuthology.orchestra.run.smithi089:> sudo apt-get update 2024-09-11T06:46:29.978 DEBUG:teuthology.orchestra.run.smithi176:> sudo apt-get update 2024-09-11T06:46:30.127 INFO:teuthology.orchestra.run.smithi071.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-11T06:46:30.127 INFO:teuthology.orchestra.run.smithi071.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-11T06:46:30.137 INFO:teuthology.orchestra.run.smithi089.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-11T06:46:30.138 INFO:teuthology.orchestra.run.smithi071.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-11T06:46:30.147 INFO:teuthology.orchestra.run.smithi176.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-11T06:46:30.147 INFO:teuthology.orchestra.run.smithi176.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-11T06:46:30.148 INFO:teuthology.orchestra.run.smithi089.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-11T06:46:30.155 INFO:teuthology.orchestra.run.smithi071.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-11T06:46:30.157 INFO:teuthology.orchestra.run.smithi176.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-11T06:46:30.165 INFO:teuthology.orchestra.run.smithi089.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-11T06:46:30.174 INFO:teuthology.orchestra.run.smithi176.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-11T06:46:30.183 INFO:teuthology.orchestra.run.smithi071.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy InRelease 2024-09-11T06:46:30.193 INFO:teuthology.orchestra.run.smithi089.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy InRelease 2024-09-11T06:46:30.204 INFO:teuthology.orchestra.run.smithi176.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy InRelease 2024-09-11T06:46:30.205 INFO:teuthology.orchestra.run.smithi071.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-11T06:46:30.215 INFO:teuthology.orchestra.run.smithi089.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-11T06:46:30.225 INFO:teuthology.orchestra.run.smithi176.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-11T06:46:30.226 INFO:teuthology.orchestra.run.smithi071.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-11T06:46:30.238 INFO:teuthology.orchestra.run.smithi089.stdout:Ign:6 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-11T06:46:30.247 INFO:teuthology.orchestra.run.smithi071.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-11T06:46:30.248 INFO:teuthology.orchestra.run.smithi176.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-11T06:46:30.259 INFO:teuthology.orchestra.run.smithi089.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,404 B] 2024-09-11T06:46:30.266 INFO:teuthology.orchestra.run.smithi089.stdout:Hit:8 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-11T06:46:30.269 INFO:teuthology.orchestra.run.smithi176.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-11T06:46:30.274 INFO:teuthology.orchestra.run.smithi071.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,404 B] 2024-09-11T06:46:30.287 INFO:teuthology.orchestra.run.smithi089.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-11T06:46:30.296 INFO:teuthology.orchestra.run.smithi176.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,404 B] 2024-09-11T06:46:30.469 INFO:teuthology.orchestra.run.smithi071.stdout:Fetched 29.0 kB in 0s (88.2 kB/s) 2024-09-11T06:46:30.491 INFO:teuthology.orchestra.run.smithi176.stdout:Fetched 29.0 kB in 0s (87.3 kB/s) 2024-09-11T06:46:30.495 INFO:teuthology.orchestra.run.smithi089.stdout:Fetched 29.0 kB in 0s (84.1 kB/s) 2024-09-11T06:46:31.453 INFO:teuthology.orchestra.run.smithi176.stdout:Reading package lists... 2024-09-11T06:46:31.455 INFO:teuthology.orchestra.run.smithi071.stdout:Reading package lists... 2024-09-11T06:46:31.470 DEBUG:teuthology.orchestra.run.smithi176:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-298-gb53b6d3e-1jammy cephadm=19.1.1-298-gb53b6d3e-1jammy ceph-mds=19.1.1-298-gb53b6d3e-1jammy ceph-mgr=19.1.1-298-gb53b6d3e-1jammy ceph-common=19.1.1-298-gb53b6d3e-1jammy ceph-fuse=19.1.1-298-gb53b6d3e-1jammy ceph-test=19.1.1-298-gb53b6d3e-1jammy ceph-volume=19.1.1-298-gb53b6d3e-1jammy radosgw=19.1.1-298-gb53b6d3e-1jammy python3-rados=19.1.1-298-gb53b6d3e-1jammy python3-rgw=19.1.1-298-gb53b6d3e-1jammy python3-cephfs=19.1.1-298-gb53b6d3e-1jammy python3-rbd=19.1.1-298-gb53b6d3e-1jammy libcephfs2=19.1.1-298-gb53b6d3e-1jammy libcephfs-dev=19.1.1-298-gb53b6d3e-1jammy librados2=19.1.1-298-gb53b6d3e-1jammy librbd1=19.1.1-298-gb53b6d3e-1jammy rbd-fuse=19.1.1-298-gb53b6d3e-1jammy rbd-nbd=19.1.1-298-gb53b6d3e-1jammy 2024-09-11T06:46:31.471 DEBUG:teuthology.orchestra.run.smithi071:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-298-gb53b6d3e-1jammy cephadm=19.1.1-298-gb53b6d3e-1jammy ceph-mds=19.1.1-298-gb53b6d3e-1jammy ceph-mgr=19.1.1-298-gb53b6d3e-1jammy ceph-common=19.1.1-298-gb53b6d3e-1jammy ceph-fuse=19.1.1-298-gb53b6d3e-1jammy ceph-test=19.1.1-298-gb53b6d3e-1jammy ceph-volume=19.1.1-298-gb53b6d3e-1jammy radosgw=19.1.1-298-gb53b6d3e-1jammy python3-rados=19.1.1-298-gb53b6d3e-1jammy python3-rgw=19.1.1-298-gb53b6d3e-1jammy python3-cephfs=19.1.1-298-gb53b6d3e-1jammy python3-rbd=19.1.1-298-gb53b6d3e-1jammy libcephfs2=19.1.1-298-gb53b6d3e-1jammy libcephfs-dev=19.1.1-298-gb53b6d3e-1jammy librados2=19.1.1-298-gb53b6d3e-1jammy librbd1=19.1.1-298-gb53b6d3e-1jammy rbd-fuse=19.1.1-298-gb53b6d3e-1jammy rbd-nbd=19.1.1-298-gb53b6d3e-1jammy 2024-09-11T06:46:31.501 INFO:teuthology.orchestra.run.smithi089.stdout:Reading package lists... 2024-09-11T06:46:31.518 DEBUG:teuthology.orchestra.run.smithi089:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-298-gb53b6d3e-1jammy cephadm=19.1.1-298-gb53b6d3e-1jammy ceph-mds=19.1.1-298-gb53b6d3e-1jammy ceph-mgr=19.1.1-298-gb53b6d3e-1jammy ceph-common=19.1.1-298-gb53b6d3e-1jammy ceph-fuse=19.1.1-298-gb53b6d3e-1jammy ceph-test=19.1.1-298-gb53b6d3e-1jammy ceph-volume=19.1.1-298-gb53b6d3e-1jammy radosgw=19.1.1-298-gb53b6d3e-1jammy python3-rados=19.1.1-298-gb53b6d3e-1jammy python3-rgw=19.1.1-298-gb53b6d3e-1jammy python3-cephfs=19.1.1-298-gb53b6d3e-1jammy python3-rbd=19.1.1-298-gb53b6d3e-1jammy libcephfs2=19.1.1-298-gb53b6d3e-1jammy libcephfs-dev=19.1.1-298-gb53b6d3e-1jammy librados2=19.1.1-298-gb53b6d3e-1jammy librbd1=19.1.1-298-gb53b6d3e-1jammy rbd-fuse=19.1.1-298-gb53b6d3e-1jammy rbd-nbd=19.1.1-298-gb53b6d3e-1jammy 2024-09-11T06:46:31.535 INFO:teuthology.orchestra.run.smithi176.stdout:Reading package lists... 2024-09-11T06:46:31.538 INFO:teuthology.orchestra.run.smithi071.stdout:Reading package lists... 2024-09-11T06:46:31.585 INFO:teuthology.orchestra.run.smithi089.stdout:Reading package lists... 2024-09-11T06:46:31.744 INFO:teuthology.orchestra.run.smithi176.stdout:Building dependency tree... 2024-09-11T06:46:31.744 INFO:teuthology.orchestra.run.smithi176.stdout:Reading state information... 2024-09-11T06:46:31.747 INFO:teuthology.orchestra.run.smithi071.stdout:Building dependency tree... 2024-09-11T06:46:31.748 INFO:teuthology.orchestra.run.smithi071.stdout:Reading state information... 2024-09-11T06:46:31.803 INFO:teuthology.orchestra.run.smithi089.stdout:Building dependency tree... 2024-09-11T06:46:31.804 INFO:teuthology.orchestra.run.smithi089.stdout:Reading state information... 2024-09-11T06:46:31.948 INFO:teuthology.orchestra.run.smithi176.stdout:The following packages were automatically installed and are no longer required: 2024-09-11T06:46:31.948 INFO:teuthology.orchestra.run.smithi176.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-11T06:46:31.948 INFO:teuthology.orchestra.run.smithi176.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-11T06:46:31.948 INFO:teuthology.orchestra.run.smithi176.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-11T06:46:31.949 INFO:teuthology.orchestra.run.smithi176.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-11T06:46:31.949 INFO:teuthology.orchestra.run.smithi176.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-11T06:46:31.949 INFO:teuthology.orchestra.run.smithi176.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-11T06:46:31.950 INFO:teuthology.orchestra.run.smithi176.stdout:The following additional packages will be installed: 2024-09-11T06:46:31.950 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-11T06:46:31.950 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-11T06:46:31.950 INFO:teuthology.orchestra.run.smithi176.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-11T06:46:31.950 INFO:teuthology.orchestra.run.smithi176.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-11T06:46:31.951 INFO:teuthology.orchestra.run.smithi176.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-11T06:46:31.951 INFO:teuthology.orchestra.run.smithi176.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-11T06:46:31.951 INFO:teuthology.orchestra.run.smithi176.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-11T06:46:31.951 INFO:teuthology.orchestra.run.smithi176.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-11T06:46:31.951 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-11T06:46:31.952 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-11T06:46:31.952 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-11T06:46:31.952 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-11T06:46:31.952 INFO:teuthology.orchestra.run.smithi176.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-11T06:46:31.952 INFO:teuthology.orchestra.run.smithi176.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-11T06:46:31.952 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-11T06:46:31.952 INFO:teuthology.orchestra.run.smithi176.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-11T06:46:31.952 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-11T06:46:31.952 INFO:teuthology.orchestra.run.smithi176.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-11T06:46:31.952 INFO:teuthology.orchestra.run.smithi176.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-11T06:46:31.952 INFO:teuthology.orchestra.run.smithi176.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-11T06:46:31.952 INFO:teuthology.orchestra.run.smithi176.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-11T06:46:31.952 INFO:teuthology.orchestra.run.smithi176.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-11T06:46:31.953 INFO:teuthology.orchestra.run.smithi176.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-11T06:46:31.953 INFO:teuthology.orchestra.run.smithi176.stdout: xmlstarlet zip 2024-09-11T06:46:31.953 INFO:teuthology.orchestra.run.smithi176.stdout:Suggested packages: 2024-09-11T06:46:31.953 INFO:teuthology.orchestra.run.smithi176.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-11T06:46:31.953 INFO:teuthology.orchestra.run.smithi176.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-11T06:46:31.953 INFO:teuthology.orchestra.run.smithi176.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-11T06:46:31.953 INFO:teuthology.orchestra.run.smithi176.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-11T06:46:31.953 INFO:teuthology.orchestra.run.smithi176.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-11T06:46:31.953 INFO:teuthology.orchestra.run.smithi176.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-11T06:46:31.954 INFO:teuthology.orchestra.run.smithi176.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-11T06:46:31.954 INFO:teuthology.orchestra.run.smithi176.stdout:Recommended packages: 2024-09-11T06:46:31.954 INFO:teuthology.orchestra.run.smithi176.stdout: btrfs-tools 2024-09-11T06:46:31.956 INFO:teuthology.orchestra.run.smithi071.stdout:The following packages were automatically installed and are no longer required: 2024-09-11T06:46:31.957 INFO:teuthology.orchestra.run.smithi071.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-11T06:46:31.957 INFO:teuthology.orchestra.run.smithi071.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-11T06:46:31.957 INFO:teuthology.orchestra.run.smithi071.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-11T06:46:31.957 INFO:teuthology.orchestra.run.smithi071.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-11T06:46:31.958 INFO:teuthology.orchestra.run.smithi071.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-11T06:46:31.958 INFO:teuthology.orchestra.run.smithi071.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-11T06:46:31.958 INFO:teuthology.orchestra.run.smithi071.stdout:The following additional packages will be installed: 2024-09-11T06:46:31.958 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-11T06:46:31.958 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-11T06:46:31.960 INFO:teuthology.orchestra.run.smithi071.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-11T06:46:31.960 INFO:teuthology.orchestra.run.smithi071.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-11T06:46:31.960 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-11T06:46:31.960 INFO:teuthology.orchestra.run.smithi071.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-11T06:46:31.960 INFO:teuthology.orchestra.run.smithi071.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-11T06:46:31.960 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-11T06:46:31.960 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-11T06:46:31.960 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-11T06:46:31.960 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-11T06:46:31.960 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-11T06:46:31.960 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-11T06:46:31.961 INFO:teuthology.orchestra.run.smithi071.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-11T06:46:31.961 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-11T06:46:31.961 INFO:teuthology.orchestra.run.smithi071.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-11T06:46:31.961 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-11T06:46:31.961 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-11T06:46:31.961 INFO:teuthology.orchestra.run.smithi071.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-11T06:46:31.961 INFO:teuthology.orchestra.run.smithi071.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-11T06:46:31.961 INFO:teuthology.orchestra.run.smithi071.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-11T06:46:31.961 INFO:teuthology.orchestra.run.smithi071.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-11T06:46:31.961 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-11T06:46:31.961 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet zip 2024-09-11T06:46:31.962 INFO:teuthology.orchestra.run.smithi071.stdout:Suggested packages: 2024-09-11T06:46:31.962 INFO:teuthology.orchestra.run.smithi071.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-11T06:46:31.962 INFO:teuthology.orchestra.run.smithi071.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-11T06:46:31.962 INFO:teuthology.orchestra.run.smithi071.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-11T06:46:31.962 INFO:teuthology.orchestra.run.smithi071.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-11T06:46:31.962 INFO:teuthology.orchestra.run.smithi071.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-11T06:46:31.962 INFO:teuthology.orchestra.run.smithi071.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-11T06:46:31.962 INFO:teuthology.orchestra.run.smithi071.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-11T06:46:31.962 INFO:teuthology.orchestra.run.smithi071.stdout:Recommended packages: 2024-09-11T06:46:31.962 INFO:teuthology.orchestra.run.smithi071.stdout: btrfs-tools 2024-09-11T06:46:32.017 INFO:teuthology.orchestra.run.smithi089.stdout:The following packages were automatically installed and are no longer required: 2024-09-11T06:46:32.017 INFO:teuthology.orchestra.run.smithi089.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-11T06:46:32.018 INFO:teuthology.orchestra.run.smithi089.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-11T06:46:32.018 INFO:teuthology.orchestra.run.smithi089.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-11T06:46:32.018 INFO:teuthology.orchestra.run.smithi089.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-11T06:46:32.019 INFO:teuthology.orchestra.run.smithi089.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-11T06:46:32.019 INFO:teuthology.orchestra.run.smithi089.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-11T06:46:32.020 INFO:teuthology.orchestra.run.smithi089.stdout:The following additional packages will be installed: 2024-09-11T06:46:32.021 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-11T06:46:32.021 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-11T06:46:32.021 INFO:teuthology.orchestra.run.smithi089.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-11T06:46:32.021 INFO:teuthology.orchestra.run.smithi089.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-11T06:46:32.021 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-11T06:46:32.021 INFO:teuthology.orchestra.run.smithi089.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-11T06:46:32.021 INFO:teuthology.orchestra.run.smithi089.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-11T06:46:32.021 INFO:teuthology.orchestra.run.smithi089.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-11T06:46:32.021 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-11T06:46:32.021 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-11T06:46:32.021 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-11T06:46:32.021 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-11T06:46:32.022 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-11T06:46:32.022 INFO:teuthology.orchestra.run.smithi089.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-11T06:46:32.022 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-11T06:46:32.022 INFO:teuthology.orchestra.run.smithi089.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-11T06:46:32.022 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-11T06:46:32.022 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-11T06:46:32.022 INFO:teuthology.orchestra.run.smithi089.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-11T06:46:32.022 INFO:teuthology.orchestra.run.smithi089.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-11T06:46:32.022 INFO:teuthology.orchestra.run.smithi089.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-11T06:46:32.022 INFO:teuthology.orchestra.run.smithi089.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-11T06:46:32.022 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-11T06:46:32.022 INFO:teuthology.orchestra.run.smithi089.stdout: xmlstarlet zip 2024-09-11T06:46:32.023 INFO:teuthology.orchestra.run.smithi089.stdout:Suggested packages: 2024-09-11T06:46:32.023 INFO:teuthology.orchestra.run.smithi089.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-11T06:46:32.023 INFO:teuthology.orchestra.run.smithi089.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-11T06:46:32.023 INFO:teuthology.orchestra.run.smithi089.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-11T06:46:32.023 INFO:teuthology.orchestra.run.smithi089.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-11T06:46:32.023 INFO:teuthology.orchestra.run.smithi089.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-11T06:46:32.023 INFO:teuthology.orchestra.run.smithi089.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-11T06:46:32.023 INFO:teuthology.orchestra.run.smithi089.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-11T06:46:32.024 INFO:teuthology.orchestra.run.smithi089.stdout:Recommended packages: 2024-09-11T06:46:32.024 INFO:teuthology.orchestra.run.smithi089.stdout: btrfs-tools 2024-09-11T06:46:32.057 INFO:teuthology.orchestra.run.smithi176.stdout:The following NEW packages will be installed: 2024-09-11T06:46:32.057 INFO:teuthology.orchestra.run.smithi176.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-11T06:46:32.057 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-11T06:46:32.057 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-11T06:46:32.059 INFO:teuthology.orchestra.run.smithi176.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-11T06:46:32.059 INFO:teuthology.orchestra.run.smithi176.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-11T06:46:32.059 INFO:teuthology.orchestra.run.smithi176.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-11T06:46:32.059 INFO:teuthology.orchestra.run.smithi176.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-11T06:46:32.059 INFO:teuthology.orchestra.run.smithi176.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-11T06:46:32.059 INFO:teuthology.orchestra.run.smithi176.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-11T06:46:32.059 INFO:teuthology.orchestra.run.smithi176.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-11T06:46:32.059 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-11T06:46:32.059 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-11T06:46:32.059 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-11T06:46:32.060 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-11T06:46:32.060 INFO:teuthology.orchestra.run.smithi176.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-11T06:46:32.060 INFO:teuthology.orchestra.run.smithi176.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-11T06:46:32.060 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-11T06:46:32.060 INFO:teuthology.orchestra.run.smithi176.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-11T06:46:32.060 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-11T06:46:32.060 INFO:teuthology.orchestra.run.smithi176.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-11T06:46:32.060 INFO:teuthology.orchestra.run.smithi176.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-11T06:46:32.060 INFO:teuthology.orchestra.run.smithi176.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-11T06:46:32.060 INFO:teuthology.orchestra.run.smithi176.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-11T06:46:32.060 INFO:teuthology.orchestra.run.smithi176.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-11T06:46:32.060 INFO:teuthology.orchestra.run.smithi176.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-11T06:46:32.060 INFO:teuthology.orchestra.run.smithi176.stdout: radosgw rbd-fuse rbd-nbd socat unzip xmlstarlet zip 2024-09-11T06:46:32.066 INFO:teuthology.orchestra.run.smithi071.stdout:The following NEW packages will be installed: 2024-09-11T06:46:32.066 INFO:teuthology.orchestra.run.smithi071.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-11T06:46:32.066 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-11T06:46:32.066 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-11T06:46:32.067 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-11T06:46:32.067 INFO:teuthology.orchestra.run.smithi071.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-11T06:46:32.067 INFO:teuthology.orchestra.run.smithi071.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-11T06:46:32.068 INFO:teuthology.orchestra.run.smithi071.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-11T06:46:32.068 INFO:teuthology.orchestra.run.smithi071.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-11T06:46:32.068 INFO:teuthology.orchestra.run.smithi071.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-11T06:46:32.068 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-11T06:46:32.068 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-11T06:46:32.068 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-11T06:46:32.068 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-11T06:46:32.068 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-11T06:46:32.068 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-11T06:46:32.068 INFO:teuthology.orchestra.run.smithi071.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-11T06:46:32.068 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-11T06:46:32.068 INFO:teuthology.orchestra.run.smithi071.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-11T06:46:32.068 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-11T06:46:32.069 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-11T06:46:32.069 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-11T06:46:32.069 INFO:teuthology.orchestra.run.smithi071.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-11T06:46:32.069 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-11T06:46:32.069 INFO:teuthology.orchestra.run.smithi071.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-11T06:46:32.069 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-11T06:46:32.069 INFO:teuthology.orchestra.run.smithi071.stdout: radosgw rbd-fuse rbd-nbd socat unzip xmlstarlet zip 2024-09-11T06:46:32.127 INFO:teuthology.orchestra.run.smithi089.stdout:The following NEW packages will be installed: 2024-09-11T06:46:32.128 INFO:teuthology.orchestra.run.smithi089.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-11T06:46:32.128 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-11T06:46:32.128 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-11T06:46:32.128 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-11T06:46:32.128 INFO:teuthology.orchestra.run.smithi089.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-11T06:46:32.128 INFO:teuthology.orchestra.run.smithi089.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-11T06:46:32.129 INFO:teuthology.orchestra.run.smithi089.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-11T06:46:32.129 INFO:teuthology.orchestra.run.smithi089.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-11T06:46:32.129 INFO:teuthology.orchestra.run.smithi089.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-11T06:46:32.129 INFO:teuthology.orchestra.run.smithi089.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-11T06:46:32.129 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-11T06:46:32.130 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-11T06:46:32.130 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-11T06:46:32.130 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-11T06:46:32.130 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-11T06:46:32.130 INFO:teuthology.orchestra.run.smithi089.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-11T06:46:32.130 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-11T06:46:32.130 INFO:teuthology.orchestra.run.smithi089.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-11T06:46:32.130 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-11T06:46:32.130 INFO:teuthology.orchestra.run.smithi089.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-11T06:46:32.130 INFO:teuthology.orchestra.run.smithi089.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-11T06:46:32.130 INFO:teuthology.orchestra.run.smithi089.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-11T06:46:32.130 INFO:teuthology.orchestra.run.smithi089.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-11T06:46:32.130 INFO:teuthology.orchestra.run.smithi089.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-11T06:46:32.130 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-11T06:46:32.130 INFO:teuthology.orchestra.run.smithi089.stdout: radosgw rbd-fuse rbd-nbd socat unzip xmlstarlet zip 2024-09-11T06:46:32.205 INFO:teuthology.orchestra.run.smithi176.stdout:0 upgraded, 119 newly installed, 0 to remove and 331 not upgraded. 2024-09-11T06:46:32.205 INFO:teuthology.orchestra.run.smithi176.stdout:Need to get 181 MB of archives. 2024-09-11T06:46:32.205 INFO:teuthology.orchestra.run.smithi176.stdout:After this operation, 830 MB of additional disk space will be used. 2024-09-11T06:46:32.205 INFO:teuthology.orchestra.run.smithi176.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-298-gb53b6d3e-1jammy [3,588 kB] 2024-09-11T06:46:32.211 INFO:teuthology.orchestra.run.smithi071.stdout:0 upgraded, 119 newly installed, 0 to remove and 331 not upgraded. 2024-09-11T06:46:32.211 INFO:teuthology.orchestra.run.smithi071.stdout:Need to get 181 MB of archives. 2024-09-11T06:46:32.211 INFO:teuthology.orchestra.run.smithi071.stdout:After this operation, 830 MB of additional disk space will be used. 2024-09-11T06:46:32.211 INFO:teuthology.orchestra.run.smithi071.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-298-gb53b6d3e-1jammy [3,588 kB] 2024-09-11T06:46:32.249 INFO:teuthology.orchestra.run.smithi176.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-11T06:46:32.255 INFO:teuthology.orchestra.run.smithi071.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-11T06:46:32.270 INFO:teuthology.orchestra.run.smithi089.stdout:0 upgraded, 119 newly installed, 0 to remove and 331 not upgraded. 2024-09-11T06:46:32.270 INFO:teuthology.orchestra.run.smithi089.stdout:Need to get 181 MB of archives. 2024-09-11T06:46:32.270 INFO:teuthology.orchestra.run.smithi089.stdout:After this operation, 830 MB of additional disk space will be used. 2024-09-11T06:46:32.270 INFO:teuthology.orchestra.run.smithi089.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-298-gb53b6d3e-1jammy [3,588 kB] 2024-09-11T06:46:32.317 INFO:teuthology.orchestra.run.smithi089.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-11T06:46:32.405 INFO:teuthology.orchestra.run.smithi176.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-298-gb53b6d3e-1jammy [3,243 kB] 2024-09-11T06:46:32.450 INFO:teuthology.orchestra.run.smithi071.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-298-gb53b6d3e-1jammy [3,243 kB] 2024-09-11T06:46:32.489 INFO:teuthology.orchestra.run.smithi176.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-298-gb53b6d3e-1jammy [719 kB] 2024-09-11T06:46:32.504 INFO:teuthology.orchestra.run.smithi176.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-298-gb53b6d3e-1jammy [356 kB] 2024-09-11T06:46:32.511 INFO:teuthology.orchestra.run.smithi176.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-298-gb53b6d3e-1jammy [32.8 kB] 2024-09-11T06:46:32.512 INFO:teuthology.orchestra.run.smithi176.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-298-gb53b6d3e-1jammy [180 kB] 2024-09-11T06:46:32.515 INFO:teuthology.orchestra.run.smithi176.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-298-gb53b6d3e-1jammy [67.6 kB] 2024-09-11T06:46:32.516 INFO:teuthology.orchestra.run.smithi176.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-298-gb53b6d3e-1jammy [333 kB] 2024-09-11T06:46:32.522 INFO:teuthology.orchestra.run.smithi176.stdout:Get:10 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-298-gb53b6d3e-1jammy [6,875 kB] 2024-09-11T06:46:32.603 INFO:teuthology.orchestra.run.smithi071.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-298-gb53b6d3e-1jammy [719 kB] 2024-09-11T06:46:32.617 INFO:teuthology.orchestra.run.smithi071.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-298-gb53b6d3e-1jammy [356 kB] 2024-09-11T06:46:32.624 INFO:teuthology.orchestra.run.smithi071.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-298-gb53b6d3e-1jammy [32.8 kB] 2024-09-11T06:46:32.625 INFO:teuthology.orchestra.run.smithi071.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-298-gb53b6d3e-1jammy [180 kB] 2024-09-11T06:46:32.629 INFO:teuthology.orchestra.run.smithi071.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-298-gb53b6d3e-1jammy [67.6 kB] 2024-09-11T06:46:32.630 INFO:teuthology.orchestra.run.smithi071.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-298-gb53b6d3e-1jammy [333 kB] 2024-09-11T06:46:32.677 INFO:teuthology.orchestra.run.smithi176.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-11T06:46:32.679 INFO:teuthology.orchestra.run.smithi071.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-11T06:46:32.681 INFO:teuthology.orchestra.run.smithi071.stdout:Get:11 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-298-gb53b6d3e-1jammy [6,875 kB] 2024-09-11T06:46:32.690 INFO:teuthology.orchestra.run.smithi176.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-11T06:46:32.692 INFO:teuthology.orchestra.run.smithi071.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-11T06:46:32.743 INFO:teuthology.orchestra.run.smithi089.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-11T06:46:32.755 INFO:teuthology.orchestra.run.smithi089.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-11T06:46:32.773 INFO:teuthology.orchestra.run.smithi176.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-11T06:46:32.775 INFO:teuthology.orchestra.run.smithi071.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-11T06:46:32.803 INFO:teuthology.orchestra.run.smithi176.stdout:Get:14 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-298-gb53b6d3e-1jammy [112 kB] 2024-09-11T06:46:32.806 INFO:teuthology.orchestra.run.smithi176.stdout:Get:15 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-298-gb53b6d3e-1jammy [472 kB] 2024-09-11T06:46:32.815 INFO:teuthology.orchestra.run.smithi176.stdout:Get:16 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-298-gb53b6d3e-1jammy [25.9 MB] 2024-09-11T06:46:32.839 INFO:teuthology.orchestra.run.smithi089.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-11T06:46:32.923 INFO:teuthology.orchestra.run.smithi071.stdout:Get:14 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-298-gb53b6d3e-1jammy [112 kB] 2024-09-11T06:46:32.925 INFO:teuthology.orchestra.run.smithi071.stdout:Get:15 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-298-gb53b6d3e-1jammy [472 kB] 2024-09-11T06:46:32.963 INFO:teuthology.orchestra.run.smithi071.stdout:Get:16 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-298-gb53b6d3e-1jammy [25.9 MB] 2024-09-11T06:46:33.009 INFO:teuthology.orchestra.run.smithi176.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-11T06:46:33.009 INFO:teuthology.orchestra.run.smithi071.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-11T06:46:33.019 INFO:teuthology.orchestra.run.smithi176.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-11T06:46:33.020 INFO:teuthology.orchestra.run.smithi071.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-11T06:46:33.049 INFO:teuthology.orchestra.run.smithi176.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-11T06:46:33.055 INFO:teuthology.orchestra.run.smithi071.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-11T06:46:33.058 INFO:teuthology.orchestra.run.smithi176.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-11T06:46:33.059 INFO:teuthology.orchestra.run.smithi089.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-298-gb53b6d3e-1jammy [3,243 kB] 2024-09-11T06:46:33.059 INFO:teuthology.orchestra.run.smithi176.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-11T06:46:33.060 INFO:teuthology.orchestra.run.smithi176.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-11T06:46:33.071 INFO:teuthology.orchestra.run.smithi071.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-11T06:46:33.072 INFO:teuthology.orchestra.run.smithi071.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-11T06:46:33.074 INFO:teuthology.orchestra.run.smithi071.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-11T06:46:33.075 INFO:teuthology.orchestra.run.smithi089.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-11T06:46:33.078 INFO:teuthology.orchestra.run.smithi176.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-11T06:46:33.083 INFO:teuthology.orchestra.run.smithi176.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-11T06:46:33.086 INFO:teuthology.orchestra.run.smithi089.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-11T06:46:33.087 INFO:teuthology.orchestra.run.smithi176.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-11T06:46:33.093 INFO:teuthology.orchestra.run.smithi176.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-11T06:46:33.108 INFO:teuthology.orchestra.run.smithi071.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-11T06:46:33.115 INFO:teuthology.orchestra.run.smithi071.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-11T06:46:33.117 INFO:teuthology.orchestra.run.smithi089.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-11T06:46:33.125 INFO:teuthology.orchestra.run.smithi071.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-11T06:46:33.126 INFO:teuthology.orchestra.run.smithi089.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-11T06:46:33.128 INFO:teuthology.orchestra.run.smithi089.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-11T06:46:33.129 INFO:teuthology.orchestra.run.smithi089.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-11T06:46:33.129 INFO:teuthology.orchestra.run.smithi071.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-11T06:46:33.147 INFO:teuthology.orchestra.run.smithi089.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-11T06:46:33.151 INFO:teuthology.orchestra.run.smithi089.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-11T06:46:33.156 INFO:teuthology.orchestra.run.smithi089.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-11T06:46:33.159 INFO:teuthology.orchestra.run.smithi089.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-11T06:46:33.174 INFO:teuthology.orchestra.run.smithi176.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-11T06:46:33.177 INFO:teuthology.orchestra.run.smithi071.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-11T06:46:33.178 INFO:teuthology.orchestra.run.smithi176.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-11T06:46:33.181 INFO:teuthology.orchestra.run.smithi071.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-11T06:46:33.183 INFO:teuthology.orchestra.run.smithi176.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-11T06:46:33.186 INFO:teuthology.orchestra.run.smithi071.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-11T06:46:33.186 INFO:teuthology.orchestra.run.smithi176.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-11T06:46:33.187 INFO:teuthology.orchestra.run.smithi176.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-11T06:46:33.188 INFO:teuthology.orchestra.run.smithi176.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-11T06:46:33.189 INFO:teuthology.orchestra.run.smithi071.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-11T06:46:33.190 INFO:teuthology.orchestra.run.smithi176.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-11T06:46:33.190 INFO:teuthology.orchestra.run.smithi176.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-11T06:46:33.191 INFO:teuthology.orchestra.run.smithi071.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-11T06:46:33.191 INFO:teuthology.orchestra.run.smithi176.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-11T06:46:33.242 INFO:teuthology.orchestra.run.smithi089.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-11T06:46:33.246 INFO:teuthology.orchestra.run.smithi089.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-11T06:46:33.250 INFO:teuthology.orchestra.run.smithi089.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-11T06:46:33.253 INFO:teuthology.orchestra.run.smithi089.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-11T06:46:33.254 INFO:teuthology.orchestra.run.smithi089.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-11T06:46:33.255 INFO:teuthology.orchestra.run.smithi089.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-11T06:46:33.257 INFO:teuthology.orchestra.run.smithi089.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-11T06:46:33.257 INFO:teuthology.orchestra.run.smithi089.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-11T06:46:33.260 INFO:teuthology.orchestra.run.smithi071.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-11T06:46:33.261 INFO:teuthology.orchestra.run.smithi176.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-11T06:46:33.262 INFO:teuthology.orchestra.run.smithi071.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-11T06:46:33.262 INFO:teuthology.orchestra.run.smithi071.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-11T06:46:33.263 INFO:teuthology.orchestra.run.smithi071.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-11T06:46:33.264 INFO:teuthology.orchestra.run.smithi071.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-11T06:46:33.275 INFO:teuthology.orchestra.run.smithi176.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-11T06:46:33.276 INFO:teuthology.orchestra.run.smithi176.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-11T06:46:33.276 INFO:teuthology.orchestra.run.smithi176.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-11T06:46:33.282 INFO:teuthology.orchestra.run.smithi176.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-11T06:46:33.283 INFO:teuthology.orchestra.run.smithi176.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-11T06:46:33.323 INFO:teuthology.orchestra.run.smithi089.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-11T06:46:33.324 INFO:teuthology.orchestra.run.smithi089.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-11T06:46:33.342 INFO:teuthology.orchestra.run.smithi071.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-11T06:46:33.343 INFO:teuthology.orchestra.run.smithi071.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-11T06:46:33.343 INFO:teuthology.orchestra.run.smithi071.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-11T06:46:33.345 INFO:teuthology.orchestra.run.smithi176.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-11T06:46:33.345 INFO:teuthology.orchestra.run.smithi176.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-11T06:46:33.347 INFO:teuthology.orchestra.run.smithi176.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-11T06:46:33.348 INFO:teuthology.orchestra.run.smithi176.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-11T06:46:33.349 INFO:teuthology.orchestra.run.smithi071.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-11T06:46:33.350 INFO:teuthology.orchestra.run.smithi071.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-11T06:46:33.405 INFO:teuthology.orchestra.run.smithi089.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-11T06:46:33.408 INFO:teuthology.orchestra.run.smithi089.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-11T06:46:33.408 INFO:teuthology.orchestra.run.smithi089.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-11T06:46:33.412 INFO:teuthology.orchestra.run.smithi089.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-11T06:46:33.413 INFO:teuthology.orchestra.run.smithi089.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-11T06:46:33.414 INFO:teuthology.orchestra.run.smithi089.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-11T06:46:33.414 INFO:teuthology.orchestra.run.smithi089.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-11T06:46:33.416 INFO:teuthology.orchestra.run.smithi089.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-11T06:46:33.424 INFO:teuthology.orchestra.run.smithi071.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-11T06:46:33.425 INFO:teuthology.orchestra.run.smithi071.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-11T06:46:33.426 INFO:teuthology.orchestra.run.smithi176.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-11T06:46:33.426 INFO:teuthology.orchestra.run.smithi071.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-11T06:46:33.427 INFO:teuthology.orchestra.run.smithi071.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-11T06:46:33.428 INFO:teuthology.orchestra.run.smithi071.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-11T06:46:33.429 INFO:teuthology.orchestra.run.smithi176.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-11T06:46:33.430 INFO:teuthology.orchestra.run.smithi176.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-11T06:46:33.431 INFO:teuthology.orchestra.run.smithi176.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-11T06:46:33.442 INFO:teuthology.orchestra.run.smithi176.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-11T06:46:33.473 INFO:teuthology.orchestra.run.smithi089.stdout:Get:35 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-298-gb53b6d3e-1jammy [719 kB] 2024-09-11T06:46:33.487 INFO:teuthology.orchestra.run.smithi089.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-11T06:46:33.487 INFO:teuthology.orchestra.run.smithi089.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-11T06:46:33.506 INFO:teuthology.orchestra.run.smithi071.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-11T06:46:33.507 INFO:teuthology.orchestra.run.smithi071.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-11T06:46:33.508 INFO:teuthology.orchestra.run.smithi071.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-11T06:46:33.509 INFO:teuthology.orchestra.run.smithi176.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-11T06:46:33.510 INFO:teuthology.orchestra.run.smithi176.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-11T06:46:33.511 INFO:teuthology.orchestra.run.smithi176.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-11T06:46:33.513 INFO:teuthology.orchestra.run.smithi176.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-11T06:46:33.517 INFO:teuthology.orchestra.run.smithi176.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-11T06:46:33.519 INFO:teuthology.orchestra.run.smithi071.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-11T06:46:33.519 INFO:teuthology.orchestra.run.smithi071.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-11T06:46:33.538 INFO:teuthology.orchestra.run.smithi089.stdout:Get:38 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-298-gb53b6d3e-1jammy [356 kB] 2024-09-11T06:46:33.568 INFO:teuthology.orchestra.run.smithi089.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-11T06:46:33.570 INFO:teuthology.orchestra.run.smithi089.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-11T06:46:33.571 INFO:teuthology.orchestra.run.smithi089.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-11T06:46:33.581 INFO:teuthology.orchestra.run.smithi089.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-11T06:46:33.582 INFO:teuthology.orchestra.run.smithi089.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-11T06:46:33.583 INFO:teuthology.orchestra.run.smithi089.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-11T06:46:33.584 INFO:teuthology.orchestra.run.smithi089.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-11T06:46:33.586 INFO:teuthology.orchestra.run.smithi089.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-11T06:46:33.588 INFO:teuthology.orchestra.run.smithi071.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-11T06:46:33.589 INFO:teuthology.orchestra.run.smithi071.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-11T06:46:33.591 INFO:teuthology.orchestra.run.smithi176.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-11T06:46:33.591 INFO:teuthology.orchestra.run.smithi071.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-11T06:46:33.594 INFO:teuthology.orchestra.run.smithi176.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-11T06:46:33.595 INFO:teuthology.orchestra.run.smithi089.stdout:Get:47 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-298-gb53b6d3e-1jammy [32.8 kB] 2024-09-11T06:46:33.595 INFO:teuthology.orchestra.run.smithi176.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-11T06:46:33.596 INFO:teuthology.orchestra.run.smithi071.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-11T06:46:33.596 INFO:teuthology.orchestra.run.smithi089.stdout:Get:48 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-298-gb53b6d3e-1jammy [180 kB] 2024-09-11T06:46:33.597 INFO:teuthology.orchestra.run.smithi071.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-11T06:46:33.600 INFO:teuthology.orchestra.run.smithi176.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-11T06:46:33.601 INFO:teuthology.orchestra.run.smithi176.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-11T06:46:33.601 INFO:teuthology.orchestra.run.smithi089.stdout:Get:49 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-298-gb53b6d3e-1jammy [67.6 kB] 2024-09-11T06:46:33.603 INFO:teuthology.orchestra.run.smithi089.stdout:Get:50 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-298-gb53b6d3e-1jammy [333 kB] 2024-09-11T06:46:33.650 INFO:teuthology.orchestra.run.smithi089.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-11T06:46:33.652 INFO:teuthology.orchestra.run.smithi089.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-11T06:46:33.659 INFO:teuthology.orchestra.run.smithi089.stdout:Get:53 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-298-gb53b6d3e-1jammy [6,875 kB] 2024-09-11T06:46:33.669 INFO:teuthology.orchestra.run.smithi071.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-11T06:46:33.671 INFO:teuthology.orchestra.run.smithi071.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-11T06:46:33.673 INFO:teuthology.orchestra.run.smithi176.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-11T06:46:33.673 INFO:teuthology.orchestra.run.smithi089.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-11T06:46:33.674 INFO:teuthology.orchestra.run.smithi089.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-11T06:46:33.675 INFO:teuthology.orchestra.run.smithi071.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-11T06:46:33.677 INFO:teuthology.orchestra.run.smithi071.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-11T06:46:33.679 INFO:teuthology.orchestra.run.smithi071.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-11T06:46:33.679 INFO:teuthology.orchestra.run.smithi089.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-11T06:46:33.680 INFO:teuthology.orchestra.run.smithi089.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-11T06:46:33.693 INFO:teuthology.orchestra.run.smithi176.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-11T06:46:33.732 INFO:teuthology.orchestra.run.smithi089.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-11T06:46:33.737 INFO:teuthology.orchestra.run.smithi089.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-11T06:46:33.751 INFO:teuthology.orchestra.run.smithi071.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-11T06:46:33.759 INFO:teuthology.orchestra.run.smithi176.stdout:Get:63 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-298-gb53b6d3e-1jammy [5,145 kB] 2024-09-11T06:46:33.856 INFO:teuthology.orchestra.run.smithi176.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-11T06:46:33.858 INFO:teuthology.orchestra.run.smithi176.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-11T06:46:33.858 INFO:teuthology.orchestra.run.smithi176.stdout:Get:66 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-298-gb53b6d3e-1jammy [245 kB] 2024-09-11T06:46:33.860 INFO:teuthology.orchestra.run.smithi176.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-11T06:46:33.861 INFO:teuthology.orchestra.run.smithi176.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-11T06:46:33.861 INFO:teuthology.orchestra.run.smithi176.stdout:Get:69 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-298-gb53b6d3e-1jammy [124 kB] 2024-09-11T06:46:33.863 INFO:teuthology.orchestra.run.smithi176.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-11T06:46:33.863 INFO:teuthology.orchestra.run.smithi176.stdout:Get:71 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-298-gb53b6d3e-1jammy [1,555 kB] 2024-09-11T06:46:33.885 INFO:teuthology.orchestra.run.smithi071.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-11T06:46:33.886 INFO:teuthology.orchestra.run.smithi071.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-11T06:46:33.889 INFO:teuthology.orchestra.run.smithi071.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-11T06:46:33.889 INFO:teuthology.orchestra.run.smithi071.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-11T06:46:33.891 INFO:teuthology.orchestra.run.smithi071.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-11T06:46:33.900 INFO:teuthology.orchestra.run.smithi089.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-11T06:46:33.901 INFO:teuthology.orchestra.run.smithi089.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-11T06:46:33.904 INFO:teuthology.orchestra.run.smithi089.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-11T06:46:33.904 INFO:teuthology.orchestra.run.smithi089.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-11T06:46:33.907 INFO:teuthology.orchestra.run.smithi089.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-11T06:46:33.925 INFO:teuthology.orchestra.run.smithi176.stdout:Get:72 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-298-gb53b6d3e-1jammy [6,143 kB] 2024-09-11T06:46:33.928 INFO:teuthology.orchestra.run.smithi176.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-11T06:46:33.931 INFO:teuthology.orchestra.run.smithi176.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-11T06:46:33.932 INFO:teuthology.orchestra.run.smithi176.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-11T06:46:33.942 INFO:teuthology.orchestra.run.smithi089.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-11T06:46:33.945 INFO:teuthology.orchestra.run.smithi089.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-11T06:46:33.946 INFO:teuthology.orchestra.run.smithi089.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-11T06:46:33.958 INFO:teuthology.orchestra.run.smithi071.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-11T06:46:33.962 INFO:teuthology.orchestra.run.smithi071.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-11T06:46:33.963 INFO:teuthology.orchestra.run.smithi071.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-11T06:46:34.005 INFO:teuthology.orchestra.run.smithi089.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-11T06:46:34.005 INFO:teuthology.orchestra.run.smithi089.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-11T06:46:34.008 INFO:teuthology.orchestra.run.smithi176.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-11T06:46:34.009 INFO:teuthology.orchestra.run.smithi176.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-11T06:46:34.029 INFO:teuthology.orchestra.run.smithi176.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-11T06:46:34.029 INFO:teuthology.orchestra.run.smithi071.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-11T06:46:34.029 INFO:teuthology.orchestra.run.smithi071.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-11T06:46:34.030 INFO:teuthology.orchestra.run.smithi176.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-11T06:46:34.030 INFO:teuthology.orchestra.run.smithi176.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-11T06:46:34.031 INFO:teuthology.orchestra.run.smithi176.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-11T06:46:34.032 INFO:teuthology.orchestra.run.smithi176.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-11T06:46:34.033 INFO:teuthology.orchestra.run.smithi176.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-11T06:46:34.040 INFO:teuthology.orchestra.run.smithi176.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-11T06:46:34.070 INFO:teuthology.orchestra.run.smithi089.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-11T06:46:34.071 INFO:teuthology.orchestra.run.smithi089.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-11T06:46:34.072 INFO:teuthology.orchestra.run.smithi089.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-11T06:46:34.073 INFO:teuthology.orchestra.run.smithi089.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-11T06:46:34.074 INFO:teuthology.orchestra.run.smithi089.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-11T06:46:34.075 INFO:teuthology.orchestra.run.smithi089.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-11T06:46:34.081 INFO:teuthology.orchestra.run.smithi089.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-11T06:46:34.085 INFO:teuthology.orchestra.run.smithi071.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-11T06:46:34.113 INFO:teuthology.orchestra.run.smithi176.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-11T06:46:34.117 INFO:teuthology.orchestra.run.smithi176.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-11T06:46:34.118 INFO:teuthology.orchestra.run.smithi176.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-11T06:46:34.131 INFO:teuthology.orchestra.run.smithi176.stdout:Get:88 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-298-gb53b6d3e-1jammy [22.8 MB] 2024-09-11T06:46:34.132 INFO:teuthology.orchestra.run.smithi071.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-11T06:46:34.133 INFO:teuthology.orchestra.run.smithi071.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-11T06:46:34.134 INFO:teuthology.orchestra.run.smithi071.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-11T06:46:34.154 INFO:teuthology.orchestra.run.smithi089.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-11T06:46:34.158 INFO:teuthology.orchestra.run.smithi089.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-11T06:46:34.159 INFO:teuthology.orchestra.run.smithi089.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-11T06:46:34.177 INFO:teuthology.orchestra.run.smithi071.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-11T06:46:34.177 INFO:teuthology.orchestra.run.smithi071.stdout:Get:78 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-298-gb53b6d3e-1jammy [5,145 kB] 2024-09-11T06:46:34.178 INFO:teuthology.orchestra.run.smithi071.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-11T06:46:34.186 INFO:teuthology.orchestra.run.smithi071.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-11T06:46:34.190 INFO:teuthology.orchestra.run.smithi071.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-11T06:46:34.194 INFO:teuthology.orchestra.run.smithi071.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-11T06:46:34.195 INFO:teuthology.orchestra.run.smithi071.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-11T06:46:34.197 INFO:teuthology.orchestra.run.smithi176.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-11T06:46:34.204 INFO:teuthology.orchestra.run.smithi176.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-11T06:46:34.207 INFO:teuthology.orchestra.run.smithi176.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-11T06:46:34.208 INFO:teuthology.orchestra.run.smithi176.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.2 [473 kB] 2024-09-11T06:46:34.219 INFO:teuthology.orchestra.run.smithi176.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-11T06:46:34.221 INFO:teuthology.orchestra.run.smithi176.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-11T06:46:34.228 INFO:teuthology.orchestra.run.smithi176.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-11T06:46:34.239 INFO:teuthology.orchestra.run.smithi089.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-11T06:46:34.245 INFO:teuthology.orchestra.run.smithi089.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-11T06:46:34.247 INFO:teuthology.orchestra.run.smithi089.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-11T06:46:34.249 INFO:teuthology.orchestra.run.smithi089.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.2 [473 kB] 2024-09-11T06:46:34.253 INFO:teuthology.orchestra.run.smithi071.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-11T06:46:34.259 INFO:teuthology.orchestra.run.smithi071.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-11T06:46:34.260 INFO:teuthology.orchestra.run.smithi089.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-11T06:46:34.261 INFO:teuthology.orchestra.run.smithi089.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-11T06:46:34.262 INFO:teuthology.orchestra.run.smithi089.stdout:Get:86 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-298-gb53b6d3e-1jammy [112 kB] 2024-09-11T06:46:34.265 INFO:teuthology.orchestra.run.smithi089.stdout:Get:87 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-298-gb53b6d3e-1jammy [472 kB] 2024-09-11T06:46:34.269 INFO:teuthology.orchestra.run.smithi089.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-11T06:46:34.281 INFO:teuthology.orchestra.run.smithi176.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-11T06:46:34.283 INFO:teuthology.orchestra.run.smithi176.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-11T06:46:34.284 INFO:teuthology.orchestra.run.smithi176.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-11T06:46:34.308 INFO:teuthology.orchestra.run.smithi071.stdout:Get:86 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-298-gb53b6d3e-1jammy [245 kB] 2024-09-11T06:46:34.313 INFO:teuthology.orchestra.run.smithi071.stdout:Get:87 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-298-gb53b6d3e-1jammy [124 kB] 2024-09-11T06:46:34.316 INFO:teuthology.orchestra.run.smithi071.stdout:Get:88 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-298-gb53b6d3e-1jammy [1,555 kB] 2024-09-11T06:46:34.319 INFO:teuthology.orchestra.run.smithi089.stdout:Get:89 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-298-gb53b6d3e-1jammy [25.9 MB] 2024-09-11T06:46:34.322 INFO:teuthology.orchestra.run.smithi089.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-11T06:46:34.324 INFO:teuthology.orchestra.run.smithi089.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-11T06:46:34.324 INFO:teuthology.orchestra.run.smithi089.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-11T06:46:34.337 INFO:teuthology.orchestra.run.smithi071.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-11T06:46:34.338 INFO:teuthology.orchestra.run.smithi071.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.2 [473 kB] 2024-09-11T06:46:34.350 INFO:teuthology.orchestra.run.smithi071.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-11T06:46:34.352 INFO:teuthology.orchestra.run.smithi071.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-11T06:46:34.359 INFO:teuthology.orchestra.run.smithi071.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-11T06:46:34.360 INFO:teuthology.orchestra.run.smithi071.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-11T06:46:34.361 INFO:teuthology.orchestra.run.smithi071.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-11T06:46:34.362 INFO:teuthology.orchestra.run.smithi071.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-11T06:46:34.366 INFO:teuthology.orchestra.run.smithi176.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-11T06:46:34.368 INFO:teuthology.orchestra.run.smithi176.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-11T06:46:34.382 INFO:teuthology.orchestra.run.smithi071.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-298-gb53b6d3e-1jammy [6,143 kB] 2024-09-11T06:46:34.387 INFO:teuthology.orchestra.run.smithi176.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-11T06:46:34.388 INFO:teuthology.orchestra.run.smithi176.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-11T06:46:34.388 INFO:teuthology.orchestra.run.smithi176.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-11T06:46:34.394 INFO:teuthology.orchestra.run.smithi176.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-11T06:46:34.395 INFO:teuthology.orchestra.run.smithi176.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-11T06:46:34.407 INFO:teuthology.orchestra.run.smithi089.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-11T06:46:34.409 INFO:teuthology.orchestra.run.smithi089.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-11T06:46:34.422 INFO:teuthology.orchestra.run.smithi071.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-11T06:46:34.424 INFO:teuthology.orchestra.run.smithi071.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-11T06:46:34.425 INFO:teuthology.orchestra.run.smithi089.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-11T06:46:34.426 INFO:teuthology.orchestra.run.smithi089.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-11T06:46:34.427 INFO:teuthology.orchestra.run.smithi089.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-11T06:46:34.432 INFO:teuthology.orchestra.run.smithi089.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-11T06:46:34.434 INFO:teuthology.orchestra.run.smithi089.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-11T06:46:34.507 INFO:teuthology.orchestra.run.smithi071.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-11T06:46:34.508 INFO:teuthology.orchestra.run.smithi071.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-11T06:46:34.509 INFO:teuthology.orchestra.run.smithi071.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-11T06:46:34.514 INFO:teuthology.orchestra.run.smithi071.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-11T06:46:34.515 INFO:teuthology.orchestra.run.smithi071.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-11T06:46:34.664 INFO:teuthology.orchestra.run.smithi071.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-298-gb53b6d3e-1jammy [22.8 MB] 2024-09-11T06:46:34.971 INFO:teuthology.orchestra.run.smithi176.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-298-gb53b6d3e-1jammy [14.1 kB] 2024-09-11T06:46:34.972 INFO:teuthology.orchestra.run.smithi176.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-298-gb53b6d3e-1jammy [877 kB] 2024-09-11T06:46:34.991 INFO:teuthology.orchestra.run.smithi176.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-298-gb53b6d3e-1jammy [2,462 kB] 2024-09-11T06:46:35.032 INFO:teuthology.orchestra.run.smithi176.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-298-gb53b6d3e-1jammy [794 kB] 2024-09-11T06:46:35.044 INFO:teuthology.orchestra.run.smithi176.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-298-gb53b6d3e-1jammy [152 kB] 2024-09-11T06:46:35.046 INFO:teuthology.orchestra.run.smithi176.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-298-gb53b6d3e-1jammy [2,284 kB] 2024-09-11T06:46:35.122 INFO:teuthology.orchestra.run.smithi176.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-298-gb53b6d3e-1jammy [8,625 kB] 2024-09-11T06:46:35.415 INFO:teuthology.orchestra.run.smithi176.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-298-gb53b6d3e-1jammy [14.2 kB] 2024-09-11T06:46:35.415 INFO:teuthology.orchestra.run.smithi176.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-298-gb53b6d3e-1jammy [51.4 MB] 2024-09-11T06:46:35.805 INFO:teuthology.orchestra.run.smithi071.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-298-gb53b6d3e-1jammy [14.1 kB] 2024-09-11T06:46:35.806 INFO:teuthology.orchestra.run.smithi071.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-298-gb53b6d3e-1jammy [877 kB] 2024-09-11T06:46:35.823 INFO:teuthology.orchestra.run.smithi071.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-298-gb53b6d3e-1jammy [2,462 kB] 2024-09-11T06:46:35.879 INFO:teuthology.orchestra.run.smithi071.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-298-gb53b6d3e-1jammy [794 kB] 2024-09-11T06:46:35.911 INFO:teuthology.orchestra.run.smithi071.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-298-gb53b6d3e-1jammy [152 kB] 2024-09-11T06:46:35.915 INFO:teuthology.orchestra.run.smithi071.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-298-gb53b6d3e-1jammy [2,284 kB] 2024-09-11T06:46:36.033 INFO:teuthology.orchestra.run.smithi071.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-298-gb53b6d3e-1jammy [8,625 kB] 2024-09-11T06:46:36.424 INFO:teuthology.orchestra.run.smithi071.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-298-gb53b6d3e-1jammy [14.2 kB] 2024-09-11T06:46:36.426 INFO:teuthology.orchestra.run.smithi071.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-298-gb53b6d3e-1jammy [51.4 MB] 2024-09-11T06:46:37.195 INFO:teuthology.orchestra.run.smithi089.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-298-gb53b6d3e-1jammy [5,145 kB] 2024-09-11T06:46:37.335 INFO:teuthology.orchestra.run.smithi176.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-298-gb53b6d3e-1jammy [130 kB] 2024-09-11T06:46:37.338 INFO:teuthology.orchestra.run.smithi176.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-298-gb53b6d3e-1jammy [39.0 kB] 2024-09-11T06:46:37.339 INFO:teuthology.orchestra.run.smithi176.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-298-gb53b6d3e-1jammy [13.6 MB] 2024-09-11T06:46:37.548 INFO:teuthology.orchestra.run.smithi089.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-298-gb53b6d3e-1jammy [245 kB] 2024-09-11T06:46:37.561 INFO:teuthology.orchestra.run.smithi089.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-298-gb53b6d3e-1jammy [124 kB] 2024-09-11T06:46:37.564 INFO:teuthology.orchestra.run.smithi089.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-298-gb53b6d3e-1jammy [1,555 kB] 2024-09-11T06:46:37.701 INFO:teuthology.orchestra.run.smithi176.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-298-gb53b6d3e-1jammy [91.5 kB] 2024-09-11T06:46:37.702 INFO:teuthology.orchestra.run.smithi176.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 rbd-nbd amd64 19.1.1-298-gb53b6d3e-1jammy [178 kB] 2024-09-11T06:46:37.723 INFO:teuthology.orchestra.run.smithi089.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-298-gb53b6d3e-1jammy [6,143 kB] 2024-09-11T06:46:38.041 INFO:teuthology.orchestra.run.smithi089.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-298-gb53b6d3e-1jammy [22.8 MB] 2024-09-11T06:46:38.280 INFO:teuthology.orchestra.run.smithi176.stdout:Fetched 181 MB in 6s (32.1 MB/s) 2024-09-11T06:46:38.443 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-11T06:46:38.528 INFO:teuthology.orchestra.run.smithi071.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-298-gb53b6d3e-1jammy [130 kB] 2024-09-11T06:46:38.531 INFO:teuthology.orchestra.run.smithi071.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-298-gb53b6d3e-1jammy [39.0 kB] 2024-09-11T06:46:38.532 INFO:teuthology.orchestra.run.smithi071.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-298-gb53b6d3e-1jammy [13.6 MB] 2024-09-11T06:46:38.543 INFO:teuthology.orchestra.run.smithi176.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165988 files and directories currently installed.) 2024-09-11T06:46:38.549 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-11T06:46:38.633 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-11T06:46:38.785 INFO:teuthology.orchestra.run.smithi071.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-298-gb53b6d3e-1jammy [91.5 kB] 2024-09-11T06:46:38.786 INFO:teuthology.orchestra.run.smithi071.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 rbd-nbd amd64 19.1.1-298-gb53b6d3e-1jammy [178 kB] 2024-09-11T06:46:38.920 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-11T06:46:38.946 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-11T06:46:38.978 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-11T06:46:39.085 INFO:teuthology.orchestra.run.smithi089.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-298-gb53b6d3e-1jammy [14.1 kB] 2024-09-11T06:46:39.086 INFO:teuthology.orchestra.run.smithi089.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-298-gb53b6d3e-1jammy [877 kB] 2024-09-11T06:46:39.103 INFO:teuthology.orchestra.run.smithi089.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-298-gb53b6d3e-1jammy [2,462 kB] 2024-09-11T06:46:39.176 INFO:teuthology.orchestra.run.smithi089.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-298-gb53b6d3e-1jammy [794 kB] 2024-09-11T06:46:39.200 INFO:teuthology.orchestra.run.smithi089.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-298-gb53b6d3e-1jammy [152 kB] 2024-09-11T06:46:39.203 INFO:teuthology.orchestra.run.smithi089.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-298-gb53b6d3e-1jammy [2,284 kB] 2024-09-11T06:46:39.281 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-11T06:46:39.283 INFO:teuthology.orchestra.run.smithi089.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-298-gb53b6d3e-1jammy [8,625 kB] 2024-09-11T06:46:39.308 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-11T06:46:39.348 INFO:teuthology.orchestra.run.smithi071.stdout:Fetched 181 MB in 7s (27.0 MB/s) 2024-09-11T06:46:39.348 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-11T06:46:39.477 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-11T06:46:39.575 INFO:teuthology.orchestra.run.smithi071.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165988 files and directories currently installed.) 2024-09-11T06:46:39.577 INFO:teuthology.orchestra.run.smithi089.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-298-gb53b6d3e-1jammy [14.2 kB] 2024-09-11T06:46:39.577 INFO:teuthology.orchestra.run.smithi089.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-298-gb53b6d3e-1jammy [51.4 MB] 2024-09-11T06:46:39.580 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-11T06:46:39.667 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-11T06:46:39.801 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-11T06:46:39.828 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-11T06:46:39.876 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T06:46:39.971 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-11T06:46:39.997 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-11T06:46:40.037 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-11T06:46:40.238 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-11T06:46:40.264 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-11T06:46:40.296 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T06:46:40.382 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-11T06:46:40.408 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-11T06:46:40.449 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-11T06:46:40.649 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-11T06:46:40.675 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-11T06:46:40.715 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T06:46:40.894 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-11T06:46:40.920 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-11T06:46:40.969 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T06:46:41.060 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-11T06:46:41.087 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-11T06:46:41.127 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-11T06:46:41.322 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-11T06:46:41.348 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-11T06:46:41.388 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T06:46:41.413 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package librados2. 2024-09-11T06:46:41.428 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../007-librados2_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:41.463 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking librados2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:41.576 INFO:teuthology.orchestra.run.smithi089.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-298-gb53b6d3e-1jammy [130 kB] 2024-09-11T06:46:41.579 INFO:teuthology.orchestra.run.smithi089.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-298-gb53b6d3e-1jammy [39.0 kB] 2024-09-11T06:46:41.580 INFO:teuthology.orchestra.run.smithi089.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-298-gb53b6d3e-1jammy [13.6 MB] 2024-09-11T06:46:41.733 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-11T06:46:41.760 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-11T06:46:41.799 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T06:46:41.929 INFO:teuthology.orchestra.run.smithi089.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-298-gb53b6d3e-1jammy [91.5 kB] 2024-09-11T06:46:41.935 INFO:teuthology.orchestra.run.smithi089.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 rbd-nbd amd64 19.1.1-298-gb53b6d3e-1jammy [178 kB] 2024-09-11T06:46:41.957 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package librbd1. 2024-09-11T06:46:41.984 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../008-librbd1_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:42.026 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking librbd1 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:42.153 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-11T06:46:42.179 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-11T06:46:42.219 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-11T06:46:42.485 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libcephfs2. 2024-09-11T06:46:42.488 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package librados2. 2024-09-11T06:46:42.504 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../007-librados2_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:42.510 INFO:teuthology.orchestra.run.smithi089.stdout:Fetched 181 MB in 10s (18.5 MB/s) 2024-09-11T06:46:42.512 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../009-libcephfs2_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:42.547 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking librados2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:42.551 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libcephfs2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:42.660 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-11T06:46:42.758 INFO:teuthology.orchestra.run.smithi089.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165988 files and directories currently installed.) 2024-09-11T06:46:42.764 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-11T06:46:42.829 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-rados. 2024-09-11T06:46:42.833 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-11T06:46:42.845 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../010-python3-rados_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:42.879 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-rados (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:43.000 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package librbd1. 2024-09-11T06:46:43.026 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../008-librbd1_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:43.066 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking librbd1 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:43.173 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-11T06:46:43.187 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-11T06:46:43.200 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-11T06:46:43.213 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-11T06:46:43.253 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-11T06:46:43.258 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-ceph-argparse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:43.494 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libcephfs2. 2024-09-11T06:46:43.509 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-cephfs. 2024-09-11T06:46:43.520 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../009-libcephfs2_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:43.525 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../012-python3-cephfs_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:43.539 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-11T06:46:43.560 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libcephfs2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:43.565 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-11T06:46:43.567 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-cephfs (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:43.631 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-11T06:46:43.838 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-rados. 2024-09-11T06:46:43.864 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../010-python3-rados_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:43.870 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-11T06:46:43.897 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../013-python3-ceph-common_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-11T06:46:43.929 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-rados (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:43.936 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-ceph-common (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:44.068 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-11T06:46:44.094 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-11T06:46:44.143 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T06:46:44.224 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-11T06:46:44.251 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-11T06:46:44.291 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-ceph-argparse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:44.384 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-11T06:46:44.411 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-11T06:46:44.526 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-cephfs. 2024-09-11T06:46:44.539 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-11T06:46:44.546 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-11T06:46:44.552 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../012-python3-cephfs_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:44.572 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-11T06:46:44.593 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-cephfs (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:44.613 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T06:46:44.817 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-prettytable. 2024-09-11T06:46:44.844 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-09-11T06:46:44.875 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-11T06:46:44.882 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-11T06:46:44.909 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-11T06:46:44.932 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-11T06:46:44.949 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T06:46:44.958 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../013-python3-ceph-common_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-11T06:46:45.020 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-ceph-common (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:45.136 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-rbd. 2024-09-11T06:46:45.152 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../016-python3-rbd_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:45.194 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-rbd (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:45.294 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-11T06:46:45.317 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-11T06:46:45.360 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-11T06:46:45.374 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-11T06:46:45.400 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-11T06:46:45.440 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-11T06:46:45.539 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-11T06:46:45.566 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-11T06:46:45.614 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-11T06:46:45.710 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-prettytable. 2024-09-11T06:46:45.722 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package librados2. 2024-09-11T06:46:45.736 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-09-11T06:46:45.748 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../007-librados2_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:45.768 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-11T06:46:45.788 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking librados2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:45.892 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-11T06:46:45.907 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-11T06:46:45.942 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-11T06:46:46.012 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-rbd. 2024-09-11T06:46:46.027 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../016-python3-rbd_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:46.070 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-rbd (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:46.208 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package librbd1. 2024-09-11T06:46:46.219 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-11T06:46:46.223 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../008-librbd1_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:46.235 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-11T06:46:46.258 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking librbd1 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:46.269 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-11T06:46:46.415 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-11T06:46:46.441 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-11T06:46:46.481 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-11T06:46:46.631 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package lua5.1. 2024-09-11T06:46:46.657 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-11T06:46:46.685 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libcephfs2. 2024-09-11T06:46:46.706 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-11T06:46:46.712 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../009-libcephfs2_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:46.752 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libcephfs2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:46.767 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-11T06:46:46.782 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-11T06:46:46.817 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-11T06:46:47.025 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package lua-any. 2024-09-11T06:46:47.041 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-09-11T06:46:47.055 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-rados. 2024-09-11T06:46:47.078 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-11T06:46:47.081 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../010-python3-rados_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:47.084 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-11T06:46:47.093 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-11T06:46:47.121 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-rados (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:47.136 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-11T06:46:47.303 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package zip. 2024-09-11T06:46:47.318 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-09-11T06:46:47.353 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking zip (3.0-12build2) ... 2024-09-11T06:46:47.441 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-11T06:46:47.448 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package lua5.1. 2024-09-11T06:46:47.467 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-11T06:46:47.474 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-11T06:46:47.507 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-ceph-argparse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:47.514 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-11T06:46:47.639 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package unzip. 2024-09-11T06:46:47.666 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-11T06:46:47.714 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-11T06:46:47.735 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-cephfs. 2024-09-11T06:46:47.750 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../012-python3-cephfs_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:47.785 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-cephfs (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:47.834 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package lua-any. 2024-09-11T06:46:47.861 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-09-11T06:46:47.901 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-11T06:46:48.088 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-11T06:46:48.093 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package luarocks. 2024-09-11T06:46:48.112 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package zip. 2024-09-11T06:46:48.114 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../013-python3-ceph-common_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-11T06:46:48.120 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-11T06:46:48.127 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-09-11T06:46:48.154 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-ceph-common (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:48.159 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-11T06:46:48.162 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking zip (3.0-12build2) ... 2024-09-11T06:46:48.473 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package unzip. 2024-09-11T06:46:48.474 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-11T06:46:48.499 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-11T06:46:48.500 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-11T06:46:48.536 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package librgw2. 2024-09-11T06:46:48.540 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-11T06:46:48.548 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-11T06:46:48.563 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../025-librgw2_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:48.602 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking librgw2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:48.860 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-prettytable. 2024-09-11T06:46:48.886 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-09-11T06:46:48.926 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-11T06:46:48.935 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package luarocks. 2024-09-11T06:46:48.961 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-11T06:46:49.002 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-11T06:46:49.146 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-rgw. 2024-09-11T06:46:49.173 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../026-python3-rgw_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:49.188 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-rbd. 2024-09-11T06:46:49.213 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-rgw (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:49.214 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../016-python3-rbd_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:49.254 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-rbd (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:49.368 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package librgw2. 2024-09-11T06:46:49.383 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../025-librgw2_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:49.426 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking librgw2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:49.549 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-11T06:46:49.576 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-11T06:46:49.615 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-11T06:46:49.824 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-11T06:46:49.850 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-11T06:46:49.891 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-11T06:46:50.145 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-rgw. 2024-09-11T06:46:50.160 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-11T06:46:50.172 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../026-python3-rgw_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:50.186 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-11T06:46:50.212 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-rgw (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:50.226 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-11T06:46:50.235 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libradosstriper1. 2024-09-11T06:46:50.262 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../028-libradosstriper1_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:50.302 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libradosstriper1 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:50.515 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-11T06:46:50.532 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-11T06:46:50.540 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-11T06:46:50.558 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-11T06:46:50.573 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-11T06:46:50.598 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-11T06:46:50.747 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-common. 2024-09-11T06:46:50.774 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../029-ceph-common_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:50.813 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-common (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:50.877 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package lua5.1. 2024-09-11T06:46:50.884 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libradosstriper1. 2024-09-11T06:46:50.902 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-11T06:46:50.911 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../028-libradosstriper1_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:50.943 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-11T06:46:50.951 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libradosstriper1 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:51.254 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package lua-any. 2024-09-11T06:46:51.269 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-09-11T06:46:51.313 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-11T06:46:51.321 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-common. 2024-09-11T06:46:51.347 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../029-ceph-common_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:51.387 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-common (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:51.574 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package zip. 2024-09-11T06:46:51.589 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-09-11T06:46:51.632 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking zip (3.0-12build2) ... 2024-09-11T06:46:51.902 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package unzip. 2024-09-11T06:46:51.917 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-11T06:46:51.952 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-11T06:46:52.135 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-base. 2024-09-11T06:46:52.161 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../030-ceph-base_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:52.222 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-base (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:52.373 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package luarocks. 2024-09-11T06:46:52.399 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-11T06:46:52.439 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-11T06:46:52.773 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-11T06:46:52.799 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-11T06:46:52.821 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package librgw2. 2024-09-11T06:46:52.830 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-11T06:46:52.848 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../025-librgw2_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:52.876 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-base. 2024-09-11T06:46:52.888 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking librgw2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:52.902 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../030-ceph-base_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:52.952 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-base (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:53.117 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-cheroot. 2024-09-11T06:46:53.143 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-11T06:46:53.183 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-11T06:46:53.466 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-rgw. 2024-09-11T06:46:53.481 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../026-python3-rgw_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:53.503 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-11T06:46:53.524 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-rgw (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:53.530 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-11T06:46:53.555 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-11T06:46:53.569 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-11T06:46:53.581 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-11T06:46:53.621 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-11T06:46:53.810 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-11T06:46:53.836 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-11T06:46:53.868 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-11T06:46:53.872 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-11T06:46:53.898 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-11T06:46:53.938 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-11T06:46:53.941 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-cheroot. 2024-09-11T06:46:53.967 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-11T06:46:54.007 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-11T06:46:54.183 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-11T06:46:54.196 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libradosstriper1. 2024-09-11T06:46:54.206 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-11T06:46:54.223 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../028-libradosstriper1_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:54.241 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-11T06:46:54.271 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libradosstriper1 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:54.310 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-11T06:46:54.326 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-11T06:46:54.360 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-11T06:46:54.544 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-tempora. 2024-09-11T06:46:54.571 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-09-11T06:46:54.596 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-11T06:46:54.610 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-11T06:46:54.622 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-11T06:46:54.663 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-11T06:46:54.725 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-common. 2024-09-11T06:46:54.751 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../029-ceph-common_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:54.791 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-common (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:54.949 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-11T06:46:54.976 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-11T06:46:55.016 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-11T06:46:55.122 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-portend. 2024-09-11T06:46:55.148 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-09-11T06:46:55.188 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-11T06:46:55.318 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-tempora. 2024-09-11T06:46:55.345 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-09-11T06:46:55.385 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-11T06:46:55.449 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-11T06:46:55.464 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-11T06:46:55.499 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-11T06:46:55.671 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-portend. 2024-09-11T06:46:55.696 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-09-11T06:46:55.738 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-11T06:46:55.844 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-11T06:46:55.870 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-11T06:46:55.910 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-11T06:46:55.999 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-11T06:46:56.015 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-11T06:46:56.057 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-11T06:46:56.255 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-natsort. 2024-09-11T06:46:56.263 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-base. 2024-09-11T06:46:56.282 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-09-11T06:46:56.289 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../030-ceph-base_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:46:56.322 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-11T06:46:56.339 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-base (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:46:56.360 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-11T06:46:56.386 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-11T06:46:56.426 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-11T06:46:56.633 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-logutils. 2024-09-11T06:46:56.659 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-09-11T06:46:56.708 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-11T06:46:56.780 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-natsort. 2024-09-11T06:46:56.806 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-09-11T06:46:56.846 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-11T06:46:56.901 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-11T06:46:56.924 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-11T06:46:56.959 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-11T06:46:56.994 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-11T06:46:57.021 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-11T06:46:57.061 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-11T06:46:57.166 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-logutils. 2024-09-11T06:46:57.193 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-09-11T06:46:57.233 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-11T06:46:57.287 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-cheroot. 2024-09-11T06:46:57.313 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-11T06:46:57.322 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-mako. 2024-09-11T06:46:57.348 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-11T06:46:57.357 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-11T06:46:57.388 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-11T06:46:57.502 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-11T06:46:57.519 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-11T06:46:57.552 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-11T06:46:57.682 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-11T06:46:57.708 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-11T06:46:57.708 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-11T06:46:57.734 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-11T06:46:57.748 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-11T06:46:57.774 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-11T06:46:57.880 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-mako. 2024-09-11T06:46:57.906 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-11T06:46:57.947 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-11T06:46:58.035 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-11T06:46:58.061 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-11T06:46:58.077 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-11T06:46:58.101 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-11T06:46:58.104 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-11T06:46:58.144 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-11T06:46:58.241 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-11T06:46:58.268 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-11T06:46:58.308 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-11T06:46:58.371 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-11T06:46:58.386 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-11T06:46:58.413 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-webob. 2024-09-11T06:46:58.420 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-11T06:46:58.439 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-11T06:46:58.471 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-11T06:46:58.577 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-11T06:46:58.604 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-11T06:46:58.644 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-11T06:46:58.699 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-tempora. 2024-09-11T06:46:58.725 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-09-11T06:46:58.765 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-11T06:46:58.783 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-waitress. 2024-09-11T06:46:58.809 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-11T06:46:58.843 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-11T06:46:58.947 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-webob. 2024-09-11T06:46:58.973 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-11T06:46:59.013 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-11T06:46:59.026 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-portend. 2024-09-11T06:46:59.052 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-09-11T06:46:59.084 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-11T06:46:59.177 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-tempita. 2024-09-11T06:46:59.203 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-11T06:46:59.244 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-11T06:46:59.358 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-waitress. 2024-09-11T06:46:59.371 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-11T06:46:59.384 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-11T06:46:59.397 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-11T06:46:59.428 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-11T06:46:59.437 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-11T06:46:59.546 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-paste. 2024-09-11T06:46:59.573 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-11T06:46:59.613 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-11T06:46:59.711 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-tempita. 2024-09-11T06:46:59.737 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-11T06:46:59.740 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-11T06:46:59.767 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-11T06:46:59.777 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-11T06:46:59.807 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-11T06:46:59.932 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-11T06:46:59.948 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-11T06:46:59.991 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-11T06:47:00.088 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-paste. 2024-09-11T06:47:00.115 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-11T06:47:00.155 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-11T06:47:00.369 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-natsort. 2024-09-11T06:47:00.395 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-09-11T06:47:00.435 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-11T06:47:00.467 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-11T06:47:00.469 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-11T06:47:00.493 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-11T06:47:00.495 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-11T06:47:00.533 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-11T06:47:00.535 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-11T06:47:00.747 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-logutils. 2024-09-11T06:47:00.773 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-09-11T06:47:00.813 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-11T06:47:00.813 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-webtest. 2024-09-11T06:47:00.819 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-11T06:47:00.840 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-09-11T06:47:00.845 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-11T06:47:00.880 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-11T06:47:00.885 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-11T06:47:01.133 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-11T06:47:01.149 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-pecan. 2024-09-11T06:47:01.160 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-11T06:47:01.176 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-11T06:47:01.188 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-webtest. 2024-09-11T06:47:01.199 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-11T06:47:01.207 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-11T06:47:01.215 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-09-11T06:47:01.255 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-11T06:47:01.502 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-mako. 2024-09-11T06:47:01.512 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-certifi. 2024-09-11T06:47:01.528 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-11T06:47:01.539 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-09-11T06:47:01.569 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-11T06:47:01.570 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-11T06:47:01.591 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-pecan. 2024-09-11T06:47:01.618 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-11T06:47:01.658 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-11T06:47:01.880 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-11T06:47:01.907 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-11T06:47:01.923 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-idna. 2024-09-11T06:47:01.947 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-11T06:47:01.949 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-11T06:47:01.989 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-11T06:47:02.012 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-certifi. 2024-09-11T06:47:02.039 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-09-11T06:47:02.079 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-11T06:47:02.250 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-11T06:47:02.276 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-11T06:47:02.293 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-urllib3. 2024-09-11T06:47:02.316 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-11T06:47:02.319 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-11T06:47:02.359 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-11T06:47:02.398 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-idna. 2024-09-11T06:47:02.425 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-11T06:47:02.467 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-11T06:47:02.594 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-webob. 2024-09-11T06:47:02.621 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-11T06:47:02.661 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-11T06:47:02.687 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-requests. 2024-09-11T06:47:02.702 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-11T06:47:02.745 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-11T06:47:02.787 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-urllib3. 2024-09-11T06:47:02.814 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-11T06:47:02.853 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-11T06:47:02.964 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-waitress. 2024-09-11T06:47:02.990 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-11T06:47:03.025 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-11T06:47:03.056 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-11T06:47:03.083 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-11T06:47:03.123 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-11T06:47:03.182 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-requests. 2024-09-11T06:47:03.209 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-11T06:47:03.248 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-11T06:47:03.275 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-tempita. 2024-09-11T06:47:03.302 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-11T06:47:03.359 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-11T06:47:03.418 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-11T06:47:03.444 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-11T06:47:03.484 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-mgr-modules-core (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:03.535 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-11T06:47:03.550 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-11T06:47:03.585 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-11T06:47:03.661 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-paste. 2024-09-11T06:47:03.688 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-11T06:47:03.728 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-11T06:47:03.872 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-11T06:47:03.872 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-11T06:47:03.898 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-11T06:47:03.899 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:03.938 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-mgr-modules-core (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:03.938 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:04.014 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-11T06:47:04.029 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-11T06:47:04.073 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-11T06:47:04.258 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-11T06:47:04.285 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-11T06:47:04.318 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-11T06:47:04.324 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-11T06:47:04.345 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:04.376 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-11T06:47:04.402 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-11T06:47:04.442 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-11T06:47:04.627 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-openssl. 2024-09-11T06:47:04.654 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-09-11T06:47:04.677 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:04.694 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-11T06:47:04.745 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-webtest. 2024-09-11T06:47:04.772 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-09-11T06:47:04.837 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-11T06:47:04.997 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-mgr. 2024-09-11T06:47:05.024 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../063-ceph-mgr_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:05.055 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-mgr (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:05.106 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-pecan. 2024-09-11T06:47:05.133 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-11T06:47:05.173 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-11T06:47:05.467 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-mon. 2024-09-11T06:47:05.494 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../064-ceph-mon_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:05.714 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-11T06:47:05.740 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-11T06:47:05.780 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-11T06:47:05.791 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-mon (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:06.058 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-openssl. 2024-09-11T06:47:06.084 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-09-11T06:47:06.125 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-11T06:47:06.345 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-certifi. 2024-09-11T06:47:06.372 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-09-11T06:47:06.411 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-11T06:47:06.470 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-mgr. 2024-09-11T06:47:06.496 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../063-ceph-mgr_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:06.536 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-mgr (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:06.714 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-idna. 2024-09-11T06:47:06.741 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-11T06:47:06.762 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-osd. 2024-09-11T06:47:06.789 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../065-ceph-osd_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:06.789 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-11T06:47:06.820 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-osd (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:06.965 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-mon. 2024-09-11T06:47:06.991 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../064-ceph-mon_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:07.031 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-mon (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:07.101 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-urllib3. 2024-09-11T06:47:07.127 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-11T06:47:07.167 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-11T06:47:07.496 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-requests. 2024-09-11T06:47:07.522 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-11T06:47:07.562 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-11T06:47:07.627 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-osd. 2024-09-11T06:47:07.653 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../065-ceph-osd_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:07.693 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-osd (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:07.873 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-11T06:47:07.900 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-11T06:47:07.923 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph. 2024-09-11T06:47:07.938 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../066-ceph_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:07.940 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-11T06:47:07.981 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:08.226 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-11T06:47:08.241 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-11T06:47:08.284 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-mgr-modules-core (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:08.309 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-fuse. 2024-09-11T06:47:08.335 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../067-ceph-fuse_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:08.384 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-fuse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:08.589 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-11T06:47:08.616 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:08.673 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:08.712 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-mds. 2024-09-11T06:47:08.738 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../068-ceph-mds_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:08.770 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-mds (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:08.771 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph. 2024-09-11T06:47:08.785 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../066-ceph_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:08.821 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:08.976 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-11T06:47:09.002 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-11T06:47:09.042 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-11T06:47:09.141 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-fuse. 2024-09-11T06:47:09.148 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package cephadm. 2024-09-11T06:47:09.167 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../067-ceph-fuse_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:09.174 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../069-cephadm_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:09.207 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-fuse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:09.214 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking cephadm (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:09.345 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-openssl. 2024-09-11T06:47:09.372 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-09-11T06:47:09.411 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-11T06:47:09.525 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-11T06:47:09.552 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-11T06:47:09.578 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-mds. 2024-09-11T06:47:09.591 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-11T06:47:09.605 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../068-ceph-mds_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:09.644 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-mds (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:09.698 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-mgr. 2024-09-11T06:47:09.724 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../063-ceph-mgr_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:09.756 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-mgr (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:09.945 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-11T06:47:09.972 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-11T06:47:10.006 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package cephadm. 2024-09-11T06:47:10.019 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-11T06:47:10.021 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../069-cephadm_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:10.064 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking cephadm (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:10.135 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-mon. 2024-09-11T06:47:10.161 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../064-ceph-mon_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:10.201 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-mon (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:10.392 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-11T06:47:10.419 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-11T06:47:10.467 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-11T06:47:10.829 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-11T06:47:10.856 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-11T06:47:10.887 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-11T06:47:10.955 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-osd. 2024-09-11T06:47:10.981 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../065-ceph-osd_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:11.021 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-osd (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:11.291 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-babel. 2024-09-11T06:47:11.315 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-11T06:47:11.349 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-11T06:47:11.711 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-jinja2. 2024-09-11T06:47:11.737 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-11T06:47:11.777 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-11T06:47:11.917 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-babel. 2024-09-11T06:47:11.944 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-11T06:47:11.983 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-11T06:47:12.066 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph. 2024-09-11T06:47:12.082 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../066-ceph_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:12.113 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-11T06:47:12.124 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:12.129 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-11T06:47:12.163 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-mgr-cephadm (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:12.312 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-jinja2. 2024-09-11T06:47:12.338 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-11T06:47:12.378 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-11T06:47:12.402 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-fuse. 2024-09-11T06:47:12.428 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../067-ceph-fuse_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:12.460 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-fuse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:12.501 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-11T06:47:12.526 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-09-11T06:47:12.566 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-11T06:47:12.732 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-11T06:47:12.758 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-11T06:47:12.798 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-mgr-cephadm (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:12.830 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-mds. 2024-09-11T06:47:12.844 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-routes. 2024-09-11T06:47:12.856 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../068-ceph-mds_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:12.870 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-11T06:47:12.896 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-mds (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:12.902 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-11T06:47:13.119 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-11T06:47:13.145 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-09-11T06:47:13.185 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-11T06:47:13.222 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-11T06:47:13.248 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-11T06:47:13.280 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-mgr-dashboard (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:13.299 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package cephadm. 2024-09-11T06:47:13.326 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../069-cephadm_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:13.366 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking cephadm (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:13.488 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-routes. 2024-09-11T06:47:13.515 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-11T06:47:13.554 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-11T06:47:13.685 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-11T06:47:13.712 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-11T06:47:13.752 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-11T06:47:13.883 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-11T06:47:13.909 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-11T06:47:13.949 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-mgr-dashboard (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:14.038 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-11T06:47:14.053 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-11T06:47:14.088 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-11T06:47:14.535 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-11T06:47:14.551 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-11T06:47:14.593 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-11T06:47:15.054 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-joblib. 2024-09-11T06:47:15.070 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-11T06:47:15.084 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-babel. 2024-09-11T06:47:15.100 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-11T06:47:15.104 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-11T06:47:15.134 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-11T06:47:15.196 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-11T06:47:15.213 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-11T06:47:15.255 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-11T06:47:15.421 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-jinja2. 2024-09-11T06:47:15.448 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-11T06:47:15.466 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-11T06:47:15.479 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-11T06:47:15.493 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-11T06:47:15.532 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-11T06:47:15.758 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-joblib. 2024-09-11T06:47:15.765 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-11T06:47:15.785 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-11T06:47:15.792 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-11T06:47:15.825 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-11T06:47:15.832 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-mgr-cephadm (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:15.852 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-sklearn. 2024-09-11T06:47:15.879 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-11T06:47:15.918 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-11T06:47:16.143 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-11T06:47:16.159 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-09-11T06:47:16.193 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-11T06:47:16.229 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-11T06:47:16.256 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-11T06:47:16.295 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-11T06:47:16.513 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-routes. 2024-09-11T06:47:16.539 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-11T06:47:16.579 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-11T06:47:16.581 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-sklearn. 2024-09-11T06:47:16.608 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-11T06:47:16.639 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-11T06:47:16.685 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-11T06:47:16.701 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-11T06:47:16.820 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:16.874 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-11T06:47:16.901 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-11T06:47:16.941 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-mgr-dashboard (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:17.357 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-11T06:47:17.385 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-11T06:47:17.510 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:17.790 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-cachetools. 2024-09-11T06:47:17.817 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-09-11T06:47:17.856 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-11T06:47:18.151 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-11T06:47:18.178 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-11T06:47:18.204 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-11T06:47:18.217 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-11T06:47:18.218 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-11T06:47:18.254 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-11T06:47:18.422 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-cachetools. 2024-09-11T06:47:18.449 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-09-11T06:47:18.488 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-11T06:47:18.513 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-11T06:47:18.540 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-11T06:47:18.571 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-11T06:47:18.750 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-joblib. 2024-09-11T06:47:18.774 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-11T06:47:18.777 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-11T06:47:18.801 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-11T06:47:18.816 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-11T06:47:18.841 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-11T06:47:18.857 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-rsa. 2024-09-11T06:47:18.884 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-09-11T06:47:18.915 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-11T06:47:19.170 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-11T06:47:19.186 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-11T06:47:19.197 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-11T06:47:19.213 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-11T06:47:19.236 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-11T06:47:19.252 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-google-auth. 2024-09-11T06:47:19.253 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-11T06:47:19.279 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-09-11T06:47:19.327 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-11T06:47:19.523 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-rsa. 2024-09-11T06:47:19.550 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-09-11T06:47:19.572 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-sklearn. 2024-09-11T06:47:19.589 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-11T06:47:19.599 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-11T06:47:19.621 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-11T06:47:19.637 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-11T06:47:19.638 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-11T06:47:19.671 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-11T06:47:19.910 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-google-auth. 2024-09-11T06:47:19.924 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-websocket. 2024-09-11T06:47:19.937 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-09-11T06:47:19.951 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-09-11T06:47:19.968 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-11T06:47:19.982 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-11T06:47:20.289 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-11T06:47:20.316 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-11T06:47:20.327 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-11T06:47:20.355 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-11T06:47:20.355 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-11T06:47:20.364 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-11T06:47:20.391 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-11T06:47:20.394 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-11T06:47:20.508 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:20.667 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-websocket. 2024-09-11T06:47:20.694 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-09-11T06:47:20.733 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-11T06:47:21.074 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-11T06:47:21.090 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-11T06:47:21.165 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:21.254 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-11T06:47:21.281 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-11T06:47:21.320 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-11T06:47:21.445 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-cachetools. 2024-09-11T06:47:21.473 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-09-11T06:47:21.511 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-11T06:47:21.693 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-11T06:47:21.721 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-11T06:47:21.760 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-11T06:47:21.822 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-11T06:47:21.849 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-11T06:47:21.889 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-11T06:47:21.985 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-11T06:47:22.013 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-11T06:47:22.051 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:22.063 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-11T06:47:22.090 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-11T06:47:22.121 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-11T06:47:22.193 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-11T06:47:22.208 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-11T06:47:22.242 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-11T06:47:22.346 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-11T06:47:22.373 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-11T06:47:22.382 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package jq. 2024-09-11T06:47:22.404 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-11T06:47:22.409 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-11T06:47:22.448 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-11T06:47:22.504 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-rsa. 2024-09-11T06:47:22.531 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-09-11T06:47:22.562 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-11T06:47:22.710 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package socat. 2024-09-11T06:47:22.724 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-11T06:47:22.737 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-11T06:47:22.751 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-11T06:47:22.776 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-11T06:47:22.790 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-11T06:47:22.891 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-google-auth. 2024-09-11T06:47:22.918 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-09-11T06:47:22.957 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-11T06:47:23.060 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package jq. 2024-09-11T06:47:23.079 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package xmlstarlet. 2024-09-11T06:47:23.087 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-11T06:47:23.095 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-11T06:47:23.126 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-11T06:47:23.137 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-11T06:47:23.286 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-11T06:47:23.313 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-11T06:47:23.352 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-11T06:47:23.363 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package socat. 2024-09-11T06:47:23.378 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-11T06:47:23.412 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-11T06:47:23.525 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-test. 2024-09-11T06:47:23.540 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../097-ceph-test_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:23.574 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-test (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:23.663 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-websocket. 2024-09-11T06:47:23.683 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package xmlstarlet. 2024-09-11T06:47:23.691 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-09-11T06:47:23.699 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-11T06:47:23.730 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-11T06:47:23.741 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-11T06:47:24.033 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-11T06:47:24.061 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-11T06:47:24.091 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-11T06:47:24.104 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-test. 2024-09-11T06:47:24.120 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../097-ceph-test_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:24.162 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-test (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:24.756 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-11T06:47:24.784 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-11T06:47:24.822 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:25.158 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-11T06:47:25.186 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-11T06:47:25.224 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-11T06:47:25.536 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-11T06:47:25.552 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-11T06:47:25.585 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-11T06:47:25.904 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-volume. 2024-09-11T06:47:25.919 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../098-ceph-volume_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-11T06:47:25.953 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-volume (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:25.955 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package jq. 2024-09-11T06:47:25.983 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-11T06:47:26.024 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-11T06:47:26.299 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-11T06:47:26.327 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../099-libcephfs-dev_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:26.365 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libcephfs-dev (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:26.500 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package socat. 2024-09-11T06:47:26.527 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-11T06:47:26.543 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-volume. 2024-09-11T06:47:26.566 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-11T06:47:26.569 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../098-ceph-volume_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-11T06:47:26.609 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-volume (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:26.644 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-11T06:47:26.671 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-11T06:47:26.710 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-11T06:47:26.886 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package xmlstarlet. 2024-09-11T06:47:26.902 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-11T06:47:26.913 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-11T06:47:26.940 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../099-libcephfs-dev_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:26.944 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-11T06:47:26.979 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libcephfs-dev (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:27.014 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-11T06:47:27.042 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-09-11T06:47:27.081 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-11T06:47:27.257 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-11T06:47:27.285 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-11T06:47:27.290 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-test. 2024-09-11T06:47:27.315 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-11T06:47:27.318 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../097-ceph-test_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:27.356 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-test (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:27.393 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package nvme-cli. 2024-09-11T06:47:27.421 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.2_amd64.deb ... 2024-09-11T06:47:27.459 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking nvme-cli (1.16-3ubuntu0.2) ... 2024-09-11T06:47:27.612 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-11T06:47:27.640 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-09-11T06:47:27.679 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-11T06:47:27.856 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package pkg-config. 2024-09-11T06:47:27.883 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-11T06:47:27.922 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-11T06:47:28.033 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package nvme-cli. 2024-09-11T06:47:28.060 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.2_amd64.deb ... 2024-09-11T06:47:28.099 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking nvme-cli (1.16-3ubuntu0.2) ... 2024-09-11T06:47:28.175 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-11T06:47:28.193 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-11T06:47:28.233 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-11T06:47:28.480 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package pkg-config. 2024-09-11T06:47:28.507 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-11T06:47:28.546 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-11T06:47:28.653 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-11T06:47:28.681 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-11T06:47:28.720 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-11T06:47:28.791 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-11T06:47:28.807 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-11T06:47:28.849 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-11T06:47:29.022 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-pastescript. 2024-09-11T06:47:29.050 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-09-11T06:47:29.088 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-11T06:47:29.245 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-11T06:47:29.272 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-11T06:47:29.311 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-11T06:47:29.409 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-pluggy. 2024-09-11T06:47:29.436 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-11T06:47:29.475 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-11T06:47:29.615 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-pastescript. 2024-09-11T06:47:29.641 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-09-11T06:47:29.680 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-11T06:47:29.744 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-volume. 2024-09-11T06:47:29.772 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../098-ceph-volume_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-11T06:47:29.780 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-psutil. 2024-09-11T06:47:29.807 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-11T06:47:29.810 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-volume (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:29.846 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-11T06:47:30.026 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-pluggy. 2024-09-11T06:47:30.054 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-11T06:47:30.092 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-11T06:47:30.115 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-11T06:47:30.143 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../099-libcephfs-dev_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:30.181 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libcephfs-dev (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:30.191 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-py. 2024-09-11T06:47:30.219 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-09-11T06:47:30.257 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-11T06:47:30.371 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-psutil. 2024-09-11T06:47:30.386 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-11T06:47:30.421 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-11T06:47:30.495 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-11T06:47:30.522 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-11T06:47:30.544 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-pygments. 2024-09-11T06:47:30.561 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-11T06:47:30.571 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-11T06:47:30.610 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-11T06:47:30.748 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-py. 2024-09-11T06:47:30.764 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-09-11T06:47:30.798 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-11T06:47:30.865 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-11T06:47:30.893 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-09-11T06:47:30.923 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-11T06:47:31.073 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-11T06:47:31.085 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-pygments. 2024-09-11T06:47:31.101 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-11T06:47:31.113 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-11T06:47:31.139 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-11T06:47:31.143 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-11T06:47:31.294 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package nvme-cli. 2024-09-11T06:47:31.322 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.2_amd64.deb ... 2024-09-11T06:47:31.361 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking nvme-cli (1.16-3ubuntu0.2) ... 2024-09-11T06:47:31.590 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-11T06:47:31.618 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-11T06:47:31.650 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-toml. 2024-09-11T06:47:31.656 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-11T06:47:31.678 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-09-11T06:47:31.716 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-11T06:47:31.791 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package pkg-config. 2024-09-11T06:47:31.818 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-11T06:47:31.857 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-11T06:47:31.918 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-toml. 2024-09-11T06:47:31.934 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-09-11T06:47:31.968 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-11T06:47:31.994 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-pytest. 2024-09-11T06:47:32.022 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-11T06:47:32.060 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-11T06:47:32.119 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-11T06:47:32.134 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-11T06:47:32.177 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-11T06:47:32.262 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-pytest. 2024-09-11T06:47:32.290 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-11T06:47:32.329 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-11T06:47:32.431 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-simplejson. 2024-09-11T06:47:32.458 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-11T06:47:32.497 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-11T06:47:32.597 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-11T06:47:32.624 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-11T06:47:32.663 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-11T06:47:32.674 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-simplejson. 2024-09-11T06:47:32.702 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-11T06:47:32.741 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-11T06:47:32.775 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-11T06:47:32.790 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-11T06:47:32.832 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-11T06:47:32.941 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-pastescript. 2024-09-11T06:47:32.956 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-09-11T06:47:32.991 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-11T06:47:33.002 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-11T06:47:33.018 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-11T06:47:33.060 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-11T06:47:33.278 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-pluggy. 2024-09-11T06:47:33.305 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-11T06:47:33.344 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-11T06:47:33.570 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package radosgw. 2024-09-11T06:47:33.598 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../116-radosgw_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:33.636 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking radosgw (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:33.655 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-psutil. 2024-09-11T06:47:33.683 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-11T06:47:33.722 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-11T06:47:33.798 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package radosgw. 2024-09-11T06:47:33.826 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../116-radosgw_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:33.864 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking radosgw (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:34.025 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-py. 2024-09-11T06:47:34.052 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-09-11T06:47:34.091 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-11T06:47:34.403 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-pygments. 2024-09-11T06:47:34.430 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-11T06:47:34.456 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package rbd-fuse. 2024-09-11T06:47:34.469 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-11T06:47:34.484 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../117-rbd-fuse_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:34.522 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking rbd-fuse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:34.726 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package rbd-fuse. 2024-09-11T06:47:34.754 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../117-rbd-fuse_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:34.791 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package rbd-nbd. 2024-09-11T06:47:34.792 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking rbd-fuse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:34.819 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../118-rbd-nbd_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:34.858 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking rbd-nbd (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:34.932 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-11T06:47:34.948 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-11T06:47:34.990 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-11T06:47:35.053 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package rbd-nbd. 2024-09-11T06:47:35.069 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../118-rbd-nbd_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:35.112 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking rbd-nbd (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:35.170 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-11T06:47:35.293 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-toml. 2024-09-11T06:47:35.320 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-09-11T06:47:35.359 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-11T06:47:35.391 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-11T06:47:35.446 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-11T06:47:35.614 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up nvme-cli (1.16-3ubuntu0.2) ... 2024-09-11T06:47:35.642 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-11T06:47:35.662 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-pytest. 2024-09-11T06:47:35.690 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-11T06:47:35.729 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-11T06:47:35.752 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up nvme-cli (1.16-3ubuntu0.2) ... 2024-09-11T06:47:35.874 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-11T06:47:36.013 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-11T06:47:36.066 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-simplejson. 2024-09-11T06:47:36.093 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-11T06:47:36.132 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-11T06:47:36.216 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-11T06:47:36.314 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-11T06:47:36.410 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-11T06:47:36.437 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-11T06:47:36.477 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-11T06:47:36.622 INFO:teuthology.orchestra.run.smithi176.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-11T06:47:36.707 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up cephadm (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:36.728 INFO:teuthology.orchestra.run.smithi071.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-11T06:47:36.887 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up cephadm (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:37.139 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package radosgw. 2024-09-11T06:47:37.166 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../116-radosgw_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:37.197 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking radosgw (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:37.315 INFO:teuthology.orchestra.run.smithi176.stdout:Adding system user cephadm....done 2024-09-11T06:47:37.403 INFO:teuthology.orchestra.run.smithi071.stdout:Adding system user cephadm....done 2024-09-11T06:47:37.463 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-11T06:47:37.560 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-11T06:47:37.739 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-11T06:47:37.845 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-11T06:47:38.015 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-11T06:47:38.016 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package rbd-fuse. 2024-09-11T06:47:38.033 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../117-rbd-fuse_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:38.075 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking rbd-fuse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:38.096 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-11T06:47:38.166 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-11T06:47:38.222 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-11T06:47:38.336 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package rbd-nbd. 2024-09-11T06:47:38.364 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../118-rbd-nbd_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-11T06:47:38.402 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking rbd-nbd (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:38.434 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-11T06:47:38.473 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-11T06:47:38.699 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-11T06:47:38.702 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-11T06:47:38.741 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-11T06:47:38.851 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-11T06:47:38.861 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-11T06:47:38.966 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-11T06:47:39.092 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up nvme-cli (1.16-3ubuntu0.2) ... 2024-09-11T06:47:39.144 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-11T06:47:39.171 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-11T06:47:39.367 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-11T06:47:39.521 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-11T06:47:39.540 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-11T06:47:39.706 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-11T06:47:39.772 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-11T06:47:39.816 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-11T06:47:40.032 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-11T06:47:40.100 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-11T06:47:40.151 INFO:teuthology.orchestra.run.smithi089.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-11T06:47:40.227 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up cephadm (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:40.292 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-11T06:47:40.393 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-11T06:47:40.544 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-11T06:47:40.670 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-11T06:47:40.693 INFO:teuthology.orchestra.run.smithi089.stdout:Adding system user cephadm....done 2024-09-11T06:47:40.803 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-11T06:47:40.850 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-11T06:47:40.921 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-11T06:47:41.063 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-ceph-argparse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:41.135 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-11T06:47:41.180 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-ceph-argparse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:41.364 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-11T06:47:41.389 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-11T06:47:41.448 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-11T06:47:41.490 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-11T06:47:41.598 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-11T06:47:41.616 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-11T06:47:41.726 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-11T06:47:41.766 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-11T06:47:41.829 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-11T06:47:41.917 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-11T06:47:41.977 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-11T06:47:42.071 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-11T06:47:42.193 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-11T06:47:42.314 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-11T06:47:42.359 INFO:teuthology.orchestra.run.smithi071.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-11T06:47:42.390 INFO:teuthology.orchestra.run.smithi071.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-11T06:47:42.440 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-11T06:47:42.455 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-11T06:47:42.582 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-11T06:47:42.582 INFO:teuthology.orchestra.run.smithi176.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-11T06:47:42.616 INFO:teuthology.orchestra.run.smithi176.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-11T06:47:42.680 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-11T06:47:42.767 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-11T06:47:42.831 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-11T06:47:42.941 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-11T06:47:43.069 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-11T06:47:43.119 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-11T06:47:43.207 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-11T06:47:43.337 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-11T06:47:43.358 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-11T06:47:43.412 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-11T06:47:43.596 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-11T06:47:43.651 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-11T06:47:43.680 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-11T06:47:43.890 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up zip (3.0-12build2) ... 2024-09-11T06:47:43.898 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-11T06:47:43.927 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-11T06:47:44.016 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-11T06:47:44.157 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-11T06:47:44.236 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up zip (3.0-12build2) ... 2024-09-11T06:47:44.362 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-11T06:47:44.416 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-11T06:47:44.659 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-11T06:47:44.767 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-ceph-argparse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:44.911 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-11T06:47:44.997 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-11T06:47:45.027 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-11T06:47:45.037 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-11T06:47:45.153 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-11T06:47:45.163 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-11T06:47:45.254 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-11T06:47:45.281 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-11T06:47:45.355 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-11T06:47:45.433 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-11T06:47:45.464 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-11T06:47:45.584 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-11T06:47:45.597 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-11T06:47:45.841 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-11T06:47:45.893 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-11T06:47:45.995 INFO:teuthology.orchestra.run.smithi089.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-11T06:47:46.029 INFO:teuthology.orchestra.run.smithi089.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-11T06:47:46.083 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-11T06:47:46.093 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-11T06:47:46.219 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-11T06:47:46.269 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-11T06:47:46.435 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-11T06:47:46.521 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-11T06:47:46.587 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-11T06:47:46.713 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-11T06:47:46.728 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-11T06:47:46.897 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-11T06:47:46.981 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-11T06:47:47.038 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-11T06:47:47.273 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-11T06:47:47.281 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-11T06:47:47.348 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-11T06:47:47.407 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-ceph-common (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:47.583 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up zip (3.0-12build2) ... 2024-09-11T06:47:47.699 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-11T06:47:47.708 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-11T06:47:47.709 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-11T06:47:47.992 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-11T06:47:48.026 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-11T06:47:48.093 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-ceph-common (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:48.386 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-11T06:47:48.436 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-11T06:47:48.704 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-11T06:47:48.704 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-11T06:47:48.830 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-11T06:47:48.956 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-11T06:47:49.183 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-11T06:47:49.248 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-11T06:47:49.401 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T06:47:49.536 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-11T06:47:49.616 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-11T06:47:49.795 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-11T06:47:49.859 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-11T06:47:49.876 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-11T06:47:49.896 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-11T06:47:49.997 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-11T06:47:50.178 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T06:47:50.236 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-11T06:47:50.299 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-11T06:47:50.312 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-11T06:47:50.512 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-11T06:47:50.550 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-11T06:47:50.580 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-11T06:47:50.722 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-11T06:47:50.810 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T06:47:50.873 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-11T06:47:50.884 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-11T06:47:50.936 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-11T06:47:51.148 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-11T06:47:51.200 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-11T06:47:51.246 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-11T06:47:51.273 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-ceph-common (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:51.480 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-11T06:47:51.509 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-11T06:47:51.566 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-11T06:47:51.748 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-11T06:47:51.777 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T06:47:51.876 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-11T06:47:51.886 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-11T06:47:52.204 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-11T06:47:52.240 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-11T06:47:52.488 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-11T06:47:52.559 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-11T06:47:52.773 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-11T06:47:52.869 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-11T06:47:53.016 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-11T06:47:53.040 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-11T06:47:53.041 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-11T06:47:53.250 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T06:47:53.300 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-11T06:47:53.304 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-11T06:47:53.393 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-11T06:47:53.431 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-11T06:47:53.584 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-11T06:47:53.644 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-11T06:47:53.693 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-11T06:47:53.734 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-11T06:47:53.770 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-11T06:47:53.896 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-11T06:47:53.961 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-11T06:47:54.016 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-11T06:47:54.087 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-11T06:47:54.214 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-11T06:47:54.284 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-11T06:47:54.372 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-11T06:47:54.419 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-11T06:47:54.490 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-11T06:47:54.537 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-11T06:47:54.609 INFO:teuthology.orchestra.run.smithi071.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-11T06:47:54.673 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-11T06:47:54.741 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T06:47:54.838 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-11T06:47:54.859 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-11T06:47:54.932 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-11T06:47:54.965 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-11T06:47:55.083 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-11T06:47:55.160 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-11T06:47:55.234 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-11T06:47:55.340 INFO:teuthology.orchestra.run.smithi176.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-11T06:47:55.341 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-11T06:47:55.428 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-11T06:47:55.569 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-11T06:47:55.592 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-11T06:47:55.688 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-11T06:47:55.695 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-11T06:47:55.719 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-11T06:47:55.964 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-11T06:47:55.986 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T06:47:56.096 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-11T06:47:56.113 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-11T06:47:56.215 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-11T06:47:56.364 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-11T06:47:56.431 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-11T06:47:56.474 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-11T06:47:56.515 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-11T06:47:56.575 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-11T06:47:56.732 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-11T06:47:56.791 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T06:47:56.843 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-11T06:47:56.917 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-11T06:47:56.944 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-11T06:47:57.000 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-11T06:47:57.126 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-11T06:47:57.251 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-11T06:47:57.254 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-11T06:47:57.394 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-11T06:47:57.563 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-11T06:47:57.594 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-11T06:47:57.662 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-11T06:47:57.831 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-11T06:47:57.879 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-11T06:47:57.957 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-11T06:47:57.990 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-11T06:47:58.005 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-11T06:47:58.083 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-11T06:47:58.173 INFO:teuthology.orchestra.run.smithi089.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-11T06:47:58.281 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-11T06:47:58.342 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-11T06:47:58.418 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-11T06:47:58.564 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-11T06:47:58.573 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-11T06:47:58.644 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-11T06:47:58.935 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-11T06:47:58.954 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-11T06:47:58.954 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-11T06:47:59.081 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up librados2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:59.216 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libsqlite3-mod-ceph (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:59.222 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-11T06:47:59.337 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-11T06:47:59.341 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-11T06:47:59.348 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-11T06:47:59.591 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T06:47:59.630 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-11T06:47:59.678 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libcephfs2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:59.716 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-11T06:47:59.804 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libradosstriper1 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:47:59.930 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-11T06:47:59.965 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-11T06:48:00.026 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-11T06:48:00.125 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up librados2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:00.207 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up librbd1 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:00.251 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libsqlite3-mod-ceph (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:00.327 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-11T06:48:00.333 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-mgr-modules-core (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:00.360 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-11T06:48:00.459 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-fuse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:00.595 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-11T06:48:00.673 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-11T06:48:00.673 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-11T06:48:00.680 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libcephfs2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:00.721 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-11T06:48:00.806 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libradosstriper1 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:00.932 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-11T06:48:01.005 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-11T06:48:01.119 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libcephfs-dev (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:01.237 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-rados (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:01.241 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up librbd1 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:01.281 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-11T06:48:01.338 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up librgw2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:01.367 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-mgr-modules-core (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:01.447 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-11T06:48:01.477 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-fuse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:01.634 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-11T06:48:01.689 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-11T06:48:01.689 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-11T06:48:02.020 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-11T06:48:02.153 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libcephfs-dev (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:02.279 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-rados (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:02.313 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-11T06:48:02.430 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up librgw2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:02.581 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-11T06:48:02.585 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-rbd (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:02.682 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-11T06:48:02.711 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up rbd-fuse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:02.800 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up librados2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:02.837 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up rbd-nbd (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:02.901 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libsqlite3-mod-ceph (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:02.963 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-rgw (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:03.010 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-11T06:48:03.081 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-cephfs (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:03.199 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-common (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:03.347 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libcephfs2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:03.473 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libradosstriper1 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:03.599 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-11T06:48:03.734 INFO:teuthology.orchestra.run.smithi071.stdout:Adding group ceph....done 2024-09-11T06:48:03.867 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up librbd1 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:04.001 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-rbd (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:04.002 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-mgr-modules-core (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:04.102 INFO:teuthology.orchestra.run.smithi071.stdout:Adding system user ceph....done 2024-09-11T06:48:04.119 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up rbd-fuse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:04.127 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-fuse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:04.253 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up rbd-nbd (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:04.279 INFO:teuthology.orchestra.run.smithi071.stdout:Setting system user ceph properties....done 2024-09-11T06:48:04.291 INFO:teuthology.orchestra.run.smithi071.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-11T06:48:04.337 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-11T06:48:04.337 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-11T06:48:04.379 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-rgw (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:04.435 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-11T06:48:04.505 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-cephfs (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:04.631 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-common (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:04.751 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-11T06:48:04.821 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libcephfs-dev (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:04.947 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-rados (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:05.234 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-test (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:05.257 INFO:teuthology.orchestra.run.smithi176.stdout:Adding group ceph....done 2024-09-11T06:48:05.369 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up radosgw (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:05.415 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up librgw2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:05.516 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-11T06:48:05.558 INFO:teuthology.orchestra.run.smithi176.stdout:Adding system user ceph....done 2024-09-11T06:48:05.719 INFO:teuthology.orchestra.run.smithi176.stdout:Setting system user ceph properties....done 2024-09-11T06:48:05.735 INFO:teuthology.orchestra.run.smithi176.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-11T06:48:05.821 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-11T06:48:05.821 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-11T06:48:05.872 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-11T06:48:06.146 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-11T06:48:06.295 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-base (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:06.623 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-test (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:06.629 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-11T06:48:06.712 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-rbd (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:06.749 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up radosgw (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:06.838 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up rbd-fuse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:06.964 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up rbd-nbd (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:07.090 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-rgw (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:07.097 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-mds (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:07.210 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-11T06:48:07.210 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-11T06:48:07.216 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-cephfs (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:07.311 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-11T06:48:07.311 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-11T06:48:07.367 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-common (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:07.651 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-base (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:07.868 INFO:teuthology.orchestra.run.smithi089.stdout:Adding group ceph....done 2024-09-11T06:48:07.993 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-11T06:48:08.105 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-mgr (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:08.253 INFO:teuthology.orchestra.run.smithi089.stdout:Adding system user ceph....done 2024-09-11T06:48:08.327 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-11T06:48:08.327 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-11T06:48:08.388 INFO:teuthology.orchestra.run.smithi089.stdout:Setting system user ceph properties....done 2024-09-11T06:48:08.401 INFO:teuthology.orchestra.run.smithi089.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-11T06:48:08.436 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-mds (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:08.538 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-11T06:48:08.649 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-11T06:48:08.649 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-11T06:48:08.764 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-osd (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:08.884 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-11T06:48:09.034 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-11T06:48:09.035 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-11T06:48:09.104 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-mgr (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:09.325 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-11T06:48:09.325 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-11T06:48:09.368 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-test (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:09.490 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-mgr-k8sevents (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:09.496 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up radosgw (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:09.616 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:09.768 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-mon (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:09.814 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-osd (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:09.979 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-11T06:48:09.979 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-11T06:48:10.067 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-11T06:48:10.067 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-11T06:48:10.112 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-11T06:48:10.112 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-11T06:48:10.453 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-mgr-cephadm (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:10.565 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-mgr-k8sevents (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:10.579 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-base (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:10.604 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:10.691 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:10.730 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-mgr-dashboard (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:10.842 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-mon (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:10.873 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-volume (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:10.915 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-11T06:48:11.055 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-11T06:48:11.055 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-11T06:48:11.343 INFO:teuthology.orchestra.run.smithi071.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-11T06:48:11.389 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-mds (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:11.477 INFO:teuthology.orchestra.run.smithi071.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-11T06:48:11.494 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-mgr-cephadm (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:11.629 INFO:teuthology.orchestra.run.smithi071.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-11T06:48:11.634 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-11T06:48:11.634 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-11T06:48:11.636 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:11.762 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-mgr-dashboard (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:11.913 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-volume (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:12.108 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-mgr (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:12.274 INFO:teuthology.orchestra.run.smithi176.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-11T06:48:12.323 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-11T06:48:12.323 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-11T06:48:12.400 INFO:teuthology.orchestra.run.smithi176.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-11T06:48:12.544 INFO:teuthology.orchestra.run.smithi176.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-11T06:48:12.747 INFO:teuthology.orchestra.run.smithi071.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-11T06:48:13.075 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-osd (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:13.341 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-11T06:48:13.341 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-11T06:48:13.403 INFO:teuthology.orchestra.run.smithi071.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-11T06:48:13.410 DEBUG:teuthology.parallel:result is None 2024-09-11T06:48:13.746 INFO:teuthology.orchestra.run.smithi176.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-11T06:48:13.781 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-mgr-k8sevents (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:13.890 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:14.067 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-mon (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:14.280 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-11T06:48:14.281 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-11T06:48:14.422 INFO:teuthology.orchestra.run.smithi176.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-11T06:48:14.427 DEBUG:teuthology.parallel:result is None 2024-09-11T06:48:14.768 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-mgr-cephadm (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:14.919 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:15.048 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-mgr-dashboard (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:15.196 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-volume (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-11T06:48:15.557 INFO:teuthology.orchestra.run.smithi089.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-11T06:48:15.658 INFO:teuthology.orchestra.run.smithi089.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-11T06:48:15.802 INFO:teuthology.orchestra.run.smithi089.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-11T06:48:16.911 INFO:teuthology.orchestra.run.smithi089.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-11T06:48:17.547 INFO:teuthology.orchestra.run.smithi089.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-11T06:48:17.552 DEBUG:teuthology.parallel:result is None 2024-09-11T06:48:17.553 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T06:48:17.706 DEBUG:teuthology.orchestra.run.smithi071:> dpkg-query -W -f '${Version}' ceph 2024-09-11T06:48:17.727 INFO:teuthology.orchestra.run.smithi071.stdout:19.1.1-298-gb53b6d3e-1jammy 2024-09-11T06:48:17.728 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298-gb53b6d3e-1jammy 2024-09-11T06:48:17.728 INFO:teuthology.task.install:The correct ceph version 19.1.1-298-gb53b6d3e-1jammy is installed. 2024-09-11T06:48:17.731 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T06:48:17.874 DEBUG:teuthology.orchestra.run.smithi089:> dpkg-query -W -f '${Version}' ceph 2024-09-11T06:48:17.893 INFO:teuthology.orchestra.run.smithi089.stdout:19.1.1-298-gb53b6d3e-1jammy 2024-09-11T06:48:17.893 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298-gb53b6d3e-1jammy 2024-09-11T06:48:17.894 INFO:teuthology.task.install:The correct ceph version 19.1.1-298-gb53b6d3e-1jammy is installed. 2024-09-11T06:48:17.895 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T06:48:18.039 DEBUG:teuthology.orchestra.run.smithi176:> dpkg-query -W -f '${Version}' ceph 2024-09-11T06:48:18.058 INFO:teuthology.orchestra.run.smithi176.stdout:19.1.1-298-gb53b6d3e-1jammy 2024-09-11T06:48:18.058 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298-gb53b6d3e-1jammy 2024-09-11T06:48:18.059 INFO:teuthology.task.install:The correct ceph version 19.1.1-298-gb53b6d3e-1jammy is installed. 2024-09-11T06:48:18.060 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-11T06:48:18.060 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T06:48:18.060 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T06:48:18.075 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-11T06:48:18.075 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T06:48:18.089 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-11T06:48:18.089 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T06:48:18.115 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-11T06:48:18.116 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T06:48:18.116 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T06:48:18.129 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T06:48:18.184 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-11T06:48:18.184 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T06:48:18.197 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T06:48:18.256 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-11T06:48:18.256 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T06:48:18.270 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T06:48:18.326 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-11T06:48:18.326 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T06:48:18.326 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T06:48:18.339 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T06:48:18.391 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-11T06:48:18.392 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T06:48:18.405 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T06:48:18.459 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-11T06:48:18.459 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T06:48:18.472 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T06:48:18.526 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-11T06:48:18.527 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T06:48:18.527 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T06:48:18.541 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T06:48:18.596 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-11T06:48:18.596 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T06:48:18.609 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T06:48:18.662 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-11T06:48:18.662 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T06:48:18.676 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T06:48:18.729 INFO:teuthology.run_tasks:Running task ceph... 2024-09-11T06:48:18.810 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-11T06:48:18.811 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 777 /var/log/ceph 2024-09-11T06:48:18.813 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 777 /var/log/ceph 2024-09-11T06:48:18.816 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod 777 /var/log/ceph 2024-09-11T06:48:18.828 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-11T06:48:18.828 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-11T06:48:18.869 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-11T06:48:18.873 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-11T06:48:18.885 INFO:tasks.ceph:Creating extra log directories... 2024-09-11T06:48:18.885 DEBUG:teuthology.orchestra.run.smithi071:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-11T06:48:18.922 DEBUG:teuthology.orchestra.run.smithi089:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-11T06:48:18.929 DEBUG:teuthology.orchestra.run.smithi176:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-11T06:48:18.952 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-11T06:48:18.952 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-11T06:48:18.953 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900631', 'branch': 'squid', 'description': 'rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_nbd}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7900631', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'daaf78de334647fc0c800f4ac495cc9475a26bae'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'], ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'], ['client.0']], 'seed': 6175, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'sleep_before_teardown': 0, 'subset': '88/128', 'suite': 'rbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_daaf78de334647fc0c800f4ac495cc9475a26bae/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'daaf78de334647fc0c800f4ac495cc9475a26bae', 'targets': {'smithi071.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAsPkUEtaeK9fdViOAbo1tNJi31yoOvkLCCbzicWSl6C3EKI0N53fBaj0cY+pT7jRddjNtDimhk3ZvXnK8/21Hw=', 'smithi089.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL+OXkIXcTcVfMSPnVHzyHnYRQsMzpHhB9KsmVGdh1RuVfEkyD3P2hIIvvJU9BIpx29hVx069Ns93Iu/wq1dH6o=', 'smithi176.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL8YgroLZ6X4Agx78ypmwC9s8cDMc0pbauE0dsOzTE48O2rhwQpTvU5VlPREuI5UfCWL+PhgtZyv3Iozp+aYRuk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'workunit': {'clients': {'client.0': ['rbd/rbd-nbd.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_21:16:03', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-11T06:48:18.953 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-11T06:48:18.977 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-11T06:48:18.985 DEBUG:teuthology.orchestra.run.smithi176:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-11T06:48:18.990 DEBUG:teuthology.orchestra.run.smithi071:> sudo install -d -m0777 -- /var/run/ceph 2024-09-11T06:48:19.025 DEBUG:teuthology.orchestra.run.smithi089:> sudo install -d -m0777 -- /var/run/ceph 2024-09-11T06:48:19.033 DEBUG:teuthology.orchestra.run.smithi176:> sudo install -d -m0777 -- /var/run/ceph 2024-09-11T06:48:19.047 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T06:48:19.047 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T06:48:19.085 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T06:48:19.085 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_1 2024-09-11T06:48:19.133 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T06:48:19.133 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T06:48:19.133 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 885 Links: 1 2024-09-11T06:48:19.133 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T06:48:19.133 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-11 06:42:48.804419275 +0000 2024-09-11T06:48:19.134 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-11 06:42:48.560421455 +0000 2024-09-11T06:48:19.134 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-11 06:42:48.560421455 +0000 2024-09-11T06:48:19.134 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: - 2024-09-11T06:48:19.134 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T06:48:19.187 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-09-11T06:48:19.187 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-09-11T06:48:19.187 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000381719 s, 1.3 MB/s 2024-09-11T06:48:19.189 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T06:48:19.237 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_2 2024-09-11T06:48:19.284 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T06:48:19.284 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T06:48:19.284 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 901 Links: 1 2024-09-11T06:48:19.285 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T06:48:19.285 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-11 06:42:49.296414880 +0000 2024-09-11T06:48:19.285 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-11 06:42:49.020417345 +0000 2024-09-11T06:48:19.285 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-11 06:42:49.020417345 +0000 2024-09-11T06:48:19.285 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: - 2024-09-11T06:48:19.285 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T06:48:19.343 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-09-11T06:48:19.343 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-09-11T06:48:19.343 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000417052 s, 1.2 MB/s 2024-09-11T06:48:19.345 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T06:48:19.397 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_3 2024-09-11T06:48:19.445 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T06:48:19.445 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T06:48:19.445 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 920 Links: 1 2024-09-11T06:48:19.445 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T06:48:19.445 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-11 06:42:49.816410236 +0000 2024-09-11T06:48:19.445 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-11 06:42:49.556412558 +0000 2024-09-11T06:48:19.445 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-11 06:42:49.556412558 +0000 2024-09-11T06:48:19.445 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: - 2024-09-11T06:48:19.445 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T06:48:19.503 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-09-11T06:48:19.503 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-09-11T06:48:19.503 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000307093 s, 1.7 MB/s 2024-09-11T06:48:19.505 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T06:48:19.554 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_4 2024-09-11T06:48:19.600 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T06:48:19.601 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T06:48:19.601 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 935 Links: 1 2024-09-11T06:48:19.601 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T06:48:19.601 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-11 06:42:50.276406126 +0000 2024-09-11T06:48:19.601 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-11 06:42:50.036408271 +0000 2024-09-11T06:48:19.601 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-11 06:42:50.036408271 +0000 2024-09-11T06:48:19.601 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: - 2024-09-11T06:48:19.601 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T06:48:19.654 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-09-11T06:48:19.654 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-09-11T06:48:19.654 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000306395 s, 1.7 MB/s 2024-09-11T06:48:19.656 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T06:48:19.705 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-11T06:48:19.705 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-11T06:48:19.705 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T06:48:19.711 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T06:48:19.711 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_1 2024-09-11T06:48:19.760 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T06:48:19.761 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T06:48:19.761 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 878 Links: 1 2024-09-11T06:48:19.761 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T06:48:19.761 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-09-11 06:42:50.115386258 +0000 2024-09-11T06:48:19.761 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-09-11 06:42:49.891388254 +0000 2024-09-11T06:48:19.761 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-09-11 06:42:49.891388254 +0000 2024-09-11T06:48:19.761 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2024-09-11T06:48:19.761 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T06:48:19.815 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-09-11T06:48:19.815 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-09-11T06:48:19.815 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000447526 s, 1.1 MB/s 2024-09-11T06:48:19.816 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T06:48:19.869 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_2 2024-09-11T06:48:19.916 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T06:48:19.916 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T06:48:19.916 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 894 Links: 1 2024-09-11T06:48:19.916 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T06:48:19.916 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-09-11 06:42:50.607381875 +0000 2024-09-11T06:48:19.916 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-09-11 06:42:50.331384334 +0000 2024-09-11T06:48:19.916 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-09-11 06:42:50.331384334 +0000 2024-09-11T06:48:19.916 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2024-09-11T06:48:19.916 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T06:48:19.971 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-09-11T06:48:19.971 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-09-11T06:48:19.971 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000420539 s, 1.2 MB/s 2024-09-11T06:48:19.972 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T06:48:20.021 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_3 2024-09-11T06:48:20.068 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T06:48:20.068 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T06:48:20.068 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 909 Links: 1 2024-09-11T06:48:20.068 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T06:48:20.068 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-09-11 06:42:51.103377456 +0000 2024-09-11T06:48:20.068 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-09-11 06:42:50.859379630 +0000 2024-09-11T06:48:20.068 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-09-11 06:42:50.859379630 +0000 2024-09-11T06:48:20.068 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2024-09-11T06:48:20.068 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T06:48:20.122 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-09-11T06:48:20.122 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-09-11T06:48:20.122 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000403483 s, 1.3 MB/s 2024-09-11T06:48:20.123 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T06:48:20.173 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_4 2024-09-11T06:48:20.219 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T06:48:20.220 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T06:48:20.220 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 929 Links: 1 2024-09-11T06:48:20.220 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T06:48:20.220 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-09-11 06:42:51.575373251 +0000 2024-09-11T06:48:20.220 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-09-11 06:42:51.339375353 +0000 2024-09-11T06:48:20.220 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-09-11 06:42:51.339375353 +0000 2024-09-11T06:48:20.220 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2024-09-11T06:48:20.220 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T06:48:20.275 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-09-11T06:48:20.275 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-09-11T06:48:20.275 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000391556 s, 1.3 MB/s 2024-09-11T06:48:20.276 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T06:48:20.325 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-11T06:48:20.325 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi071.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi089.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-11T06:48:20.325 INFO:tasks.ceph:Generating config... 2024-09-11T06:48:20.326 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-11T06:48:20.327 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-11T06:48:20.327 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-11T06:48:20.327 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-11T06:48:20.327 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-11T06:48:20.327 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-11T06:48:20.327 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-11T06:48:20.327 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-11T06:48:20.327 INFO:tasks.ceph:[osd] bluefs allocator = hybrid 2024-09-11T06:48:20.327 INFO:tasks.ceph:[osd] bluestore allocator = hybrid 2024-09-11T06:48:20.327 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-11T06:48:20.327 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-11T06:48:20.327 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-11T06:48:20.327 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-11T06:48:20.327 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-11T06:48:20.327 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-11T06:48:20.328 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-11T06:48:20.328 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-11T06:48:20.328 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-11T06:48:20.328 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-11T06:48:20.328 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-11T06:48:20.328 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-11T06:48:20.328 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-11T06:48:20.328 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-11T06:48:20.328 INFO:tasks.ceph:Setting up mon.a... 2024-09-11T06:48:20.328 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-11T06:48:20.363 INFO:teuthology.orchestra.run.smithi071.stdout:creating /etc/ceph/ceph.keyring 2024-09-11T06:48:20.380 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-11T06:48:20.457 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-11T06:48:20.511 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.71'), ('mon.c', '[v2:172.21.15.71:3301,v1:172.21.15.71:6790]'), ('mon.b', '172.21.15.89')] 2024-09-11T06:48:20.512 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.71,[v2:172.21.15.71:3301,v1:172.21.15.71:6790],172.21.15.89', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-11T06:48:20.512 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T06:48:20.513 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-11T06:48:20.561 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.71 --addv c '[v2:172.21.15.71:3301,v1:172.21.15.71:6790]' --add b 172.21.15.89 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-11T06:48:20.646 INFO:teuthology.orchestra.run.smithi071.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-11T06:48:20.646 INFO:teuthology.orchestra.run.smithi071.stdout:monmaptool: generated fsid bd5dcb33-e9f7-4a5f-a26c-a451452e8aa2 2024-09-11T06:48:20.646 INFO:teuthology.orchestra.run.smithi071.stdout:setting min_mon_release = quincy 2024-09-11T06:48:20.647 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 0 2024-09-11T06:48:20.647 INFO:teuthology.orchestra.run.smithi071.stdout:fsid bd5dcb33-e9f7-4a5f-a26c-a451452e8aa2 2024-09-11T06:48:20.647 INFO:teuthology.orchestra.run.smithi071.stdout:last_changed 2024-09-11T06:48:20.647173+0000 2024-09-11T06:48:20.647 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-09-11T06:48:20.647173+0000 2024-09-11T06:48:20.647 INFO:teuthology.orchestra.run.smithi071.stdout:min_mon_release 17 (quincy) 2024-09-11T06:48:20.647 INFO:teuthology.orchestra.run.smithi071.stdout:election_strategy: 1 2024-09-11T06:48:20.647 INFO:teuthology.orchestra.run.smithi071.stdout:0: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.a 2024-09-11T06:48:20.647 INFO:teuthology.orchestra.run.smithi071.stdout:1: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.b 2024-09-11T06:48:20.648 INFO:teuthology.orchestra.run.smithi071.stdout:2: [v2:172.21.15.71:3301/0,v1:172.21.15.71:6790/0] mon.c 2024-09-11T06:48:20.648 INFO:teuthology.orchestra.run.smithi071.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-11T06:48:20.649 DEBUG:teuthology.orchestra.run.smithi071:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-11T06:48:20.697 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID bd5dcb33-e9f7-4a5f-a26c-a451452e8aa2... 2024-09-11T06:48:20.698 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-11T06:48:20.741 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-11T06:48:20.744 DEBUG:teuthology.orchestra.run.smithi176:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-11T06:48:20.770 INFO:teuthology.orchestra.run.smithi071.stdout:[global] 2024-09-11T06:48:20.770 INFO:teuthology.orchestra.run.smithi071.stdout: chdir = "" 2024-09-11T06:48:20.770 INFO:teuthology.orchestra.run.smithi071.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-11T06:48:20.770 INFO:teuthology.orchestra.run.smithi071.stdout: auth supported = cephx 2024-09-11T06:48:20.770 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.770 INFO:teuthology.orchestra.run.smithi071.stdout: filestore xattr use omap = true 2024-09-11T06:48:20.770 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.771 INFO:teuthology.orchestra.run.smithi071.stdout: mon clock drift allowed = 1.000 2024-09-11T06:48:20.771 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.771 INFO:teuthology.orchestra.run.smithi071.stdout: osd crush chooseleaf type = 0 2024-09-11T06:48:20.771 INFO:teuthology.orchestra.run.smithi071.stdout: auth debug = true 2024-09-11T06:48:20.771 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.771 INFO:teuthology.orchestra.run.smithi071.stdout: ms die on old message = true 2024-09-11T06:48:20.771 INFO:teuthology.orchestra.run.smithi071.stdout: ms die on bug = true 2024-09-11T06:48:20.771 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.771 INFO:teuthology.orchestra.run.smithi071.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-11T06:48:20.771 INFO:teuthology.orchestra.run.smithi071.stdout: mon pg warn max object skew = 0 2024-09-11T06:48:20.771 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.771 INFO:teuthology.orchestra.run.smithi071.stdout: # disable pg_autoscaler by default for new pools 2024-09-11T06:48:20.771 INFO:teuthology.orchestra.run.smithi071.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-11T06:48:20.771 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.772 INFO:teuthology.orchestra.run.smithi071.stdout: osd pool default size = 2 2024-09-11T06:48:20.772 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.772 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd allow primary affinity = true 2024-09-11T06:48:20.772 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd allow pg remap = true 2024-09-11T06:48:20.772 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on legacy crush tunables = false 2024-09-11T06:48:20.772 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on crush straw calc version zero = false 2024-09-11T06:48:20.772 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on no sortbitwise = false 2024-09-11T06:48:20.772 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on osd down out interval zero = false 2024-09-11T06:48:20.772 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on too few osds = false 2024-09-11T06:48:20.772 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-11T06:48:20.772 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-11T06:48:20.772 INFO:teuthology.orchestra.run.smithi071.stdout: mon_allow_pool_size_one = true 2024-09-11T06:48:20.772 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.772 INFO:teuthology.orchestra.run.smithi071.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-11T06:48:20.773 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.773 INFO:teuthology.orchestra.run.smithi071.stdout: osd default data pool replay window = 5 2024-09-11T06:48:20.773 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.773 INFO:teuthology.orchestra.run.smithi071.stdout: mon allow pool delete = true 2024-09-11T06:48:20.773 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.773 INFO:teuthology.orchestra.run.smithi071.stdout: mon cluster log file level = debug 2024-09-11T06:48:20.773 INFO:teuthology.orchestra.run.smithi071.stdout: debug asserts on shutdown = true 2024-09-11T06:48:20.773 INFO:teuthology.orchestra.run.smithi071.stdout: mon health detail to clog = false 2024-09-11T06:48:20.773 INFO:teuthology.orchestra.run.smithi071.stdout: mon host = "172.21.15.71,[v2:172.21.15.71:3301,v1:172.21.15.71:6790],172.21.15.89" 2024-09-11T06:48:20.773 INFO:teuthology.orchestra.run.smithi071.stdout: mon client directed command retry = 5 2024-09-11T06:48:20.773 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on pool no app = False 2024-09-11T06:48:20.773 INFO:teuthology.orchestra.run.smithi071.stdout: ms inject socket failures = 5000 2024-09-11T06:48:20.774 INFO:teuthology.orchestra.run.smithi071.stdout: fsid = bd5dcb33-e9f7-4a5f-a26c-a451452e8aa2 2024-09-11T06:48:20.774 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.774 INFO:teuthology.orchestra.run.smithi071.stdout:[osd] 2024-09-11T06:48:20.774 INFO:teuthology.orchestra.run.smithi071.stdout: osd journal size = 100 2024-09-11T06:48:20.774 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.774 INFO:teuthology.orchestra.run.smithi071.stdout: osd scrub load threshold = 5.0 2024-09-11T06:48:20.774 INFO:teuthology.orchestra.run.smithi071.stdout: osd scrub max interval = 600 2024-09-11T06:48:20.774 INFO:teuthology.orchestra.run.smithi071.stdout: osd mclock profile = high_recovery_ops 2024-09-11T06:48:20.774 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.774 INFO:teuthology.orchestra.run.smithi071.stdout: osd recover clone overlap = true 2024-09-11T06:48:20.774 INFO:teuthology.orchestra.run.smithi071.stdout: osd recovery max chunk = 1048576 2024-09-11T06:48:20.774 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.774 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug shutdown = true 2024-09-11T06:48:20.774 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug op order = true 2024-09-11T06:48:20.774 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug verify stray on activate = true 2024-09-11T06:48:20.774 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug trim objects = true 2024-09-11T06:48:20.775 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.775 INFO:teuthology.orchestra.run.smithi071.stdout: osd open classes on start = true 2024-09-11T06:48:20.775 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug pg log writeout = true 2024-09-11T06:48:20.775 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.775 INFO:teuthology.orchestra.run.smithi071.stdout: osd deep scrub update digest min age = 30 2024-09-11T06:48:20.775 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.775 INFO:teuthology.orchestra.run.smithi071.stdout: osd map max advance = 10 2024-09-11T06:48:20.775 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.775 INFO:teuthology.orchestra.run.smithi071.stdout: journal zero on create = true 2024-09-11T06:48:20.775 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.775 INFO:teuthology.orchestra.run.smithi071.stdout: filestore ondisk finisher threads = 3 2024-09-11T06:48:20.775 INFO:teuthology.orchestra.run.smithi071.stdout: filestore apply finisher threads = 3 2024-09-11T06:48:20.775 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.776 INFO:teuthology.orchestra.run.smithi071.stdout: bdev debug aio = true 2024-09-11T06:48:20.776 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug misdirected ops = true 2024-09-11T06:48:20.776 INFO:teuthology.orchestra.run.smithi071.stdout: bluefs allocator = hybrid 2024-09-11T06:48:20.776 INFO:teuthology.orchestra.run.smithi071.stdout: bluestore allocator = hybrid 2024-09-11T06:48:20.776 INFO:teuthology.orchestra.run.smithi071.stdout: bluestore block size = 96636764160 2024-09-11T06:48:20.776 INFO:teuthology.orchestra.run.smithi071.stdout: bluestore fsck on mount = True 2024-09-11T06:48:20.776 INFO:teuthology.orchestra.run.smithi071.stdout: debug bluefs = 1/20 2024-09-11T06:48:20.776 INFO:teuthology.orchestra.run.smithi071.stdout: debug bluestore = 1/20 2024-09-11T06:48:20.776 INFO:teuthology.orchestra.run.smithi071.stdout: debug ms = 1 2024-09-11T06:48:20.776 INFO:teuthology.orchestra.run.smithi071.stdout: debug osd = 20 2024-09-11T06:48:20.776 INFO:teuthology.orchestra.run.smithi071.stdout: debug rocksdb = 4/10 2024-09-11T06:48:20.776 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd backfillfull_ratio = 0.85 2024-09-11T06:48:20.776 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd full ratio = 0.9 2024-09-11T06:48:20.776 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd nearfull ratio = 0.8 2024-09-11T06:48:20.777 INFO:teuthology.orchestra.run.smithi071.stdout: osd blocked scrub grace period = 3600 2024-09-11T06:48:20.777 INFO:teuthology.orchestra.run.smithi071.stdout: osd failsafe full ratio = 0.95 2024-09-11T06:48:20.777 INFO:teuthology.orchestra.run.smithi071.stdout: osd max markdown count = 1000 2024-09-11T06:48:20.777 INFO:teuthology.orchestra.run.smithi071.stdout: osd objectstore = bluestore 2024-09-11T06:48:20.777 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.777 INFO:teuthology.orchestra.run.smithi071.stdout:[mgr] 2024-09-11T06:48:20.777 INFO:teuthology.orchestra.run.smithi071.stdout: debug ms = 1 2024-09-11T06:48:20.777 INFO:teuthology.orchestra.run.smithi071.stdout: debug mgr = 20 2024-09-11T06:48:20.777 INFO:teuthology.orchestra.run.smithi071.stdout: debug mon = 20 2024-09-11T06:48:20.777 INFO:teuthology.orchestra.run.smithi071.stdout: debug auth = 20 2024-09-11T06:48:20.777 INFO:teuthology.orchestra.run.smithi071.stdout: mon reweight min pgs per osd = 4 2024-09-11T06:48:20.777 INFO:teuthology.orchestra.run.smithi071.stdout: mon reweight min bytes per osd = 10 2024-09-11T06:48:20.777 INFO:teuthology.orchestra.run.smithi071.stdout: mgr/telemetry/nag = false 2024-09-11T06:48:20.777 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.778 INFO:teuthology.orchestra.run.smithi071.stdout:[mon] 2024-09-11T06:48:20.778 INFO:teuthology.orchestra.run.smithi071.stdout: debug ms = 1 2024-09-11T06:48:20.778 INFO:teuthology.orchestra.run.smithi071.stdout: debug mon = 20 2024-09-11T06:48:20.778 INFO:teuthology.orchestra.run.smithi071.stdout: debug paxos = 20 2024-09-11T06:48:20.778 INFO:teuthology.orchestra.run.smithi071.stdout: debug auth = 20 2024-09-11T06:48:20.778 INFO:teuthology.orchestra.run.smithi071.stdout: mon data avail warn = 5 2024-09-11T06:48:20.778 INFO:teuthology.orchestra.run.smithi071.stdout: mon mgr mkfs grace = 240 2024-09-11T06:48:20.778 INFO:teuthology.orchestra.run.smithi071.stdout: mon reweight min pgs per osd = 4 2024-09-11T06:48:20.778 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd reporter subtree level = osd 2024-09-11T06:48:20.778 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd prime pg temp = true 2024-09-11T06:48:20.778 INFO:teuthology.orchestra.run.smithi071.stdout: mon reweight min bytes per osd = 10 2024-09-11T06:48:20.778 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.778 INFO:teuthology.orchestra.run.smithi071.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-11T06:48:20.778 INFO:teuthology.orchestra.run.smithi071.stdout: auth mon ticket ttl = 660 # 11m 2024-09-11T06:48:20.778 INFO:teuthology.orchestra.run.smithi071.stdout: auth service ticket ttl = 240 # 4m 2024-09-11T06:48:20.779 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.779 INFO:teuthology.orchestra.run.smithi071.stdout: # don't complain about insecure global_id in the test suite 2024-09-11T06:48:20.779 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-11T06:48:20.779 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-11T06:48:20.779 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.779 INFO:teuthology.orchestra.run.smithi071.stdout: # 1m isn't quite enough 2024-09-11T06:48:20.779 INFO:teuthology.orchestra.run.smithi071.stdout: mon_down_mkfs_grace = 2m 2024-09-11T06:48:20.779 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.779 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_filestore_osds = false 2024-09-11T06:48:20.779 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:20.779 INFO:teuthology.orchestra.run.smithi071.stdout:[client] 2024-09-11T06:48:20.779 INFO:teuthology.orchestra.run.smithi071.stdout: rgw cache enabled = true 2024-09-11T06:48:20.779 INFO:teuthology.orchestra.run.smithi071.stdout: rgw enable ops log = true 2024-09-11T06:48:20.779 INFO:teuthology.orchestra.run.smithi071.stdout: rgw enable usage log = true 2024-09-11T06:48:20.780 INFO:teuthology.orchestra.run.smithi071.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-11T06:48:20.780 INFO:teuthology.orchestra.run.smithi071.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-11T06:48:20.780 INFO:teuthology.orchestra.run.smithi071.stdout:[mon.a] 2024-09-11T06:48:20.780 INFO:teuthology.orchestra.run.smithi071.stdout:[mon.c] 2024-09-11T06:48:20.780 INFO:teuthology.orchestra.run.smithi071.stdout:[mon.b] 2024-09-11T06:48:20.780 INFO:teuthology.orchestra.run.smithi176.stdout:[global] 2024-09-11T06:48:20.781 INFO:teuthology.orchestra.run.smithi176.stdout: chdir = "" 2024-09-11T06:48:20.781 INFO:teuthology.orchestra.run.smithi176.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-11T06:48:20.781 INFO:teuthology.orchestra.run.smithi176.stdout: auth supported = cephx 2024-09-11T06:48:20.781 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.781 INFO:teuthology.orchestra.run.smithi176.stdout: filestore xattr use omap = true 2024-09-11T06:48:20.781 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.781 INFO:teuthology.orchestra.run.smithi176.stdout: mon clock drift allowed = 1.000 2024-09-11T06:48:20.781 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.781 INFO:teuthology.orchestra.run.smithi176.stdout: osd crush chooseleaf type = 0 2024-09-11T06:48:20.781 INFO:teuthology.orchestra.run.smithi176.stdout: auth debug = true 2024-09-11T06:48:20.781 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.781 INFO:teuthology.orchestra.run.smithi176.stdout: ms die on old message = true 2024-09-11T06:48:20.782 INFO:teuthology.orchestra.run.smithi176.stdout: ms die on bug = true 2024-09-11T06:48:20.782 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.782 INFO:teuthology.orchestra.run.smithi176.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-11T06:48:20.782 INFO:teuthology.orchestra.run.smithi176.stdout: mon pg warn max object skew = 0 2024-09-11T06:48:20.782 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.782 INFO:teuthology.orchestra.run.smithi176.stdout: # disable pg_autoscaler by default for new pools 2024-09-11T06:48:20.782 INFO:teuthology.orchestra.run.smithi176.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-11T06:48:20.782 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.782 INFO:teuthology.orchestra.run.smithi176.stdout: osd pool default size = 2 2024-09-11T06:48:20.782 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.782 INFO:teuthology.orchestra.run.smithi176.stdout: mon osd allow primary affinity = true 2024-09-11T06:48:20.782 INFO:teuthology.orchestra.run.smithi176.stdout: mon osd allow pg remap = true 2024-09-11T06:48:20.782 INFO:teuthology.orchestra.run.smithi176.stdout: mon warn on legacy crush tunables = false 2024-09-11T06:48:20.782 INFO:teuthology.orchestra.run.smithi176.stdout: mon warn on crush straw calc version zero = false 2024-09-11T06:48:20.783 INFO:teuthology.orchestra.run.smithi176.stdout: mon warn on no sortbitwise = false 2024-09-11T06:48:20.783 INFO:teuthology.orchestra.run.smithi176.stdout: mon warn on osd down out interval zero = false 2024-09-11T06:48:20.783 INFO:teuthology.orchestra.run.smithi176.stdout: mon warn on too few osds = false 2024-09-11T06:48:20.783 INFO:teuthology.orchestra.run.smithi176.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-11T06:48:20.783 INFO:teuthology.orchestra.run.smithi176.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-11T06:48:20.783 INFO:teuthology.orchestra.run.smithi176.stdout: mon_allow_pool_size_one = true 2024-09-11T06:48:20.783 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.783 INFO:teuthology.orchestra.run.smithi176.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-11T06:48:20.783 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.783 INFO:teuthology.orchestra.run.smithi176.stdout: osd default data pool replay window = 5 2024-09-11T06:48:20.783 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.783 INFO:teuthology.orchestra.run.smithi176.stdout: mon allow pool delete = true 2024-09-11T06:48:20.783 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.784 INFO:teuthology.orchestra.run.smithi176.stdout: mon cluster log file level = debug 2024-09-11T06:48:20.784 INFO:teuthology.orchestra.run.smithi176.stdout: debug asserts on shutdown = true 2024-09-11T06:48:20.784 INFO:teuthology.orchestra.run.smithi176.stdout: mon health detail to clog = false 2024-09-11T06:48:20.784 INFO:teuthology.orchestra.run.smithi176.stdout: mon host = "172.21.15.71,[v2:172.21.15.71:3301,v1:172.21.15.71:6790],172.21.15.89" 2024-09-11T06:48:20.784 INFO:teuthology.orchestra.run.smithi176.stdout: mon client directed command retry = 5 2024-09-11T06:48:20.784 INFO:teuthology.orchestra.run.smithi176.stdout: mon warn on pool no app = False 2024-09-11T06:48:20.784 INFO:teuthology.orchestra.run.smithi176.stdout: ms inject socket failures = 5000 2024-09-11T06:48:20.784 INFO:teuthology.orchestra.run.smithi176.stdout: fsid = bd5dcb33-e9f7-4a5f-a26c-a451452e8aa2 2024-09-11T06:48:20.784 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.784 INFO:teuthology.orchestra.run.smithi176.stdout:[osd] 2024-09-11T06:48:20.784 INFO:teuthology.orchestra.run.smithi176.stdout: osd journal size = 100 2024-09-11T06:48:20.784 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.785 INFO:teuthology.orchestra.run.smithi176.stdout: osd scrub load threshold = 5.0 2024-09-11T06:48:20.785 INFO:teuthology.orchestra.run.smithi176.stdout: osd scrub max interval = 600 2024-09-11T06:48:20.785 INFO:teuthology.orchestra.run.smithi176.stdout: osd mclock profile = high_recovery_ops 2024-09-11T06:48:20.785 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.785 INFO:teuthology.orchestra.run.smithi176.stdout: osd recover clone overlap = true 2024-09-11T06:48:20.785 INFO:teuthology.orchestra.run.smithi176.stdout: osd recovery max chunk = 1048576 2024-09-11T06:48:20.786 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.786 INFO:teuthology.orchestra.run.smithi176.stdout: osd debug shutdown = true 2024-09-11T06:48:20.786 INFO:teuthology.orchestra.run.smithi176.stdout: osd debug op order = true 2024-09-11T06:48:20.786 INFO:teuthology.orchestra.run.smithi176.stdout: osd debug verify stray on activate = true 2024-09-11T06:48:20.786 INFO:teuthology.orchestra.run.smithi176.stdout: osd debug trim objects = true 2024-09-11T06:48:20.786 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.786 INFO:teuthology.orchestra.run.smithi176.stdout: osd open classes on start = true 2024-09-11T06:48:20.786 INFO:teuthology.orchestra.run.smithi176.stdout: osd debug pg log writeout = true 2024-09-11T06:48:20.786 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.787 INFO:teuthology.orchestra.run.smithi176.stdout: osd deep scrub update digest min age = 30 2024-09-11T06:48:20.787 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.787 INFO:teuthology.orchestra.run.smithi176.stdout: osd map max advance = 10 2024-09-11T06:48:20.787 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.787 INFO:teuthology.orchestra.run.smithi176.stdout: journal zero on create = true 2024-09-11T06:48:20.787 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.787 INFO:teuthology.orchestra.run.smithi176.stdout: filestore ondisk finisher threads = 3 2024-09-11T06:48:20.787 INFO:teuthology.orchestra.run.smithi176.stdout: filestore apply finisher threads = 3 2024-09-11T06:48:20.787 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.787 INFO:teuthology.orchestra.run.smithi176.stdout: bdev debug aio = true 2024-09-11T06:48:20.787 INFO:teuthology.orchestra.run.smithi176.stdout: osd debug misdirected ops = true 2024-09-11T06:48:20.787 INFO:teuthology.orchestra.run.smithi176.stdout: bluefs allocator = hybrid 2024-09-11T06:48:20.787 INFO:teuthology.orchestra.run.smithi176.stdout: bluestore allocator = hybrid 2024-09-11T06:48:20.787 INFO:teuthology.orchestra.run.smithi176.stdout: bluestore block size = 96636764160 2024-09-11T06:48:20.788 INFO:teuthology.orchestra.run.smithi176.stdout: bluestore fsck on mount = True 2024-09-11T06:48:20.788 INFO:teuthology.orchestra.run.smithi176.stdout: debug bluefs = 1/20 2024-09-11T06:48:20.788 INFO:teuthology.orchestra.run.smithi176.stdout: debug bluestore = 1/20 2024-09-11T06:48:20.788 INFO:teuthology.orchestra.run.smithi176.stdout: debug ms = 1 2024-09-11T06:48:20.788 INFO:teuthology.orchestra.run.smithi176.stdout: debug osd = 20 2024-09-11T06:48:20.788 INFO:teuthology.orchestra.run.smithi176.stdout: debug rocksdb = 4/10 2024-09-11T06:48:20.788 INFO:teuthology.orchestra.run.smithi176.stdout: mon osd backfillfull_ratio = 0.85 2024-09-11T06:48:20.788 INFO:teuthology.orchestra.run.smithi176.stdout: mon osd full ratio = 0.9 2024-09-11T06:48:20.788 INFO:teuthology.orchestra.run.smithi176.stdout: mon osd nearfull ratio = 0.8 2024-09-11T06:48:20.788 INFO:teuthology.orchestra.run.smithi176.stdout: osd blocked scrub grace period = 3600 2024-09-11T06:48:20.788 INFO:teuthology.orchestra.run.smithi176.stdout: osd failsafe full ratio = 0.95 2024-09-11T06:48:20.788 INFO:teuthology.orchestra.run.smithi176.stdout: osd max markdown count = 1000 2024-09-11T06:48:20.788 INFO:teuthology.orchestra.run.smithi176.stdout: osd objectstore = bluestore 2024-09-11T06:48:20.788 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.789 INFO:teuthology.orchestra.run.smithi176.stdout:[mgr] 2024-09-11T06:48:20.789 INFO:teuthology.orchestra.run.smithi176.stdout: debug ms = 1 2024-09-11T06:48:20.789 INFO:teuthology.orchestra.run.smithi176.stdout: debug mgr = 20 2024-09-11T06:48:20.789 INFO:teuthology.orchestra.run.smithi176.stdout: debug mon = 20 2024-09-11T06:48:20.789 INFO:teuthology.orchestra.run.smithi176.stdout: debug auth = 20 2024-09-11T06:48:20.789 INFO:teuthology.orchestra.run.smithi176.stdout: mon reweight min pgs per osd = 4 2024-09-11T06:48:20.789 INFO:teuthology.orchestra.run.smithi176.stdout: mon reweight min bytes per osd = 10 2024-09-11T06:48:20.789 INFO:teuthology.orchestra.run.smithi176.stdout: mgr/telemetry/nag = false 2024-09-11T06:48:20.789 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.789 INFO:teuthology.orchestra.run.smithi176.stdout:[mon] 2024-09-11T06:48:20.789 INFO:teuthology.orchestra.run.smithi176.stdout: debug ms = 1 2024-09-11T06:48:20.789 INFO:teuthology.orchestra.run.smithi176.stdout: debug mon = 20 2024-09-11T06:48:20.789 INFO:teuthology.orchestra.run.smithi176.stdout: debug paxos = 20 2024-09-11T06:48:20.789 INFO:teuthology.orchestra.run.smithi176.stdout: debug auth = 20 2024-09-11T06:48:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: mon data avail warn = 5 2024-09-11T06:48:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: mon mgr mkfs grace = 240 2024-09-11T06:48:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: mon reweight min pgs per osd = 4 2024-09-11T06:48:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: mon osd reporter subtree level = osd 2024-09-11T06:48:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: mon osd prime pg temp = true 2024-09-11T06:48:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: mon reweight min bytes per osd = 10 2024-09-11T06:48:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-11T06:48:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: auth mon ticket ttl = 660 # 11m 2024-09-11T06:48:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: auth service ticket ttl = 240 # 4m 2024-09-11T06:48:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: # don't complain about insecure global_id in the test suite 2024-09-11T06:48:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-11T06:48:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-11T06:48:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.791 INFO:teuthology.orchestra.run.smithi176.stdout: # 1m isn't quite enough 2024-09-11T06:48:20.791 INFO:teuthology.orchestra.run.smithi176.stdout: mon_down_mkfs_grace = 2m 2024-09-11T06:48:20.791 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.791 INFO:teuthology.orchestra.run.smithi176.stdout: mon_warn_on_filestore_osds = false 2024-09-11T06:48:20.791 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-11T06:48:20.791 INFO:teuthology.orchestra.run.smithi176.stdout:[client] 2024-09-11T06:48:20.791 INFO:teuthology.orchestra.run.smithi176.stdout: rgw cache enabled = true 2024-09-11T06:48:20.791 INFO:teuthology.orchestra.run.smithi176.stdout: rgw enable ops log = true 2024-09-11T06:48:20.791 INFO:teuthology.orchestra.run.smithi176.stdout: rgw enable usage log = true 2024-09-11T06:48:20.791 INFO:teuthology.orchestra.run.smithi176.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-11T06:48:20.791 INFO:teuthology.orchestra.run.smithi176.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-11T06:48:20.791 INFO:teuthology.orchestra.run.smithi176.stdout:[mon.a] 2024-09-11T06:48:20.791 INFO:teuthology.orchestra.run.smithi176.stdout:[mon.c] 2024-09-11T06:48:20.791 INFO:teuthology.orchestra.run.smithi176.stdout:[mon.b] 2024-09-11T06:48:20.792 INFO:teuthology.orchestra.run.smithi089.stdout:[global] 2024-09-11T06:48:20.792 INFO:teuthology.orchestra.run.smithi089.stdout: chdir = "" 2024-09-11T06:48:20.792 INFO:teuthology.orchestra.run.smithi089.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-11T06:48:20.792 INFO:teuthology.orchestra.run.smithi089.stdout: auth supported = cephx 2024-09-11T06:48:20.792 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.792 INFO:teuthology.orchestra.run.smithi089.stdout: filestore xattr use omap = true 2024-09-11T06:48:20.792 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.792 INFO:teuthology.orchestra.run.smithi089.stdout: mon clock drift allowed = 1.000 2024-09-11T06:48:20.792 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.792 INFO:teuthology.orchestra.run.smithi089.stdout: osd crush chooseleaf type = 0 2024-09-11T06:48:20.792 INFO:teuthology.orchestra.run.smithi089.stdout: auth debug = true 2024-09-11T06:48:20.793 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.793 INFO:teuthology.orchestra.run.smithi089.stdout: ms die on old message = true 2024-09-11T06:48:20.793 INFO:teuthology.orchestra.run.smithi089.stdout: ms die on bug = true 2024-09-11T06:48:20.793 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.793 INFO:teuthology.orchestra.run.smithi089.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-11T06:48:20.793 INFO:teuthology.orchestra.run.smithi089.stdout: mon pg warn max object skew = 0 2024-09-11T06:48:20.793 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.793 INFO:teuthology.orchestra.run.smithi089.stdout: # disable pg_autoscaler by default for new pools 2024-09-11T06:48:20.793 INFO:teuthology.orchestra.run.smithi089.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-11T06:48:20.793 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.793 INFO:teuthology.orchestra.run.smithi089.stdout: osd pool default size = 2 2024-09-11T06:48:20.794 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.794 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd allow primary affinity = true 2024-09-11T06:48:20.794 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd allow pg remap = true 2024-09-11T06:48:20.794 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on legacy crush tunables = false 2024-09-11T06:48:20.794 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on crush straw calc version zero = false 2024-09-11T06:48:20.794 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on no sortbitwise = false 2024-09-11T06:48:20.794 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on osd down out interval zero = false 2024-09-11T06:48:20.794 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on too few osds = false 2024-09-11T06:48:20.794 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-11T06:48:20.794 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-11T06:48:20.794 INFO:teuthology.orchestra.run.smithi089.stdout: mon_allow_pool_size_one = true 2024-09-11T06:48:20.794 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.794 INFO:teuthology.orchestra.run.smithi089.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-11T06:48:20.794 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.795 INFO:teuthology.orchestra.run.smithi089.stdout: osd default data pool replay window = 5 2024-09-11T06:48:20.795 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.795 INFO:teuthology.orchestra.run.smithi089.stdout: mon allow pool delete = true 2024-09-11T06:48:20.795 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.795 INFO:teuthology.orchestra.run.smithi089.stdout: mon cluster log file level = debug 2024-09-11T06:48:20.795 INFO:teuthology.orchestra.run.smithi089.stdout: debug asserts on shutdown = true 2024-09-11T06:48:20.795 INFO:teuthology.orchestra.run.smithi089.stdout: mon health detail to clog = false 2024-09-11T06:48:20.795 INFO:teuthology.orchestra.run.smithi089.stdout: mon host = "172.21.15.71,[v2:172.21.15.71:3301,v1:172.21.15.71:6790],172.21.15.89" 2024-09-11T06:48:20.795 INFO:teuthology.orchestra.run.smithi089.stdout: mon client directed command retry = 5 2024-09-11T06:48:20.795 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on pool no app = False 2024-09-11T06:48:20.795 INFO:teuthology.orchestra.run.smithi089.stdout: ms inject socket failures = 5000 2024-09-11T06:48:20.795 INFO:teuthology.orchestra.run.smithi089.stdout: fsid = bd5dcb33-e9f7-4a5f-a26c-a451452e8aa2 2024-09-11T06:48:20.795 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.795 INFO:teuthology.orchestra.run.smithi089.stdout:[osd] 2024-09-11T06:48:20.795 INFO:teuthology.orchestra.run.smithi089.stdout: osd journal size = 100 2024-09-11T06:48:20.795 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.796 INFO:teuthology.orchestra.run.smithi089.stdout: osd scrub load threshold = 5.0 2024-09-11T06:48:20.796 INFO:teuthology.orchestra.run.smithi089.stdout: osd scrub max interval = 600 2024-09-11T06:48:20.796 INFO:teuthology.orchestra.run.smithi089.stdout: osd mclock profile = high_recovery_ops 2024-09-11T06:48:20.796 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.796 INFO:teuthology.orchestra.run.smithi089.stdout: osd recover clone overlap = true 2024-09-11T06:48:20.796 INFO:teuthology.orchestra.run.smithi089.stdout: osd recovery max chunk = 1048576 2024-09-11T06:48:20.796 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.796 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug shutdown = true 2024-09-11T06:48:20.796 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug op order = true 2024-09-11T06:48:20.796 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug verify stray on activate = true 2024-09-11T06:48:20.796 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug trim objects = true 2024-09-11T06:48:20.796 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.796 INFO:teuthology.orchestra.run.smithi089.stdout: osd open classes on start = true 2024-09-11T06:48:20.796 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug pg log writeout = true 2024-09-11T06:48:20.796 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.797 INFO:teuthology.orchestra.run.smithi089.stdout: osd deep scrub update digest min age = 30 2024-09-11T06:48:20.797 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.797 INFO:teuthology.orchestra.run.smithi089.stdout: osd map max advance = 10 2024-09-11T06:48:20.797 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.797 INFO:teuthology.orchestra.run.smithi089.stdout: journal zero on create = true 2024-09-11T06:48:20.797 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.797 INFO:teuthology.orchestra.run.smithi089.stdout: filestore ondisk finisher threads = 3 2024-09-11T06:48:20.797 INFO:teuthology.orchestra.run.smithi089.stdout: filestore apply finisher threads = 3 2024-09-11T06:48:20.797 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.797 INFO:teuthology.orchestra.run.smithi089.stdout: bdev debug aio = true 2024-09-11T06:48:20.797 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug misdirected ops = true 2024-09-11T06:48:20.797 INFO:teuthology.orchestra.run.smithi089.stdout: bluefs allocator = hybrid 2024-09-11T06:48:20.797 INFO:teuthology.orchestra.run.smithi089.stdout: bluestore allocator = hybrid 2024-09-11T06:48:20.797 INFO:teuthology.orchestra.run.smithi089.stdout: bluestore block size = 96636764160 2024-09-11T06:48:20.797 INFO:teuthology.orchestra.run.smithi089.stdout: bluestore fsck on mount = True 2024-09-11T06:48:20.798 INFO:teuthology.orchestra.run.smithi089.stdout: debug bluefs = 1/20 2024-09-11T06:48:20.798 INFO:teuthology.orchestra.run.smithi089.stdout: debug bluestore = 1/20 2024-09-11T06:48:20.798 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2024-09-11T06:48:20.798 INFO:teuthology.orchestra.run.smithi089.stdout: debug osd = 20 2024-09-11T06:48:20.798 INFO:teuthology.orchestra.run.smithi089.stdout: debug rocksdb = 4/10 2024-09-11T06:48:20.798 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd backfillfull_ratio = 0.85 2024-09-11T06:48:20.798 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd full ratio = 0.9 2024-09-11T06:48:20.798 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd nearfull ratio = 0.8 2024-09-11T06:48:20.798 INFO:teuthology.orchestra.run.smithi089.stdout: osd blocked scrub grace period = 3600 2024-09-11T06:48:20.798 INFO:teuthology.orchestra.run.smithi089.stdout: osd failsafe full ratio = 0.95 2024-09-11T06:48:20.798 INFO:teuthology.orchestra.run.smithi089.stdout: osd max markdown count = 1000 2024-09-11T06:48:20.798 INFO:teuthology.orchestra.run.smithi089.stdout: osd objectstore = bluestore 2024-09-11T06:48:20.798 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.798 INFO:teuthology.orchestra.run.smithi089.stdout:[mgr] 2024-09-11T06:48:20.798 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2024-09-11T06:48:20.799 INFO:teuthology.orchestra.run.smithi089.stdout: debug mgr = 20 2024-09-11T06:48:20.799 INFO:teuthology.orchestra.run.smithi089.stdout: debug mon = 20 2024-09-11T06:48:20.799 INFO:teuthology.orchestra.run.smithi089.stdout: debug auth = 20 2024-09-11T06:48:20.799 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min pgs per osd = 4 2024-09-11T06:48:20.799 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min bytes per osd = 10 2024-09-11T06:48:20.799 INFO:teuthology.orchestra.run.smithi089.stdout: mgr/telemetry/nag = false 2024-09-11T06:48:20.799 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.799 INFO:teuthology.orchestra.run.smithi089.stdout:[mon] 2024-09-11T06:48:20.799 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2024-09-11T06:48:20.799 INFO:teuthology.orchestra.run.smithi089.stdout: debug mon = 20 2024-09-11T06:48:20.799 INFO:teuthology.orchestra.run.smithi089.stdout: debug paxos = 20 2024-09-11T06:48:20.799 INFO:teuthology.orchestra.run.smithi089.stdout: debug auth = 20 2024-09-11T06:48:20.799 INFO:teuthology.orchestra.run.smithi089.stdout: mon data avail warn = 5 2024-09-11T06:48:20.799 INFO:teuthology.orchestra.run.smithi089.stdout: mon mgr mkfs grace = 240 2024-09-11T06:48:20.800 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min pgs per osd = 4 2024-09-11T06:48:20.800 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd reporter subtree level = osd 2024-09-11T06:48:20.800 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd prime pg temp = true 2024-09-11T06:48:20.800 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min bytes per osd = 10 2024-09-11T06:48:20.800 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.800 INFO:teuthology.orchestra.run.smithi089.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-11T06:48:20.800 INFO:teuthology.orchestra.run.smithi089.stdout: auth mon ticket ttl = 660 # 11m 2024-09-11T06:48:20.800 INFO:teuthology.orchestra.run.smithi089.stdout: auth service ticket ttl = 240 # 4m 2024-09-11T06:48:20.800 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.800 INFO:teuthology.orchestra.run.smithi089.stdout: # don't complain about insecure global_id in the test suite 2024-09-11T06:48:20.800 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-11T06:48:20.800 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-11T06:48:20.800 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.800 INFO:teuthology.orchestra.run.smithi089.stdout: # 1m isn't quite enough 2024-09-11T06:48:20.800 INFO:teuthology.orchestra.run.smithi089.stdout: mon_down_mkfs_grace = 2m 2024-09-11T06:48:20.801 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.801 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_filestore_osds = false 2024-09-11T06:48:20.801 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-11T06:48:20.801 INFO:teuthology.orchestra.run.smithi089.stdout:[client] 2024-09-11T06:48:20.801 INFO:teuthology.orchestra.run.smithi089.stdout: rgw cache enabled = true 2024-09-11T06:48:20.801 INFO:teuthology.orchestra.run.smithi089.stdout: rgw enable ops log = true 2024-09-11T06:48:20.801 INFO:teuthology.orchestra.run.smithi089.stdout: rgw enable usage log = true 2024-09-11T06:48:20.801 INFO:teuthology.orchestra.run.smithi089.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-11T06:48:20.801 INFO:teuthology.orchestra.run.smithi089.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-11T06:48:20.801 INFO:teuthology.orchestra.run.smithi089.stdout:[mon.a] 2024-09-11T06:48:20.801 INFO:teuthology.orchestra.run.smithi089.stdout:[mon.c] 2024-09-11T06:48:20.801 INFO:teuthology.orchestra.run.smithi089.stdout:[mon.b] 2024-09-11T06:48:20.803 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-11T06:48:20.803 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-11T06:48:20.892 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-11T06:48:20.892 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T06:48:20.893 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-11T06:48:20.897 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T06:48:20.897 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-11T06:48:20.948 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi071.front.sepia.ceph.com 2024-09-11T06:48:20.949 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T06:48:20.949 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-11T06:48:20.949 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-11T06:48:21.012 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T06:48:21.013 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-11T06:48:21.060 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi089.front.sepia.ceph.com 2024-09-11T06:48:21.060 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-11T06:48:21.060 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-11T06:48:21.060 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-11T06:48:21.083 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-11T06:48:21.083 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-11T06:48:21.132 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi176.front.sepia.ceph.com 2024-09-11T06:48:21.132 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-11T06:48:21.132 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-11T06:48:21.132 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-11T06:48:21.155 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-11T06:48:21.155 DEBUG:teuthology.orchestra.run.smithi176:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-11T06:48:21.203 INFO:tasks.ceph:Setting up mon nodes... 2024-09-11T06:48:21.203 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-11T06:48:21.203 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-11T06:48:21.248 INFO:teuthology.orchestra.run.smithi089.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-11T06:48:21.256 INFO:tasks.ceph:Setting up mds nodes... 2024-09-11T06:48:21.256 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-11T06:48:21.256 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-11T06:48:21.291 INFO:teuthology.orchestra.run.smithi176.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-11T06:48:21.322 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-11T06:48:21.322 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi071.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi089.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-11T06:48:21.322 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-11T06:48:21.337 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-11T06:48:21.337 INFO:tasks.ceph:role: osd.0 2024-09-11T06:48:21.337 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi071.front.sepia.ceph.com 2024-09-11T06:48:21.337 DEBUG:teuthology.orchestra.run.smithi071:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-11T06:48:21.400 INFO:teuthology.orchestra.run.smithi071.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T06:48:21.400 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T06:48:21.400 INFO:teuthology.orchestra.run.smithi071.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T06:48:21.400 INFO:teuthology.orchestra.run.smithi071.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-11T06:48:21.400 INFO:teuthology.orchestra.run.smithi071.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T06:48:21.400 INFO:teuthology.orchestra.run.smithi071.stdout: = sunit=0 swidth=0 blks 2024-09-11T06:48:21.401 INFO:teuthology.orchestra.run.smithi071.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T06:48:21.401 INFO:teuthology.orchestra.run.smithi071.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-11T06:48:21.401 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T06:48:21.401 INFO:teuthology.orchestra.run.smithi071.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T06:48:21.857 INFO:teuthology.orchestra.run.smithi071.stdout:Discarding blocks...Done. 2024-09-11T06:48:21.859 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi071.front.sepia.ceph.com -o noatime 2024-09-11T06:48:21.859 DEBUG:teuthology.orchestra.run.smithi071:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-11T06:48:21.958 DEBUG:teuthology.orchestra.run.smithi071:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-11T06:48:21.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T06:48:21.968 INFO:teuthology.orchestra.run.smithi071.stderr:sudo: /sbin/restorecon: command not found 2024-09-11T06:48:21.969 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-11T06:48:22.023 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-11T06:48:22.023 INFO:tasks.ceph:role: osd.1 2024-09-11T06:48:22.023 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi071.front.sepia.ceph.com 2024-09-11T06:48:22.023 DEBUG:teuthology.orchestra.run.smithi071:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-11T06:48:22.083 INFO:teuthology.orchestra.run.smithi071.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T06:48:22.083 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T06:48:22.083 INFO:teuthology.orchestra.run.smithi071.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T06:48:22.083 INFO:teuthology.orchestra.run.smithi071.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-11T06:48:22.084 INFO:teuthology.orchestra.run.smithi071.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T06:48:22.084 INFO:teuthology.orchestra.run.smithi071.stdout: = sunit=0 swidth=0 blks 2024-09-11T06:48:22.084 INFO:teuthology.orchestra.run.smithi071.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T06:48:22.084 INFO:teuthology.orchestra.run.smithi071.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-11T06:48:22.084 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T06:48:22.084 INFO:teuthology.orchestra.run.smithi071.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T06:48:22.553 INFO:teuthology.orchestra.run.smithi071.stdout:Discarding blocks...Done. 2024-09-11T06:48:22.555 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi071.front.sepia.ceph.com -o noatime 2024-09-11T06:48:22.555 DEBUG:teuthology.orchestra.run.smithi071:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-11T06:48:22.682 DEBUG:teuthology.orchestra.run.smithi071:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-11T06:48:22.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T06:48:22.693 INFO:teuthology.orchestra.run.smithi071.stderr:sudo: /sbin/restorecon: command not found 2024-09-11T06:48:22.693 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-11T06:48:22.746 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-11T06:48:22.747 INFO:tasks.ceph:role: osd.2 2024-09-11T06:48:22.747 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi071.front.sepia.ceph.com 2024-09-11T06:48:22.747 DEBUG:teuthology.orchestra.run.smithi071:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-11T06:48:22.807 INFO:teuthology.orchestra.run.smithi071.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T06:48:22.807 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T06:48:22.807 INFO:teuthology.orchestra.run.smithi071.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T06:48:22.807 INFO:teuthology.orchestra.run.smithi071.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-11T06:48:22.807 INFO:teuthology.orchestra.run.smithi071.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T06:48:22.807 INFO:teuthology.orchestra.run.smithi071.stdout: = sunit=0 swidth=0 blks 2024-09-11T06:48:22.807 INFO:teuthology.orchestra.run.smithi071.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T06:48:22.807 INFO:teuthology.orchestra.run.smithi071.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-11T06:48:22.807 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T06:48:22.807 INFO:teuthology.orchestra.run.smithi071.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T06:48:23.304 INFO:teuthology.orchestra.run.smithi071.stdout:Discarding blocks...Done. 2024-09-11T06:48:23.307 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi071.front.sepia.ceph.com -o noatime 2024-09-11T06:48:23.307 DEBUG:teuthology.orchestra.run.smithi071:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-11T06:48:23.450 DEBUG:teuthology.orchestra.run.smithi071:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-11T06:48:23.461 INFO:teuthology.orchestra.run.smithi071.stderr:sudo: /sbin/restorecon: command not found 2024-09-11T06:48:23.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T06:48:23.462 DEBUG:teuthology.orchestra.run.smithi071:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T06:48:23.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-11T06:48:23.567+0000 7f568d98d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-11T06:48:23.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-11T06:48:23.571+0000 7f568d98d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-11T06:48:23.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-11T06:48:23.571+0000 7f568d98d7c0 -1 bdev(0x55e1173cd800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-11T06:48:23.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-11T06:48:23.571+0000 7f568d98d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-11T06:48:26.696 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-11T06:48:26.709 DEBUG:teuthology.orchestra.run.smithi071:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T06:48:26.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-11T06:48:26.787+0000 7f6f2c3a07c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-11T06:48:26.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-11T06:48:26.787+0000 7f6f2c3a07c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-11T06:48:26.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-11T06:48:26.787+0000 7f6f2c3a07c0 -1 bdev(0x557651127800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-11T06:48:26.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-11T06:48:26.787+0000 7f6f2c3a07c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-11T06:48:29.920 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-11T06:48:29.976 DEBUG:teuthology.orchestra.run.smithi071:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T06:48:30.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-11T06:48:30.054+0000 7ff05f3967c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-11T06:48:30.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-11T06:48:30.054+0000 7ff05f3967c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-11T06:48:30.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-11T06:48:30.054+0000 7ff05f3967c0 -1 bdev(0x560c73baf800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-11T06:48:30.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-11T06:48:30.054+0000 7ff05f3967c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-11T06:48:33.178 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-11T06:48:33.232 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-11T06:48:33.247 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-11T06:48:33.247 INFO:tasks.ceph:role: osd.3 2024-09-11T06:48:33.248 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi089.front.sepia.ceph.com 2024-09-11T06:48:33.248 DEBUG:teuthology.orchestra.run.smithi089:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-11T06:48:33.313 INFO:teuthology.orchestra.run.smithi089.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T06:48:33.313 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T06:48:33.313 INFO:teuthology.orchestra.run.smithi089.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T06:48:33.313 INFO:teuthology.orchestra.run.smithi089.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-11T06:48:33.313 INFO:teuthology.orchestra.run.smithi089.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T06:48:33.313 INFO:teuthology.orchestra.run.smithi089.stdout: = sunit=0 swidth=0 blks 2024-09-11T06:48:33.313 INFO:teuthology.orchestra.run.smithi089.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T06:48:33.314 INFO:teuthology.orchestra.run.smithi089.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-11T06:48:33.314 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T06:48:33.314 INFO:teuthology.orchestra.run.smithi089.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T06:48:33.786 INFO:teuthology.orchestra.run.smithi089.stdout:Discarding blocks...Done. 2024-09-11T06:48:33.788 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi089.front.sepia.ceph.com -o noatime 2024-09-11T06:48:33.788 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-09-11T06:48:33.876 DEBUG:teuthology.orchestra.run.smithi089:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-11T06:48:33.928 INFO:teuthology.orchestra.run.smithi089.stderr:sudo: /sbin/restorecon: command not found 2024-09-11T06:48:33.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T06:48:33.929 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-11T06:48:33.983 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-11T06:48:33.983 INFO:tasks.ceph:role: osd.4 2024-09-11T06:48:33.983 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi089.front.sepia.ceph.com 2024-09-11T06:48:33.983 DEBUG:teuthology.orchestra.run.smithi089:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-11T06:48:34.046 INFO:teuthology.orchestra.run.smithi089.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T06:48:34.046 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T06:48:34.046 INFO:teuthology.orchestra.run.smithi089.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T06:48:34.047 INFO:teuthology.orchestra.run.smithi089.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-11T06:48:34.047 INFO:teuthology.orchestra.run.smithi089.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T06:48:34.047 INFO:teuthology.orchestra.run.smithi089.stdout: = sunit=0 swidth=0 blks 2024-09-11T06:48:34.047 INFO:teuthology.orchestra.run.smithi089.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T06:48:34.047 INFO:teuthology.orchestra.run.smithi089.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-11T06:48:34.047 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T06:48:34.047 INFO:teuthology.orchestra.run.smithi089.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T06:48:34.522 INFO:teuthology.orchestra.run.smithi089.stdout:Discarding blocks...Done. 2024-09-11T06:48:34.524 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi089.front.sepia.ceph.com -o noatime 2024-09-11T06:48:34.525 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-09-11T06:48:34.666 DEBUG:teuthology.orchestra.run.smithi089:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-11T06:48:34.677 INFO:teuthology.orchestra.run.smithi089.stderr:sudo: /sbin/restorecon: command not found 2024-09-11T06:48:34.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T06:48:34.678 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-11T06:48:34.731 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-11T06:48:34.731 INFO:tasks.ceph:role: osd.5 2024-09-11T06:48:34.731 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi089.front.sepia.ceph.com 2024-09-11T06:48:34.731 DEBUG:teuthology.orchestra.run.smithi089:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-11T06:48:34.794 INFO:teuthology.orchestra.run.smithi089.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T06:48:34.795 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T06:48:34.795 INFO:teuthology.orchestra.run.smithi089.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T06:48:34.795 INFO:teuthology.orchestra.run.smithi089.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-11T06:48:34.795 INFO:teuthology.orchestra.run.smithi089.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T06:48:34.795 INFO:teuthology.orchestra.run.smithi089.stdout: = sunit=0 swidth=0 blks 2024-09-11T06:48:34.795 INFO:teuthology.orchestra.run.smithi089.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T06:48:34.795 INFO:teuthology.orchestra.run.smithi089.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-11T06:48:34.795 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T06:48:34.795 INFO:teuthology.orchestra.run.smithi089.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T06:48:35.274 INFO:teuthology.orchestra.run.smithi089.stdout:Discarding blocks...Done. 2024-09-11T06:48:35.276 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi089.front.sepia.ceph.com -o noatime 2024-09-11T06:48:35.276 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5 2024-09-11T06:48:35.418 DEBUG:teuthology.orchestra.run.smithi089:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-11T06:48:35.430 INFO:teuthology.orchestra.run.smithi089.stderr:sudo: /sbin/restorecon: command not found 2024-09-11T06:48:35.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T06:48:35.431 DEBUG:teuthology.orchestra.run.smithi089:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T06:48:35.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-11T06:48:35.534+0000 7f85705707c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-11T06:48:35.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-11T06:48:35.534+0000 7f85705707c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-11T06:48:35.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-11T06:48:35.534+0000 7f85705707c0 -1 bdev(0x561922523800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-11T06:48:35.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-11T06:48:35.534+0000 7f85705707c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-11T06:48:38.684 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-11T06:48:38.699 DEBUG:teuthology.orchestra.run.smithi089:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T06:48:38.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-11T06:48:38.782+0000 7f878e6507c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-11T06:48:38.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-11T06:48:38.782+0000 7f878e6507c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-11T06:48:38.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-11T06:48:38.782+0000 7f878e6507c0 -1 bdev(0x55c3802b1800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-11T06:48:38.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-11T06:48:38.782+0000 7f878e6507c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-11T06:48:41.917 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-11T06:48:41.971 DEBUG:teuthology.orchestra.run.smithi089:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T06:48:42.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-11T06:48:42.050+0000 7fc1a94517c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-11T06:48:42.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-11T06:48:42.050+0000 7fc1a94517c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-11T06:48:42.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-11T06:48:42.050+0000 7fc1a94517c0 -1 bdev(0x55d45468d800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-11T06:48:42.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-11T06:48:42.050+0000 7fc1a94517c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-11T06:48:45.231 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-11T06:48:45.288 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-11T06:48:45.289 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T06:48:45.289 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-11T06:48:45.304 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T06:48:45.304 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-11T06:48:45.360 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T06:48:45.360 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-11T06:48:45.414 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-11T06:48:45.415 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-11T06:48:45.427 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-11T06:48:45.427 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-11T06:48:45.484 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-11T06:48:45.484 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-11T06:48:45.538 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-11T06:48:45.538 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-11T06:48:45.594 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-11T06:48:45.594 DEBUG:teuthology.orchestra.run.smithi176:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-11T06:48:45.600 INFO:tasks.ceph:Adding keys to all mons... 2024-09-11T06:48:45.601 DEBUG:teuthology.orchestra.run.smithi071:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-11T06:48:45.605 DEBUG:teuthology.orchestra.run.smithi089:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-11T06:48:45.637 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.0] 2024-09-11T06:48:45.637 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQA3PeFmk74nIhAA7QsLw1o1gxuzY/eD+4CIpg== 2024-09-11T06:48:45.637 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.1] 2024-09-11T06:48:45.637 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQA6PeFmJ+0sLxAATYSJLGs/IGrmFKoCBso/yg== 2024-09-11T06:48:45.637 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.2] 2024-09-11T06:48:45.637 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQA+PeFm/M6KAxAAYd4LFWdKQADEElVrFgPaKA== 2024-09-11T06:48:45.638 INFO:teuthology.orchestra.run.smithi071.stdout:[mgr.x] 2024-09-11T06:48:45.638 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQA1PeFmyCfiDhAAjsrEGUE/y8qCXNPdXj3CdQ== 2024-09-11T06:48:45.638 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.3] 2024-09-11T06:48:45.638 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBDPeFm5ZYoIBAADPClG9LhT1YaCsSgnx5xwQ== 2024-09-11T06:48:45.638 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.4] 2024-09-11T06:48:45.638 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBGPeFmj7nlLhAAjowGIls8LEZH24RJ07CC7g== 2024-09-11T06:48:45.638 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.5] 2024-09-11T06:48:45.638 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBKPeFmSbZEAxAAEskm7/EHbgh7v/EzbK/EAQ== 2024-09-11T06:48:45.638 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2024-09-11T06:48:45.638 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQA1PeFmyWRjERAAiJqKT+m513szKEB29/WCkA== 2024-09-11T06:48:45.645 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.0] 2024-09-11T06:48:45.645 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQA3PeFmk74nIhAA7QsLw1o1gxuzY/eD+4CIpg== 2024-09-11T06:48:45.645 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.1] 2024-09-11T06:48:45.645 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQA6PeFmJ+0sLxAATYSJLGs/IGrmFKoCBso/yg== 2024-09-11T06:48:45.645 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.2] 2024-09-11T06:48:45.645 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQA+PeFm/M6KAxAAYd4LFWdKQADEElVrFgPaKA== 2024-09-11T06:48:45.645 INFO:teuthology.orchestra.run.smithi089.stdout:[mgr.x] 2024-09-11T06:48:45.645 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQA1PeFmyCfiDhAAjsrEGUE/y8qCXNPdXj3CdQ== 2024-09-11T06:48:45.645 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.3] 2024-09-11T06:48:45.645 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBDPeFm5ZYoIBAADPClG9LhT1YaCsSgnx5xwQ== 2024-09-11T06:48:45.645 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.4] 2024-09-11T06:48:45.646 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBGPeFmj7nlLhAAjowGIls8LEZH24RJ07CC7g== 2024-09-11T06:48:45.646 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.5] 2024-09-11T06:48:45.646 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBKPeFmSbZEAxAAEskm7/EHbgh7v/EzbK/EAQ== 2024-09-11T06:48:45.646 INFO:teuthology.orchestra.run.smithi089.stdout:[client.0] 2024-09-11T06:48:45.646 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQA1PeFmyWRjERAAiJqKT+m513szKEB29/WCkA== 2024-09-11T06:48:45.647 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T06:48:45.685 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T06:48:45.735 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T06:48:45.773 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T06:48:45.814 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T06:48:45.857 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T06:48:45.898 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-11T06:48:45.941 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-11T06:48:45.981 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T06:48:46.025 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T06:48:46.069 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T06:48:46.109 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T06:48:46.151 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T06:48:46.193 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T06:48:46.236 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-11T06:48:46.277 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-11T06:48:46.321 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-11T06:48:46.321 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-11T06:48:46.370 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-11T06:48:46.463 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-11T06:48:46.473 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-11T06:48:46.524 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-11T06:48:46.610 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-11T06:48:46.661 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-11T06:48:46.676 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-11T06:48:46.770 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-11T06:48:46.781 DEBUG:teuthology.orchestra.run.smithi071:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-11T06:48:46.784 DEBUG:teuthology.orchestra.run.smithi089:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-11T06:48:46.827 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-11T06:48:46.827 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-11T06:48:46.827 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-11T06:48:46.830 INFO:tasks.ceph.mon.a:Started 2024-09-11T06:48:46.830 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-11T06:48:46.830 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-11T06:48:46.832 INFO:tasks.ceph.mon.c:Started 2024-09-11T06:48:46.832 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-11T06:48:46.832 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-11T06:48:46.872 INFO:tasks.ceph.mon.b:Started 2024-09-11T06:48:46.872 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-11T06:48:46.873 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-11T06:48:46.873 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-11T06:48:46.875 INFO:tasks.ceph.mgr.x:Started 2024-09-11T06:48:46.876 INFO:tasks.ceph:Setting crush tunables to default 2024-09-11T06:48:46.876 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph osd crush tunables default 2024-09-11T06:48:52.377 INFO:teuthology.orchestra.run.smithi071.stderr:adjusted tunables profile to default 2024-09-11T06:48:52.396 INFO:tasks.ceph:check_enable_crimson: False 2024-09-11T06:48:52.396 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-11T06:48:52.396 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T06:48:52.397 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-11T06:48:52.409 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T06:48:52.409 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-11T06:48:52.463 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T06:48:52.463 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-11T06:48:52.496 INFO:tasks.ceph.mgr.x.smithi089.stderr:2024-09-11T06:48:52.494+0000 7f1ec64d1180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T06:48:52.519 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-11T06:48:52.519 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-11T06:48:52.532 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-11T06:48:52.532 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-11T06:48:52.586 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-11T06:48:52.586 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-11T06:48:52.638 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd new 3c9e35bf-d3db-49db-b10c-d2cf38c7eb7e 0 2024-09-11T06:48:52.900 INFO:teuthology.orchestra.run.smithi089.stdout:0 2024-09-11T06:48:52.917 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd new 693649dc-db7c-43cf-bdbe-87f6128ef311 1 2024-09-11T06:48:53.033 INFO:tasks.ceph.mgr.x.smithi089.stderr:2024-09-11T06:48:53.030+0000 7f1ec64d1180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T06:48:53.205 INFO:tasks.ceph.mgr.x.smithi089.stderr:2024-09-11T06:48:53.202+0000 7f1ec64d1180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T06:48:53.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-11T06:48:53.234+0000 7f7ebe7fc640 0 --2- 172.21.15.89:0/2722556636 >> v2:172.21.15.89:3300/0 conn(0x7f7ec0073a50 0x7f7ec0134590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-11T06:48:53.324 INFO:teuthology.orchestra.run.smithi089.stdout:1 2024-09-11T06:48:53.340 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd new 89421ae7-fead-409c-83f7-ede424db6004 2 2024-09-11T06:48:53.545 INFO:teuthology.orchestra.run.smithi089.stdout:2 2024-09-11T06:48:53.561 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd new f4512e48-63f1-42b6-81be-594bd82e3210 3 2024-09-11T06:48:53.762 INFO:teuthology.orchestra.run.smithi089.stdout:3 2024-09-11T06:48:53.778 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd new 52d47b6f-af30-455e-8326-e53c598911ae 4 2024-09-11T06:48:53.783 INFO:tasks.ceph.mgr.x.smithi089.stderr:2024-09-11T06:48:53.782+0000 7f1ec64d1180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T06:48:53.839 INFO:tasks.ceph.mgr.x.smithi089.stderr:2024-09-11T06:48:53.838+0000 7f1ec64d1180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T06:48:53.894 INFO:tasks.ceph.mgr.x.smithi089.stderr:2024-09-11T06:48:53.890+0000 7f1ec64d1180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T06:48:53.972 INFO:tasks.ceph.mgr.x.smithi089.stderr:2024-09-11T06:48:53.970+0000 7f1ec64d1180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T06:48:53.973 INFO:teuthology.orchestra.run.smithi089.stdout:4 2024-09-11T06:48:53.989 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd new 6b4f9416-0ba0-4c58-883e-8cbdb362adcc 5 2024-09-11T06:48:54.026 INFO:tasks.ceph.mgr.x.smithi089.stderr:2024-09-11T06:48:54.022+0000 7f1ec64d1180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T06:48:54.105 INFO:tasks.ceph.mgr.x.smithi089.stderr:2024-09-11T06:48:54.102+0000 7f1ec64d1180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T06:48:54.160 INFO:tasks.ceph.mgr.x.smithi089.stderr:2024-09-11T06:48:54.158+0000 7f1ec64d1180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T06:48:54.216 INFO:teuthology.orchestra.run.smithi089.stdout:5 2024-09-11T06:48:54.232 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-11T06:48:54.233 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-11T06:48:54.235 INFO:tasks.ceph.osd.0:Started 2024-09-11T06:48:54.235 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-11T06:48:54.235 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-11T06:48:54.238 INFO:tasks.ceph.osd.1:Started 2024-09-11T06:48:54.238 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-11T06:48:54.238 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-11T06:48:54.240 INFO:tasks.ceph.osd.2:Started 2024-09-11T06:48:54.240 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-11T06:48:54.241 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-11T06:48:54.242 INFO:tasks.ceph.osd.3:Started 2024-09-11T06:48:54.242 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-11T06:48:54.243 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-11T06:48:54.244 INFO:tasks.ceph.osd.4:Started 2024-09-11T06:48:54.244 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-11T06:48:54.244 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-11T06:48:54.246 INFO:tasks.ceph.osd.5:Started 2024-09-11T06:48:54.246 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T06:48:54.326 INFO:tasks.ceph.mgr.x.smithi089.stderr:2024-09-11T06:48:54.322+0000 7f1ec64d1180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T06:48:54.387 INFO:tasks.ceph.mgr.x.smithi089.stderr:2024-09-11T06:48:54.386+0000 7f1ec64d1180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T06:48:54.442 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:48:54.442 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":8,"fsid":"bd5dcb33-e9f7-4a5f-a26c-a451452e8aa2","created":"2024-09-11T06:48:52.191706+0000","modified":"2024-09-11T06:48:54.213376+0000","last_up_change":"0.000000","last_in_change":"2024-09-11T06:48:54.213376+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3c9e35bf-d3db-49db-b10c-d2cf38c7eb7e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"693649dc-db7c-43cf-bdbe-87f6128ef311","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"89421ae7-fead-409c-83f7-ede424db6004","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f4512e48-63f1-42b6-81be-594bd82e3210","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"52d47b6f-af30-455e-8326-e53c598911ae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6b4f9416-0ba0-4c58-883e-8cbdb362adcc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T06:48:54.459 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-11T06:48:54.459 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-11T06:48:55.065 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-11T06:48:55.076 INFO:tasks.ceph.mgr.x.smithi089.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-11T06:48:55.076 INFO:tasks.ceph.mgr.x.smithi089.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-11T06:48:55.076 INFO:tasks.ceph.mgr.x.smithi089.stderr: from numpy import show_config as show_numpy_config 2024-09-11T06:48:55.132 INFO:tasks.ceph.mgr.x.smithi089.stderr:2024-09-11T06:48:55.130+0000 7f1ec64d1180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T06:48:55.253 INFO:teuthology.misc.health.smithi071.stdout: 2024-09-11T06:48:55.253 INFO:teuthology.misc.health.smithi071.stdout:{"epoch":8,"fsid":"bd5dcb33-e9f7-4a5f-a26c-a451452e8aa2","created":"2024-09-11T06:48:52.191706+0000","modified":"2024-09-11T06:48:54.213376+0000","last_up_change":"0.000000","last_in_change":"2024-09-11T06:48:54.213376+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3c9e35bf-d3db-49db-b10c-d2cf38c7eb7e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"693649dc-db7c-43cf-bdbe-87f6128ef311","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"89421ae7-fead-409c-83f7-ede424db6004","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f4512e48-63f1-42b6-81be-594bd82e3210","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"52d47b6f-af30-455e-8326-e53c598911ae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6b4f9416-0ba0-4c58-883e-8cbdb362adcc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T06:48:55.269 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-11T06:48:55.292 INFO:tasks.ceph.mgr.x.smithi089.stderr:2024-09-11T06:48:55.290+0000 7f1ec64d1180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T06:48:55.373 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:48:55.370+0000 7f33c7aa77c0 -1 Falling back to public interface 2024-09-11T06:48:55.421 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:48:55.418+0000 7f5be7a9e7c0 -1 Falling back to public interface 2024-09-11T06:48:55.454 INFO:tasks.ceph.mgr.x.smithi089.stderr:2024-09-11T06:48:55.450+0000 7f1ec64d1180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T06:48:55.523 INFO:tasks.ceph.mgr.x.smithi089.stderr:2024-09-11T06:48:55.522+0000 7f1ec64d1180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T06:48:55.584 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:48:55.582+0000 7f4b670987c0 -1 Falling back to public interface 2024-09-11T06:48:55.601 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:48:55.598+0000 7ff975bef7c0 -1 Falling back to public interface 2024-09-11T06:48:55.632 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:48:55.630+0000 7f472db237c0 -1 Falling back to public interface 2024-09-11T06:48:55.672 INFO:tasks.ceph.mgr.x.smithi089.stderr:2024-09-11T06:48:55.670+0000 7f1ec64d1180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T06:48:55.857 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:48:55.854+0000 7fd529c627c0 -1 Falling back to public interface 2024-09-11T06:48:56.297 INFO:tasks.ceph.mgr.x.smithi089.stderr:2024-09-11T06:48:56.294+0000 7f1ec64d1180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T06:48:56.371 INFO:tasks.ceph.mgr.x.smithi089.stderr:2024-09-11T06:48:56.370+0000 7f1ec64d1180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T06:48:56.553 INFO:tasks.ceph.mgr.x.smithi089.stderr:2024-09-11T06:48:56.550+0000 7f1ec64d1180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T06:48:56.626 INFO:tasks.ceph.mgr.x.smithi089.stderr:2024-09-11T06:48:56.622+0000 7f1ec64d1180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T06:48:56.915 INFO:tasks.ceph.mgr.x.smithi089.stderr:2024-09-11T06:48:56.914+0000 7f1ec64d1180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T06:48:56.977 INFO:tasks.ceph.mgr.x.smithi089.stderr:2024-09-11T06:48:56.974+0000 7f1ec64d1180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T06:48:58.015 INFO:tasks.ceph.mgr.x.smithi089.stderr:2024-09-11T06:48:58.014+0000 7f1e6d008640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-11T06:48:58.022 INFO:tasks.ceph.mgr.x.smithi089.stderr:2024-09-11T06:48:58.018+0000 7f1e6d008640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-11T06:48:59.273 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:48:59.270+0000 7f33c7aa77c0 -1 osd.0 0 log_to_monitors true 2024-09-11T06:48:59.422 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:48:59.418+0000 7f5be7a9e7c0 -1 osd.1 0 log_to_monitors true 2024-09-11T06:48:59.594 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:48:59.590+0000 7f4b670987c0 -1 osd.5 0 log_to_monitors true 2024-09-11T06:48:59.703 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:48:59.702+0000 7f472db237c0 -1 osd.3 0 log_to_monitors true 2024-09-11T06:48:59.791 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:48:59.790+0000 7fd529c627c0 -1 osd.4 0 log_to_monitors true 2024-09-11T06:48:59.982 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:48:59.978+0000 7ff975bef7c0 -1 osd.2 0 log_to_monitors true 2024-09-11T06:49:01.875 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-11T06:49:02.220 INFO:teuthology.misc.health.smithi071.stdout: 2024-09-11T06:49:02.220 INFO:teuthology.misc.health.smithi071.stdout:{"epoch":10,"fsid":"bd5dcb33-e9f7-4a5f-a26c-a451452e8aa2","created":"2024-09-11T06:48:52.191706+0000","modified":"2024-09-11T06:49:01.061500+0000","last_up_change":"0.000000","last_in_change":"2024-09-11T06:48:54.213376+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3c9e35bf-d3db-49db-b10c-d2cf38c7eb7e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"693649dc-db7c-43cf-bdbe-87f6128ef311","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"89421ae7-fead-409c-83f7-ede424db6004","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f4512e48-63f1-42b6-81be-594bd82e3210","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"52d47b6f-af30-455e-8326-e53c598911ae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6b4f9416-0ba0-4c58-883e-8cbdb362adcc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T06:49:02.244 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-11T06:49:02.963 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:02.958+0000 7f5be3a47640 -1 osd.1 0 waiting for initial osdmap 2024-09-11T06:49:02.975 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:02.970+0000 7f33c3a50640 -1 osd.0 0 waiting for initial osdmap 2024-09-11T06:49:02.982 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:02.978+0000 7ff971b98640 -1 osd.2 0 waiting for initial osdmap 2024-09-11T06:49:02.985 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:02.982+0000 7f5bdb87c640 -1 osd.1 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T06:49:02.990 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:02.990+0000 7f4b63853640 -1 osd.5 0 waiting for initial osdmap 2024-09-11T06:49:03.000 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:02.998+0000 7f33bb885640 -1 osd.0 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T06:49:03.001 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:02.998+0000 7f4729acc640 -1 osd.3 0 waiting for initial osdmap 2024-09-11T06:49:03.015 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:03.014+0000 7f4b5ae76640 -1 osd.5 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T06:49:03.016 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:03.014+0000 7ff9699cd640 -1 osd.2 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T06:49:03.031 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:03.030+0000 7fd525c0b640 -1 osd.4 0 waiting for initial osdmap 2024-09-11T06:49:03.035 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:03.034+0000 7f4721901640 -1 osd.3 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T06:49:03.059 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:03.058+0000 7fd51da40640 -1 osd.4 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T06:49:08.850 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-11T06:49:09.173 INFO:teuthology.misc.health.smithi071.stdout: 2024-09-11T06:49:09.173 INFO:teuthology.misc.health.smithi071.stdout:{"epoch":14,"fsid":"bd5dcb33-e9f7-4a5f-a26c-a451452e8aa2","created":"2024-09-11T06:48:52.191706+0000","modified":"2024-09-11T06:49:07.188021+0000","last_up_change":"2024-09-11T06:49:04.007203+0000","last_in_change":"2024-09-11T06:48:54.213376+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T06:49:05.084515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c9e35bf-d3db-49db-b10c-d2cf38c7eb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1158147777},{"type":"v1","addr":"172.21.15.71:6801","nonce":1158147777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1158147777},{"type":"v1","addr":"172.21.15.71:6803","nonce":1158147777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1158147777},{"type":"v1","addr":"172.21.15.71:6807","nonce":1158147777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1158147777},{"type":"v1","addr":"172.21.15.71:6805","nonce":1158147777}]},"public_addr":"172.21.15.71:6801/1158147777","cluster_addr":"172.21.15.71:6803/1158147777","heartbeat_back_addr":"172.21.15.71:6807/1158147777","heartbeat_front_addr":"172.21.15.71:6805/1158147777","state":["exists","up"]},{"osd":1,"uuid":"693649dc-db7c-43cf-bdbe-87f6128ef311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":620364628},{"type":"v1","addr":"172.21.15.71:6809","nonce":620364628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":620364628},{"type":"v1","addr":"172.21.15.71:6811","nonce":620364628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":620364628},{"type":"v1","addr":"172.21.15.71:6815","nonce":620364628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":620364628},{"type":"v1","addr":"172.21.15.71:6813","nonce":620364628}]},"public_addr":"172.21.15.71:6809/620364628","cluster_addr":"172.21.15.71:6811/620364628","heartbeat_back_addr":"172.21.15.71:6815/620364628","heartbeat_front_addr":"172.21.15.71:6813/620364628","state":["exists","up"]},{"osd":2,"uuid":"89421ae7-fead-409c-83f7-ede424db6004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2926882521},{"type":"v1","addr":"172.21.15.71:6817","nonce":2926882521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2926882521},{"type":"v1","addr":"172.21.15.71:6819","nonce":2926882521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2926882521},{"type":"v1","addr":"172.21.15.71:6823","nonce":2926882521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2926882521},{"type":"v1","addr":"172.21.15.71:6821","nonce":2926882521}]},"public_addr":"172.21.15.71:6817/2926882521","cluster_addr":"172.21.15.71:6819/2926882521","heartbeat_back_addr":"172.21.15.71:6823/2926882521","heartbeat_front_addr":"172.21.15.71:6821/2926882521","state":["exists","up"]},{"osd":3,"uuid":"f4512e48-63f1-42b6-81be-594bd82e3210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":558942315},{"type":"v1","addr":"172.21.15.89:6809","nonce":558942315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":558942315},{"type":"v1","addr":"172.21.15.89:6811","nonce":558942315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":558942315},{"type":"v1","addr":"172.21.15.89:6815","nonce":558942315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":558942315},{"type":"v1","addr":"172.21.15.89:6813","nonce":558942315}]},"public_addr":"172.21.15.89:6809/558942315","cluster_addr":"172.21.15.89:6811/558942315","heartbeat_back_addr":"172.21.15.89:6815/558942315","heartbeat_front_addr":"172.21.15.89:6813/558942315","state":["exists","up"]},{"osd":4,"uuid":"52d47b6f-af30-455e-8326-e53c598911ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3878488003},{"type":"v1","addr":"172.21.15.89:6817","nonce":3878488003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3878488003},{"type":"v1","addr":"172.21.15.89:6819","nonce":3878488003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":3878488003},{"type":"v1","addr":"172.21.15.89:6823","nonce":3878488003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":3878488003},{"type":"v1","addr":"172.21.15.89:6821","nonce":3878488003}]},"public_addr":"172.21.15.89:6817/3878488003","cluster_addr":"172.21.15.89:6819/3878488003","heartbeat_back_addr":"172.21.15.89:6823/3878488003","heartbeat_front_addr":"172.21.15.89:6821/3878488003","state":["exists","up"]},{"osd":5,"uuid":"6b4f9416-0ba0-4c58-883e-8cbdb362adcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2840204107},{"type":"v1","addr":"172.21.15.89:6801","nonce":2840204107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2840204107},{"type":"v1","addr":"172.21.15.89:6803","nonce":2840204107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2840204107},{"type":"v1","addr":"172.21.15.89:6807","nonce":2840204107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2840204107},{"type":"v1","addr":"172.21.15.89:6805","nonce":2840204107}]},"public_addr":"172.21.15.89:6801/2840204107","cluster_addr":"172.21.15.89:6803/2840204107","heartbeat_back_addr":"172.21.15.89:6807/2840204107","heartbeat_front_addr":"172.21.15.89:6805/2840204107","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.282785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.381259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.986977+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.732329+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.844988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.639989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T06:49:09.191 DEBUG:teuthology.misc:6 of 6 OSDs are up 2024-09-11T06:49:09.191 INFO:tasks.ceph:Creating RBD pool 2024-09-11T06:49:09.192 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-11T06:49:10.216 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'rbd' created 2024-09-11T06:49:10.242 DEBUG:teuthology.orchestra.run.smithi071:> rbd --cluster ceph pool init rbd 2024-09-11T06:49:13.261 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-11T06:49:13.261 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-11T06:49:13.261 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-11T06:49:13.262 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-11T06:49:13.262 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-11T06:49:13.613 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:49:13.630 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":5,"flags":0,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":3062838185},{"type":"v1","addr":"172.21.15.89:6825","nonce":3062838185}]},"active_addr":"172.21.15.89:6825/3062838185","active_change":"2024-09-11T06:48:56.999921+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":3986164104}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":2875175839}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":1516973092}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":2948952555}]}]} 2024-09-11T06:49:13.632 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-11T06:49:13.632 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-11T06:49:13.632 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T06:49:13.926 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:49:13.926 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":18,"fsid":"bd5dcb33-e9f7-4a5f-a26c-a451452e8aa2","created":"2024-09-11T06:48:52.191706+0000","modified":"2024-09-11T06:49:13.232875+0000","last_up_change":"2024-09-11T06:49:04.007203+0000","last_in_change":"2024-09-11T06:48:54.213376+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T06:49:05.084515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-11T06:49:09.530167+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c9e35bf-d3db-49db-b10c-d2cf38c7eb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1158147777},{"type":"v1","addr":"172.21.15.71:6801","nonce":1158147777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1158147777},{"type":"v1","addr":"172.21.15.71:6803","nonce":1158147777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1158147777},{"type":"v1","addr":"172.21.15.71:6807","nonce":1158147777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1158147777},{"type":"v1","addr":"172.21.15.71:6805","nonce":1158147777}]},"public_addr":"172.21.15.71:6801/1158147777","cluster_addr":"172.21.15.71:6803/1158147777","heartbeat_back_addr":"172.21.15.71:6807/1158147777","heartbeat_front_addr":"172.21.15.71:6805/1158147777","state":["exists","up"]},{"osd":1,"uuid":"693649dc-db7c-43cf-bdbe-87f6128ef311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":620364628},{"type":"v1","addr":"172.21.15.71:6809","nonce":620364628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":620364628},{"type":"v1","addr":"172.21.15.71:6811","nonce":620364628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":620364628},{"type":"v1","addr":"172.21.15.71:6815","nonce":620364628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":620364628},{"type":"v1","addr":"172.21.15.71:6813","nonce":620364628}]},"public_addr":"172.21.15.71:6809/620364628","cluster_addr":"172.21.15.71:6811/620364628","heartbeat_back_addr":"172.21.15.71:6815/620364628","heartbeat_front_addr":"172.21.15.71:6813/620364628","state":["exists","up"]},{"osd":2,"uuid":"89421ae7-fead-409c-83f7-ede424db6004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2926882521},{"type":"v1","addr":"172.21.15.71:6817","nonce":2926882521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2926882521},{"type":"v1","addr":"172.21.15.71:6819","nonce":2926882521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2926882521},{"type":"v1","addr":"172.21.15.71:6823","nonce":2926882521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2926882521},{"type":"v1","addr":"172.21.15.71:6821","nonce":2926882521}]},"public_addr":"172.21.15.71:6817/2926882521","cluster_addr":"172.21.15.71:6819/2926882521","heartbeat_back_addr":"172.21.15.71:6823/2926882521","heartbeat_front_addr":"172.21.15.71:6821/2926882521","state":["exists","up"]},{"osd":3,"uuid":"f4512e48-63f1-42b6-81be-594bd82e3210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":558942315},{"type":"v1","addr":"172.21.15.89:6809","nonce":558942315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":558942315},{"type":"v1","addr":"172.21.15.89:6811","nonce":558942315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":558942315},{"type":"v1","addr":"172.21.15.89:6815","nonce":558942315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":558942315},{"type":"v1","addr":"172.21.15.89:6813","nonce":558942315}]},"public_addr":"172.21.15.89:6809/558942315","cluster_addr":"172.21.15.89:6811/558942315","heartbeat_back_addr":"172.21.15.89:6815/558942315","heartbeat_front_addr":"172.21.15.89:6813/558942315","state":["exists","up"]},{"osd":4,"uuid":"52d47b6f-af30-455e-8326-e53c598911ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3878488003},{"type":"v1","addr":"172.21.15.89:6817","nonce":3878488003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3878488003},{"type":"v1","addr":"172.21.15.89:6819","nonce":3878488003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":3878488003},{"type":"v1","addr":"172.21.15.89:6823","nonce":3878488003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":3878488003},{"type":"v1","addr":"172.21.15.89:6821","nonce":3878488003}]},"public_addr":"172.21.15.89:6817/3878488003","cluster_addr":"172.21.15.89:6819/3878488003","heartbeat_back_addr":"172.21.15.89:6823/3878488003","heartbeat_front_addr":"172.21.15.89:6821/3878488003","state":["exists","up"]},{"osd":5,"uuid":"6b4f9416-0ba0-4c58-883e-8cbdb362adcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2840204107},{"type":"v1","addr":"172.21.15.89:6801","nonce":2840204107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2840204107},{"type":"v1","addr":"172.21.15.89:6803","nonce":2840204107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2840204107},{"type":"v1","addr":"172.21.15.89:6807","nonce":2840204107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2840204107},{"type":"v1","addr":"172.21.15.89:6805","nonce":2840204107}]},"public_addr":"172.21.15.89:6801/2840204107","cluster_addr":"172.21.15.89:6803/2840204107","heartbeat_back_addr":"172.21.15.89:6807/2840204107","heartbeat_front_addr":"172.21.15.89:6805/2840204107","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.282785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.381259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.986977+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.732329+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.844988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.639989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T06:49:13.945 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-11T06:49:13.945 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T06:49:14.246 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:49:14.246 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":18,"fsid":"bd5dcb33-e9f7-4a5f-a26c-a451452e8aa2","created":"2024-09-11T06:48:52.191706+0000","modified":"2024-09-11T06:49:13.232875+0000","last_up_change":"2024-09-11T06:49:04.007203+0000","last_in_change":"2024-09-11T06:48:54.213376+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T06:49:05.084515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-11T06:49:09.530167+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c9e35bf-d3db-49db-b10c-d2cf38c7eb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1158147777},{"type":"v1","addr":"172.21.15.71:6801","nonce":1158147777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1158147777},{"type":"v1","addr":"172.21.15.71:6803","nonce":1158147777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1158147777},{"type":"v1","addr":"172.21.15.71:6807","nonce":1158147777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1158147777},{"type":"v1","addr":"172.21.15.71:6805","nonce":1158147777}]},"public_addr":"172.21.15.71:6801/1158147777","cluster_addr":"172.21.15.71:6803/1158147777","heartbeat_back_addr":"172.21.15.71:6807/1158147777","heartbeat_front_addr":"172.21.15.71:6805/1158147777","state":["exists","up"]},{"osd":1,"uuid":"693649dc-db7c-43cf-bdbe-87f6128ef311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":620364628},{"type":"v1","addr":"172.21.15.71:6809","nonce":620364628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":620364628},{"type":"v1","addr":"172.21.15.71:6811","nonce":620364628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":620364628},{"type":"v1","addr":"172.21.15.71:6815","nonce":620364628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":620364628},{"type":"v1","addr":"172.21.15.71:6813","nonce":620364628}]},"public_addr":"172.21.15.71:6809/620364628","cluster_addr":"172.21.15.71:6811/620364628","heartbeat_back_addr":"172.21.15.71:6815/620364628","heartbeat_front_addr":"172.21.15.71:6813/620364628","state":["exists","up"]},{"osd":2,"uuid":"89421ae7-fead-409c-83f7-ede424db6004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2926882521},{"type":"v1","addr":"172.21.15.71:6817","nonce":2926882521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2926882521},{"type":"v1","addr":"172.21.15.71:6819","nonce":2926882521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2926882521},{"type":"v1","addr":"172.21.15.71:6823","nonce":2926882521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2926882521},{"type":"v1","addr":"172.21.15.71:6821","nonce":2926882521}]},"public_addr":"172.21.15.71:6817/2926882521","cluster_addr":"172.21.15.71:6819/2926882521","heartbeat_back_addr":"172.21.15.71:6823/2926882521","heartbeat_front_addr":"172.21.15.71:6821/2926882521","state":["exists","up"]},{"osd":3,"uuid":"f4512e48-63f1-42b6-81be-594bd82e3210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":558942315},{"type":"v1","addr":"172.21.15.89:6809","nonce":558942315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":558942315},{"type":"v1","addr":"172.21.15.89:6811","nonce":558942315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":558942315},{"type":"v1","addr":"172.21.15.89:6815","nonce":558942315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":558942315},{"type":"v1","addr":"172.21.15.89:6813","nonce":558942315}]},"public_addr":"172.21.15.89:6809/558942315","cluster_addr":"172.21.15.89:6811/558942315","heartbeat_back_addr":"172.21.15.89:6815/558942315","heartbeat_front_addr":"172.21.15.89:6813/558942315","state":["exists","up"]},{"osd":4,"uuid":"52d47b6f-af30-455e-8326-e53c598911ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3878488003},{"type":"v1","addr":"172.21.15.89:6817","nonce":3878488003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3878488003},{"type":"v1","addr":"172.21.15.89:6819","nonce":3878488003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":3878488003},{"type":"v1","addr":"172.21.15.89:6823","nonce":3878488003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":3878488003},{"type":"v1","addr":"172.21.15.89:6821","nonce":3878488003}]},"public_addr":"172.21.15.89:6817/3878488003","cluster_addr":"172.21.15.89:6819/3878488003","heartbeat_back_addr":"172.21.15.89:6823/3878488003","heartbeat_front_addr":"172.21.15.89:6821/3878488003","state":["exists","up"]},{"osd":5,"uuid":"6b4f9416-0ba0-4c58-883e-8cbdb362adcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2840204107},{"type":"v1","addr":"172.21.15.89:6801","nonce":2840204107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2840204107},{"type":"v1","addr":"172.21.15.89:6803","nonce":2840204107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2840204107},{"type":"v1","addr":"172.21.15.89:6807","nonce":2840204107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2840204107},{"type":"v1","addr":"172.21.15.89:6805","nonce":2840204107}]},"public_addr":"172.21.15.89:6801/2840204107","cluster_addr":"172.21.15.89:6803/2840204107","heartbeat_back_addr":"172.21.15.89:6807/2840204107","heartbeat_front_addr":"172.21.15.89:6805/2840204107","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.282785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.381259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.986977+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.732329+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.844988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.639989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T06:49:14.265 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-11T06:49:14.265 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-11T06:49:14.266 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-11T06:49:14.266 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-11T06:49:14.266 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-11T06:49:14.267 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-11T06:49:14.425 INFO:teuthology.orchestra.run.smithi071.stdout:47244640262 2024-09-11T06:49:14.426 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T06:49:14.441 INFO:teuthology.orchestra.run.smithi071.stdout:47244640263 2024-09-11T06:49:14.442 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T06:49:14.485 INFO:teuthology.orchestra.run.smithi071.stdout:47244640262 2024-09-11T06:49:14.485 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T06:49:14.509 INFO:teuthology.orchestra.run.smithi071.stdout:47244640261 2024-09-11T06:49:14.510 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-11T06:49:14.510 INFO:teuthology.orchestra.run.smithi071.stdout:47244640261 2024-09-11T06:49:14.511 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-11T06:49:14.512 INFO:teuthology.orchestra.run.smithi071.stdout:47244640260 2024-09-11T06:49:14.512 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-11T06:49:14.840 INFO:teuthology.orchestra.run.smithi071.stdout:47244640260 2024-09-11T06:49:14.857 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640260 for osd.2 2024-09-11T06:49:14.865 INFO:teuthology.orchestra.run.smithi071.stdout:47244640261 2024-09-11T06:49:14.888 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640263 got 47244640261 for osd.0 2024-09-11T06:49:14.898 INFO:teuthology.orchestra.run.smithi071.stdout:47244640260 2024-09-11T06:49:14.915 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640260 for osd.1 2024-09-11T06:49:14.935 INFO:teuthology.orchestra.run.smithi071.stdout:47244640259 2024-09-11T06:49:14.952 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640259 for osd.3 2024-09-11T06:49:14.979 INFO:teuthology.orchestra.run.smithi071.stdout:47244640259 2024-09-11T06:49:15.001 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640259 for osd.4 2024-09-11T06:49:15.022 INFO:teuthology.orchestra.run.smithi071.stdout:47244640258 2024-09-11T06:49:15.038 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640258 for osd.5 2024-09-11T06:49:15.858 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T06:49:15.890 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T06:49:15.916 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T06:49:15.953 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-11T06:49:16.002 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-11T06:49:16.039 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-11T06:49:16.288 INFO:teuthology.orchestra.run.smithi071.stdout:47244640262 2024-09-11T06:49:16.296 INFO:teuthology.orchestra.run.smithi071.stdout:47244640262 2024-09-11T06:49:16.315 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640262 for osd.1 2024-09-11T06:49:16.316 DEBUG:teuthology.parallel:result is None 2024-09-11T06:49:16.316 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640262 for osd.2 2024-09-11T06:49:16.317 DEBUG:teuthology.parallel:result is None 2024-09-11T06:49:16.337 INFO:teuthology.orchestra.run.smithi071.stdout:47244640263 2024-09-11T06:49:16.354 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640263 got 47244640263 for osd.0 2024-09-11T06:49:16.354 DEBUG:teuthology.parallel:result is None 2024-09-11T06:49:16.368 INFO:teuthology.orchestra.run.smithi071.stdout:47244640261 2024-09-11T06:49:16.384 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.4 2024-09-11T06:49:16.385 DEBUG:teuthology.parallel:result is None 2024-09-11T06:49:16.409 INFO:teuthology.orchestra.run.smithi071.stdout:47244640261 2024-09-11T06:49:16.425 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.3 2024-09-11T06:49:16.426 DEBUG:teuthology.parallel:result is None 2024-09-11T06:49:16.462 INFO:teuthology.orchestra.run.smithi071.stdout:47244640260 2024-09-11T06:49:16.478 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640260 for osd.5 2024-09-11T06:49:16.478 DEBUG:teuthology.parallel:result is None 2024-09-11T06:49:16.478 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T06:49:16.479 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:49:16.791 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:49:16.791 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:49:16.807 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-11T06:49:15.012977+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":164392,"kb_used_data":2176,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566066648,"statfs":{"total":579820584960,"available":579652247552,"internally_reserved":0,"allocated":2228224,"data_stored":1521508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9565,"internal_metadata":165927587},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.800105"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.393788+0000","last_change":"2024-09-11T06:49:13.393788+0000","last_active":"2024-09-11T06:49:13.393788+0000","last_peered":"2024-09-11T06:49:13.393788+0000","last_clean":"2024-09-11T06:49:13.393788+0000","last_became_active":"2024-09-11T06:49:11.433437+0000","last_became_peered":"2024-09-11T06:49:11.433437+0000","last_unstale":"2024-09-11T06:49:13.393788+0000","last_undegraded":"2024-09-11T06:49:13.393788+0000","last_fullsized":"2024-09-11T06:49:13.393788+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:20:04.339208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001403348,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.245269+0000","last_change":"2024-09-11T06:49:13.245269+0000","last_active":"2024-09-11T06:49:13.245269+0000","last_peered":"2024-09-11T06:49:13.245269+0000","last_clean":"2024-09-11T06:49:13.245269+0000","last_became_active":"2024-09-11T06:49:11.232461+0000","last_became_peered":"2024-09-11T06:49:11.232461+0000","last_unstale":"2024-09-11T06:49:13.245269+0000","last_undegraded":"2024-09-11T06:49:13.245269+0000","last_fullsized":"2024-09-11T06:49:13.245269+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:27.571820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001050942,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.393298+0000","last_change":"2024-09-11T06:49:13.393298+0000","last_active":"2024-09-11T06:49:13.393298+0000","last_peered":"2024-09-11T06:49:13.393298+0000","last_clean":"2024-09-11T06:49:13.393298+0000","last_became_active":"2024-09-11T06:49:11.315089+0000","last_became_peered":"2024-09-11T06:49:11.315089+0000","last_unstale":"2024-09-11T06:49:13.393298+0000","last_undegraded":"2024-09-11T06:49:13.393298+0000","last_fullsized":"2024-09-11T06:49:13.393298+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:31.769443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095172900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.246270+0000","last_change":"2024-09-11T06:49:13.246270+0000","last_active":"2024-09-11T06:49:13.246270+0000","last_peered":"2024-09-11T06:49:13.246270+0000","last_clean":"2024-09-11T06:49:13.246270+0000","last_became_active":"2024-09-11T06:49:11.314240+0000","last_became_peered":"2024-09-11T06:49:11.314240+0000","last_unstale":"2024-09-11T06:49:13.246270+0000","last_undegraded":"2024-09-11T06:49:13.246270+0000","last_fullsized":"2024-09-11T06:49:13.246270+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:19:15.572073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012625239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":21,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.253391+0000","last_change":"2024-09-11T06:49:13.251855+0000","last_active":"2024-09-11T06:49:13.253391+0000","last_peered":"2024-09-11T06:49:13.253391+0000","last_clean":"2024-09-11T06:49:13.253391+0000","last_became_active":"2024-09-11T06:49:11.235757+0000","last_became_peered":"2024-09-11T06:49:11.235757+0000","last_unstale":"2024-09-11T06:49:13.253391+0000","last_undegraded":"2024-09-11T06:49:13.253391+0000","last_fullsized":"2024-09-11T06:49:13.253391+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:43.514517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001910244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.244282+0000","last_change":"2024-09-11T06:49:13.244282+0000","last_active":"2024-09-11T06:49:13.244282+0000","last_peered":"2024-09-11T06:49:13.244282+0000","last_clean":"2024-09-11T06:49:13.244282+0000","last_became_active":"2024-09-11T06:49:11.227993+0000","last_became_peered":"2024-09-11T06:49:11.227993+0000","last_unstale":"2024-09-11T06:49:13.244282+0000","last_undegraded":"2024-09-11T06:49:13.244282+0000","last_fullsized":"2024-09-11T06:49:13.244282+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:59:22.954416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010163780000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.393060+0000","last_change":"2024-09-11T06:49:13.393060+0000","last_active":"2024-09-11T06:49:13.393060+0000","last_peered":"2024-09-11T06:49:13.393060+0000","last_clean":"2024-09-11T06:49:13.393060+0000","last_became_active":"2024-09-11T06:49:11.229017+0000","last_became_peered":"2024-09-11T06:49:11.229017+0000","last_unstale":"2024-09-11T06:49:13.393060+0000","last_undegraded":"2024-09-11T06:49:13.393060+0000","last_fullsized":"2024-09-11T06:49:13.393060+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:28.343297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077615599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.251115+0000","last_change":"2024-09-11T06:49:13.251115+0000","last_active":"2024-09-11T06:49:13.251115+0000","last_peered":"2024-09-11T06:49:13.251115+0000","last_clean":"2024-09-11T06:49:13.251115+0000","last_became_active":"2024-09-11T06:49:11.235231+0000","last_became_peered":"2024-09-11T06:49:11.235231+0000","last_unstale":"2024-09-11T06:49:13.251115+0000","last_undegraded":"2024-09-11T06:49:13.251115+0000","last_fullsized":"2024-09-11T06:49:13.251115+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:01:21.028466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011212469999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'117","reported_seq":170,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.392147+0000","last_change":"2024-09-11T06:49:06.112585+0000","last_active":"2024-09-11T06:49:13.392147+0000","last_peered":"2024-09-11T06:49:13.392147+0000","last_clean":"2024-09-11T06:49:13.392147+0000","last_became_active":"2024-09-11T06:49:06.110690+0000","last_became_peered":"2024-09-11T06:49:06.110690+0000","last_unstale":"2024-09-11T06:49:13.392147+0000","last_undegraded":"2024-09-11T06:49:13.392147+0000","last_fullsized":"2024-09-11T06:49:13.392147+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_clean_scrub_stamp":"2024-09-11T06:49:05.088839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:51.167301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":56808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640261,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":172032,"data_stored":56789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":765952,"data_stored":647157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640262,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":172032,"data_stored":56789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640262,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":56808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640263,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":765952,"data_stored":647157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:49:16.808 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:49:17.081 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:49:17.081 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:49:17.097 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-11T06:49:17.013529+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":164392,"kb_used_data":2176,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566066648,"statfs":{"total":579820584960,"available":579652247552,"internally_reserved":0,"allocated":2228224,"data_stored":1521508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9565,"internal_metadata":165927587},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.792361"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.393788+0000","last_change":"2024-09-11T06:49:13.393788+0000","last_active":"2024-09-11T06:49:13.393788+0000","last_peered":"2024-09-11T06:49:13.393788+0000","last_clean":"2024-09-11T06:49:13.393788+0000","last_became_active":"2024-09-11T06:49:11.433437+0000","last_became_peered":"2024-09-11T06:49:11.433437+0000","last_unstale":"2024-09-11T06:49:13.393788+0000","last_undegraded":"2024-09-11T06:49:13.393788+0000","last_fullsized":"2024-09-11T06:49:13.393788+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:20:04.339208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001403348,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.245269+0000","last_change":"2024-09-11T06:49:13.245269+0000","last_active":"2024-09-11T06:49:13.245269+0000","last_peered":"2024-09-11T06:49:13.245269+0000","last_clean":"2024-09-11T06:49:13.245269+0000","last_became_active":"2024-09-11T06:49:11.232461+0000","last_became_peered":"2024-09-11T06:49:11.232461+0000","last_unstale":"2024-09-11T06:49:13.245269+0000","last_undegraded":"2024-09-11T06:49:13.245269+0000","last_fullsized":"2024-09-11T06:49:13.245269+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:27.571820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001050942,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.393298+0000","last_change":"2024-09-11T06:49:13.393298+0000","last_active":"2024-09-11T06:49:13.393298+0000","last_peered":"2024-09-11T06:49:13.393298+0000","last_clean":"2024-09-11T06:49:13.393298+0000","last_became_active":"2024-09-11T06:49:11.315089+0000","last_became_peered":"2024-09-11T06:49:11.315089+0000","last_unstale":"2024-09-11T06:49:13.393298+0000","last_undegraded":"2024-09-11T06:49:13.393298+0000","last_fullsized":"2024-09-11T06:49:13.393298+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:31.769443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095172900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.246270+0000","last_change":"2024-09-11T06:49:13.246270+0000","last_active":"2024-09-11T06:49:13.246270+0000","last_peered":"2024-09-11T06:49:13.246270+0000","last_clean":"2024-09-11T06:49:13.246270+0000","last_became_active":"2024-09-11T06:49:11.314240+0000","last_became_peered":"2024-09-11T06:49:11.314240+0000","last_unstale":"2024-09-11T06:49:13.246270+0000","last_undegraded":"2024-09-11T06:49:13.246270+0000","last_fullsized":"2024-09-11T06:49:13.246270+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:19:15.572073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012625239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":21,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.253391+0000","last_change":"2024-09-11T06:49:13.251855+0000","last_active":"2024-09-11T06:49:13.253391+0000","last_peered":"2024-09-11T06:49:13.253391+0000","last_clean":"2024-09-11T06:49:13.253391+0000","last_became_active":"2024-09-11T06:49:11.235757+0000","last_became_peered":"2024-09-11T06:49:11.235757+0000","last_unstale":"2024-09-11T06:49:13.253391+0000","last_undegraded":"2024-09-11T06:49:13.253391+0000","last_fullsized":"2024-09-11T06:49:13.253391+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:43.514517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001910244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.244282+0000","last_change":"2024-09-11T06:49:13.244282+0000","last_active":"2024-09-11T06:49:13.244282+0000","last_peered":"2024-09-11T06:49:13.244282+0000","last_clean":"2024-09-11T06:49:13.244282+0000","last_became_active":"2024-09-11T06:49:11.227993+0000","last_became_peered":"2024-09-11T06:49:11.227993+0000","last_unstale":"2024-09-11T06:49:13.244282+0000","last_undegraded":"2024-09-11T06:49:13.244282+0000","last_fullsized":"2024-09-11T06:49:13.244282+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:59:22.954416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010163780000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.393060+0000","last_change":"2024-09-11T06:49:13.393060+0000","last_active":"2024-09-11T06:49:13.393060+0000","last_peered":"2024-09-11T06:49:13.393060+0000","last_clean":"2024-09-11T06:49:13.393060+0000","last_became_active":"2024-09-11T06:49:11.229017+0000","last_became_peered":"2024-09-11T06:49:11.229017+0000","last_unstale":"2024-09-11T06:49:13.393060+0000","last_undegraded":"2024-09-11T06:49:13.393060+0000","last_fullsized":"2024-09-11T06:49:13.393060+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:28.343297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077615599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.251115+0000","last_change":"2024-09-11T06:49:13.251115+0000","last_active":"2024-09-11T06:49:13.251115+0000","last_peered":"2024-09-11T06:49:13.251115+0000","last_clean":"2024-09-11T06:49:13.251115+0000","last_became_active":"2024-09-11T06:49:11.235231+0000","last_became_peered":"2024-09-11T06:49:11.235231+0000","last_unstale":"2024-09-11T06:49:13.251115+0000","last_undegraded":"2024-09-11T06:49:13.251115+0000","last_fullsized":"2024-09-11T06:49:13.251115+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:01:21.028466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011212469999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'117","reported_seq":170,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.392147+0000","last_change":"2024-09-11T06:49:06.112585+0000","last_active":"2024-09-11T06:49:13.392147+0000","last_peered":"2024-09-11T06:49:13.392147+0000","last_clean":"2024-09-11T06:49:13.392147+0000","last_became_active":"2024-09-11T06:49:06.110690+0000","last_became_peered":"2024-09-11T06:49:06.110690+0000","last_unstale":"2024-09-11T06:49:13.392147+0000","last_undegraded":"2024-09-11T06:49:13.392147+0000","last_fullsized":"2024-09-11T06:49:13.392147+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_clean_scrub_stamp":"2024-09-11T06:49:05.088839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:51.167301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":56808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640261,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":172032,"data_stored":56789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":765952,"data_stored":647157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640262,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":172032,"data_stored":56789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640262,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":56808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640263,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":765952,"data_stored":647157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:49:17.098 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T06:49:17.098 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-11T06:49:17.098 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-11T06:49:17.099 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-11T06:49:17.406 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:49:17.406 INFO:teuthology.orchestra.run.smithi071.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-11T06:49:17.423 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-11T06:49:17.423 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-11T06:49:17.433 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-11T06:49:17.433 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'daaf78de334647fc0c800f4ac495cc9475a26bae'}} 2024-09-11T06:49:17.433 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-11T06:49:17.434 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-11T06:49:17.730 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-11T06:49:17.731 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T06:49:17.731 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:49:18.008 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:49:18.009 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:49:18.025 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-11T06:49:17.013529+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":164392,"kb_used_data":2176,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566066648,"statfs":{"total":579820584960,"available":579652247552,"internally_reserved":0,"allocated":2228224,"data_stored":1521508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9565,"internal_metadata":165927587},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.792361"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.393788+0000","last_change":"2024-09-11T06:49:13.393788+0000","last_active":"2024-09-11T06:49:13.393788+0000","last_peered":"2024-09-11T06:49:13.393788+0000","last_clean":"2024-09-11T06:49:13.393788+0000","last_became_active":"2024-09-11T06:49:11.433437+0000","last_became_peered":"2024-09-11T06:49:11.433437+0000","last_unstale":"2024-09-11T06:49:13.393788+0000","last_undegraded":"2024-09-11T06:49:13.393788+0000","last_fullsized":"2024-09-11T06:49:13.393788+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:20:04.339208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001403348,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.245269+0000","last_change":"2024-09-11T06:49:13.245269+0000","last_active":"2024-09-11T06:49:13.245269+0000","last_peered":"2024-09-11T06:49:13.245269+0000","last_clean":"2024-09-11T06:49:13.245269+0000","last_became_active":"2024-09-11T06:49:11.232461+0000","last_became_peered":"2024-09-11T06:49:11.232461+0000","last_unstale":"2024-09-11T06:49:13.245269+0000","last_undegraded":"2024-09-11T06:49:13.245269+0000","last_fullsized":"2024-09-11T06:49:13.245269+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:27.571820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001050942,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.393298+0000","last_change":"2024-09-11T06:49:13.393298+0000","last_active":"2024-09-11T06:49:13.393298+0000","last_peered":"2024-09-11T06:49:13.393298+0000","last_clean":"2024-09-11T06:49:13.393298+0000","last_became_active":"2024-09-11T06:49:11.315089+0000","last_became_peered":"2024-09-11T06:49:11.315089+0000","last_unstale":"2024-09-11T06:49:13.393298+0000","last_undegraded":"2024-09-11T06:49:13.393298+0000","last_fullsized":"2024-09-11T06:49:13.393298+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:31.769443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095172900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.246270+0000","last_change":"2024-09-11T06:49:13.246270+0000","last_active":"2024-09-11T06:49:13.246270+0000","last_peered":"2024-09-11T06:49:13.246270+0000","last_clean":"2024-09-11T06:49:13.246270+0000","last_became_active":"2024-09-11T06:49:11.314240+0000","last_became_peered":"2024-09-11T06:49:11.314240+0000","last_unstale":"2024-09-11T06:49:13.246270+0000","last_undegraded":"2024-09-11T06:49:13.246270+0000","last_fullsized":"2024-09-11T06:49:13.246270+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:19:15.572073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012625239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":21,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.253391+0000","last_change":"2024-09-11T06:49:13.251855+0000","last_active":"2024-09-11T06:49:13.253391+0000","last_peered":"2024-09-11T06:49:13.253391+0000","last_clean":"2024-09-11T06:49:13.253391+0000","last_became_active":"2024-09-11T06:49:11.235757+0000","last_became_peered":"2024-09-11T06:49:11.235757+0000","last_unstale":"2024-09-11T06:49:13.253391+0000","last_undegraded":"2024-09-11T06:49:13.253391+0000","last_fullsized":"2024-09-11T06:49:13.253391+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:43.514517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001910244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.244282+0000","last_change":"2024-09-11T06:49:13.244282+0000","last_active":"2024-09-11T06:49:13.244282+0000","last_peered":"2024-09-11T06:49:13.244282+0000","last_clean":"2024-09-11T06:49:13.244282+0000","last_became_active":"2024-09-11T06:49:11.227993+0000","last_became_peered":"2024-09-11T06:49:11.227993+0000","last_unstale":"2024-09-11T06:49:13.244282+0000","last_undegraded":"2024-09-11T06:49:13.244282+0000","last_fullsized":"2024-09-11T06:49:13.244282+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:59:22.954416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010163780000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.393060+0000","last_change":"2024-09-11T06:49:13.393060+0000","last_active":"2024-09-11T06:49:13.393060+0000","last_peered":"2024-09-11T06:49:13.393060+0000","last_clean":"2024-09-11T06:49:13.393060+0000","last_became_active":"2024-09-11T06:49:11.229017+0000","last_became_peered":"2024-09-11T06:49:11.229017+0000","last_unstale":"2024-09-11T06:49:13.393060+0000","last_undegraded":"2024-09-11T06:49:13.393060+0000","last_fullsized":"2024-09-11T06:49:13.393060+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:28.343297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077615599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.251115+0000","last_change":"2024-09-11T06:49:13.251115+0000","last_active":"2024-09-11T06:49:13.251115+0000","last_peered":"2024-09-11T06:49:13.251115+0000","last_clean":"2024-09-11T06:49:13.251115+0000","last_became_active":"2024-09-11T06:49:11.235231+0000","last_became_peered":"2024-09-11T06:49:11.235231+0000","last_unstale":"2024-09-11T06:49:13.251115+0000","last_undegraded":"2024-09-11T06:49:13.251115+0000","last_fullsized":"2024-09-11T06:49:13.251115+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:01:21.028466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011212469999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'117","reported_seq":170,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.392147+0000","last_change":"2024-09-11T06:49:06.112585+0000","last_active":"2024-09-11T06:49:13.392147+0000","last_peered":"2024-09-11T06:49:13.392147+0000","last_clean":"2024-09-11T06:49:13.392147+0000","last_became_active":"2024-09-11T06:49:06.110690+0000","last_became_peered":"2024-09-11T06:49:06.110690+0000","last_unstale":"2024-09-11T06:49:13.392147+0000","last_undegraded":"2024-09-11T06:49:13.392147+0000","last_fullsized":"2024-09-11T06:49:13.392147+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_clean_scrub_stamp":"2024-09-11T06:49:05.088839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:51.167301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":56808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640261,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":172032,"data_stored":56789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":765952,"data_stored":647157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640262,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":172032,"data_stored":56789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640262,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":56808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640263,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":765952,"data_stored":647157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:49:18.026 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:49:18.304 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:49:18.305 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:49:18.321 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-11T06:49:17.013529+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":164392,"kb_used_data":2176,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566066648,"statfs":{"total":579820584960,"available":579652247552,"internally_reserved":0,"allocated":2228224,"data_stored":1521508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9565,"internal_metadata":165927587},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.792361"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.393788+0000","last_change":"2024-09-11T06:49:13.393788+0000","last_active":"2024-09-11T06:49:13.393788+0000","last_peered":"2024-09-11T06:49:13.393788+0000","last_clean":"2024-09-11T06:49:13.393788+0000","last_became_active":"2024-09-11T06:49:11.433437+0000","last_became_peered":"2024-09-11T06:49:11.433437+0000","last_unstale":"2024-09-11T06:49:13.393788+0000","last_undegraded":"2024-09-11T06:49:13.393788+0000","last_fullsized":"2024-09-11T06:49:13.393788+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:20:04.339208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001403348,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.245269+0000","last_change":"2024-09-11T06:49:13.245269+0000","last_active":"2024-09-11T06:49:13.245269+0000","last_peered":"2024-09-11T06:49:13.245269+0000","last_clean":"2024-09-11T06:49:13.245269+0000","last_became_active":"2024-09-11T06:49:11.232461+0000","last_became_peered":"2024-09-11T06:49:11.232461+0000","last_unstale":"2024-09-11T06:49:13.245269+0000","last_undegraded":"2024-09-11T06:49:13.245269+0000","last_fullsized":"2024-09-11T06:49:13.245269+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:27.571820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001050942,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.393298+0000","last_change":"2024-09-11T06:49:13.393298+0000","last_active":"2024-09-11T06:49:13.393298+0000","last_peered":"2024-09-11T06:49:13.393298+0000","last_clean":"2024-09-11T06:49:13.393298+0000","last_became_active":"2024-09-11T06:49:11.315089+0000","last_became_peered":"2024-09-11T06:49:11.315089+0000","last_unstale":"2024-09-11T06:49:13.393298+0000","last_undegraded":"2024-09-11T06:49:13.393298+0000","last_fullsized":"2024-09-11T06:49:13.393298+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:31.769443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095172900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.246270+0000","last_change":"2024-09-11T06:49:13.246270+0000","last_active":"2024-09-11T06:49:13.246270+0000","last_peered":"2024-09-11T06:49:13.246270+0000","last_clean":"2024-09-11T06:49:13.246270+0000","last_became_active":"2024-09-11T06:49:11.314240+0000","last_became_peered":"2024-09-11T06:49:11.314240+0000","last_unstale":"2024-09-11T06:49:13.246270+0000","last_undegraded":"2024-09-11T06:49:13.246270+0000","last_fullsized":"2024-09-11T06:49:13.246270+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:19:15.572073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012625239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":21,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.253391+0000","last_change":"2024-09-11T06:49:13.251855+0000","last_active":"2024-09-11T06:49:13.253391+0000","last_peered":"2024-09-11T06:49:13.253391+0000","last_clean":"2024-09-11T06:49:13.253391+0000","last_became_active":"2024-09-11T06:49:11.235757+0000","last_became_peered":"2024-09-11T06:49:11.235757+0000","last_unstale":"2024-09-11T06:49:13.253391+0000","last_undegraded":"2024-09-11T06:49:13.253391+0000","last_fullsized":"2024-09-11T06:49:13.253391+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:43.514517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001910244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.244282+0000","last_change":"2024-09-11T06:49:13.244282+0000","last_active":"2024-09-11T06:49:13.244282+0000","last_peered":"2024-09-11T06:49:13.244282+0000","last_clean":"2024-09-11T06:49:13.244282+0000","last_became_active":"2024-09-11T06:49:11.227993+0000","last_became_peered":"2024-09-11T06:49:11.227993+0000","last_unstale":"2024-09-11T06:49:13.244282+0000","last_undegraded":"2024-09-11T06:49:13.244282+0000","last_fullsized":"2024-09-11T06:49:13.244282+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:59:22.954416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010163780000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.393060+0000","last_change":"2024-09-11T06:49:13.393060+0000","last_active":"2024-09-11T06:49:13.393060+0000","last_peered":"2024-09-11T06:49:13.393060+0000","last_clean":"2024-09-11T06:49:13.393060+0000","last_became_active":"2024-09-11T06:49:11.229017+0000","last_became_peered":"2024-09-11T06:49:11.229017+0000","last_unstale":"2024-09-11T06:49:13.393060+0000","last_undegraded":"2024-09-11T06:49:13.393060+0000","last_fullsized":"2024-09-11T06:49:13.393060+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:28.343297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077615599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.251115+0000","last_change":"2024-09-11T06:49:13.251115+0000","last_active":"2024-09-11T06:49:13.251115+0000","last_peered":"2024-09-11T06:49:13.251115+0000","last_clean":"2024-09-11T06:49:13.251115+0000","last_became_active":"2024-09-11T06:49:11.235231+0000","last_became_peered":"2024-09-11T06:49:11.235231+0000","last_unstale":"2024-09-11T06:49:13.251115+0000","last_undegraded":"2024-09-11T06:49:13.251115+0000","last_fullsized":"2024-09-11T06:49:13.251115+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:01:21.028466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011212469999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'117","reported_seq":170,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-11T06:49:13.392147+0000","last_change":"2024-09-11T06:49:06.112585+0000","last_active":"2024-09-11T06:49:13.392147+0000","last_peered":"2024-09-11T06:49:13.392147+0000","last_clean":"2024-09-11T06:49:13.392147+0000","last_became_active":"2024-09-11T06:49:06.110690+0000","last_became_peered":"2024-09-11T06:49:06.110690+0000","last_unstale":"2024-09-11T06:49:13.392147+0000","last_undegraded":"2024-09-11T06:49:13.392147+0000","last_fullsized":"2024-09-11T06:49:13.392147+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_clean_scrub_stamp":"2024-09-11T06:49:05.088839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:51.167301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":56808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640261,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":172032,"data_stored":56789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":765952,"data_stored":647157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640262,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":172032,"data_stored":56789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640262,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":56808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640263,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":765952,"data_stored":647157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:49:18.321 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T06:49:18.322 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-11T06:49:18.592 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 19 2024-09-11T06:49:18.592 INFO:teuthology.orchestra.run.smithi071.stdout:fsid bd5dcb33-e9f7-4a5f-a26c-a451452e8aa2 2024-09-11T06:49:18.592 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-09-11T06:48:52.191706+0000 2024-09-11T06:49:18.592 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-09-11T06:49:16.913361+0000 2024-09-11T06:49:18.592 INFO:teuthology.orchestra.run.smithi071.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-11T06:49:18.592 INFO:teuthology.orchestra.run.smithi071.stdout:crush_version 4 2024-09-11T06:49:18.593 INFO:teuthology.orchestra.run.smithi071.stdout:full_ratio 0.95 2024-09-11T06:49:18.593 INFO:teuthology.orchestra.run.smithi071.stdout:backfillfull_ratio 0.9 2024-09-11T06:49:18.593 INFO:teuthology.orchestra.run.smithi071.stdout:nearfull_ratio 0.85 2024-09-11T06:49:18.593 INFO:teuthology.orchestra.run.smithi071.stdout:require_min_compat_client luminous 2024-09-11T06:49:18.593 INFO:teuthology.orchestra.run.smithi071.stdout:min_compat_client jewel 2024-09-11T06:49:18.593 INFO:teuthology.orchestra.run.smithi071.stdout:require_osd_release squid 2024-09-11T06:49:18.593 INFO:teuthology.orchestra.run.smithi071.stdout:stretch_mode_enabled false 2024-09-11T06:49:18.593 INFO:teuthology.orchestra.run.smithi071.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 14 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 6.06 2024-09-11T06:49:18.593 INFO:teuthology.orchestra.run.smithi071.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 18 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 2.25 2024-09-11T06:49:18.593 INFO:teuthology.orchestra.run.smithi071.stdout:max_osd 6 2024-09-11T06:49:18.593 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 up in weight 1 up_from 11 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6800/1158147777,v1:172.21.15.71:6801/1158147777] [v2:172.21.15.71:6802/1158147777,v1:172.21.15.71:6803/1158147777] exists,up 3c9e35bf-d3db-49db-b10c-d2cf38c7eb7e 2024-09-11T06:49:18.594 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6808/620364628,v1:172.21.15.71:6809/620364628] [v2:172.21.15.71:6810/620364628,v1:172.21.15.71:6811/620364628] exists,up 693649dc-db7c-43cf-bdbe-87f6128ef311 2024-09-11T06:49:18.594 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6816/2926882521,v1:172.21.15.71:6817/2926882521] [v2:172.21.15.71:6818/2926882521,v1:172.21.15.71:6819/2926882521] exists,up 89421ae7-fead-409c-83f7-ede424db6004 2024-09-11T06:49:18.594 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6808/558942315,v1:172.21.15.89:6809/558942315] [v2:172.21.15.89:6810/558942315,v1:172.21.15.89:6811/558942315] exists,up f4512e48-63f1-42b6-81be-594bd82e3210 2024-09-11T06:49:18.594 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 up in weight 1 up_from 11 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6816/3878488003,v1:172.21.15.89:6817/3878488003] [v2:172.21.15.89:6818/3878488003,v1:172.21.15.89:6819/3878488003] exists,up 52d47b6f-af30-455e-8326-e53c598911ae 2024-09-11T06:49:18.594 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6800/2840204107,v1:172.21.15.89:6801/2840204107] [v2:172.21.15.89:6802/2840204107,v1:172.21.15.89:6803/2840204107] exists,up 6b4f9416-0ba0-4c58-883e-8cbdb362adcc 2024-09-11T06:49:18.609 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 11 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6800/1158147777,v1:172.21.15.71:6801/1158147777] [v2:172.21.15.71:6802/1158147777,v1:172.21.15.71:6803/1158147777] exists,up 3c9e35bf-d3db-49db-b10c-d2cf38c7eb7e', 'osd.1 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6808/620364628,v1:172.21.15.71:6809/620364628] [v2:172.21.15.71:6810/620364628,v1:172.21.15.71:6811/620364628] exists,up 693649dc-db7c-43cf-bdbe-87f6128ef311', 'osd.2 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6816/2926882521,v1:172.21.15.71:6817/2926882521] [v2:172.21.15.71:6818/2926882521,v1:172.21.15.71:6819/2926882521] exists,up 89421ae7-fead-409c-83f7-ede424db6004', 'osd.3 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6808/558942315,v1:172.21.15.89:6809/558942315] [v2:172.21.15.89:6810/558942315,v1:172.21.15.89:6811/558942315] exists,up f4512e48-63f1-42b6-81be-594bd82e3210', 'osd.4 up in weight 1 up_from 11 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6816/3878488003,v1:172.21.15.89:6817/3878488003] [v2:172.21.15.89:6818/3878488003,v1:172.21.15.89:6819/3878488003] exists,up 52d47b6f-af30-455e-8326-e53c598911ae', 'osd.5 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6800/2840204107,v1:172.21.15.89:6801/2840204107] [v2:172.21.15.89:6802/2840204107,v1:172.21.15.89:6803/2840204107] exists,up 6b4f9416-0ba0-4c58-883e-8cbdb362adcc'] 2024-09-11T06:49:18.610 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-11T06:49:18.726 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:18.727 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mon.a", 2024-09-11T06:49:18.727 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster": "ceph", 2024-09-11T06:49:18.727 INFO:teuthology.orchestra.run.smithi071.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-11T06:49:18.727 INFO:teuthology.orchestra.run.smithi071.stdout: "admin_socket_mode": "", 2024-09-11T06:49:18.727 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-11T06:49:18.727 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_client_required": "cephx, none", 2024-09-11T06:49:18.727 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_cluster_required": "cephx", 2024-09-11T06:49:18.727 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_debug": "true", 2024-09-11T06:49:18.727 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-11T06:49:18.727 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-11T06:49:18.727 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_service_required": "cephx", 2024-09-11T06:49:18.728 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-11T06:49:18.728 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_supported": "cephx", 2024-09-11T06:49:18.728 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_aio": "true", 2024-09-11T06:49:18.728 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-11T06:49:18.728 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_aio_poll_ms": "250", 2024-09-11T06:49:18.728 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_aio_reap_max": "16", 2024-09-11T06:49:18.728 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_async_discard": "false", 2024-09-11T06:49:18.728 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_block_size": "4096", 2024-09-11T06:49:18.728 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_debug_aio": "false", 2024-09-11T06:49:18.728 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-11T06:49:18.728 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-11T06:49:18.728 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_debug_inflight_ios": "false", 2024-09-11T06:49:18.728 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_enable_discard": "false", 2024-09-11T06:49:18.729 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_flock_retry": "3", 2024-09-11T06:49:18.729 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-11T06:49:18.729 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_inject_crash": "0", 2024-09-11T06:49:18.729 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-11T06:49:18.729 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_ioring": "false", 2024-09-11T06:49:18.729 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_ioring_hipri": "false", 2024-09-11T06:49:18.729 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-11T06:49:18.729 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-11T06:49:18.729 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-11T06:49:18.729 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-11T06:49:18.729 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-11T06:49:18.729 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-11T06:49:18.729 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_type": "", 2024-09-11T06:49:18.730 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_alloc_size": "1048576", 2024-09-11T06:49:18.730 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_allocator": "hybrid", 2024-09-11T06:49:18.730 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_buffered_io": "true", 2024-09-11T06:49:18.730 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_check_for_zeros": "false", 2024-09-11T06:49:18.730 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-11T06:49:18.730 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-11T06:49:18.730 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_compact_log_sync": "false", 2024-09-11T06:49:18.730 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-11T06:49:18.730 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-11T06:49:18.730 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-11T06:49:18.730 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-11T06:49:18.730 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_max_log_runway": "4194304", 2024-09-11T06:49:18.730 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_max_prefetch": "1048576", 2024-09-11T06:49:18.730 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_min_flush_size": "524288", 2024-09-11T06:49:18.731 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_min_log_runway": "1048576", 2024-09-11T06:49:18.731 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_replay_recovery": "false", 2024-09-11T06:49:18.731 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-11T06:49:18.731 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-11T06:49:18.731 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_sync_write": "false", 2024-09-11T06:49:18.731 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-11T06:49:18.731 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-11T06:49:18.731 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-11T06:49:18.731 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_allocation_from_file": "true", 2024-09-11T06:49:18.731 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_allocator": "hybrid", 2024-09-11T06:49:18.731 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-11T06:49:18.731 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-11T06:49:18.731 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-11T06:49:18.732 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-11T06:49:18.732 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_bdev_label_multi": "true", 2024-09-11T06:49:18.732 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-11T06:49:18.732 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-11T06:49:18.732 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-11T06:49:18.732 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-11T06:49:18.732 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-11T06:49:18.732 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_block_create": "true", 2024-09-11T06:49:18.732 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_block_db_create": "false", 2024-09-11T06:49:18.732 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_block_db_path": "", 2024-09-11T06:49:18.732 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_block_db_size": "0", 2024-09-11T06:49:18.733 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_block_path": "", 2024-09-11T06:49:18.733 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_block_preallocate_file": "false", 2024-09-11T06:49:18.733 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_block_size": "107374182400", 2024-09-11T06:49:18.733 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_block_wal_create": "false", 2024-09-11T06:49:18.733 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_block_wal_path": "", 2024-09-11T06:49:18.733 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_block_wal_size": "100663296", 2024-09-11T06:49:18.733 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_bluefs": "true", 2024-09-11T06:49:18.733 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-11T06:49:18.733 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-11T06:49:18.733 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-11T06:49:18.733 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-11T06:49:18.733 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-11T06:49:18.733 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-11T06:49:18.734 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-11T06:49:18.734 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-11T06:49:18.734 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_autotune": "true", 2024-09-11T06:49:18.734 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-11T06:49:18.734 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-11T06:49:18.734 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-11T06:49:18.734 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-11T06:49:18.734 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_size": "0", 2024-09-11T06:49:18.734 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-11T06:49:18.734 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-11T06:49:18.734 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-11T06:49:18.734 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-11T06:49:18.734 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_type": "2q", 2024-09-11T06:49:18.734 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-11T06:49:18.735 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_clone_cow": "true", 2024-09-11T06:49:18.735 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-11T06:49:18.735 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-11T06:49:18.735 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-11T06:49:18.735 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-11T06:49:18.735 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-11T06:49:18.735 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-11T06:49:18.735 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-11T06:49:18.735 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_compression_mode": "none", 2024-09-11T06:49:18.735 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-11T06:49:18.735 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_csum_type": "crc32c", 2024-09-11T06:49:18.735 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-11T06:49:18.735 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_freelist": "false", 2024-09-11T06:49:18.736 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-11T06:49:18.736 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-11T06:49:18.736 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-11T06:49:18.736 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-11T06:49:18.736 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-11T06:49:18.736 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-11T06:49:18.736 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-11T06:49:18.736 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-11T06:49:18.737 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-11T06:49:18.737 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-11T06:49:18.737 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-11T06:49:18.737 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-11T06:49:18.737 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_small_allocations": "0", 2024-09-11T06:49:18.737 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-11T06:49:18.737 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_default_buffered_read": "true", 2024-09-11T06:49:18.737 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_default_buffered_write": "false", 2024-09-11T06:49:18.737 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-11T06:49:18.738 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-11T06:49:18.738 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-11T06:49:18.738 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-11T06:49:18.738 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-11T06:49:18.738 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-11T06:49:18.738 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-11T06:49:18.738 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-11T06:49:18.738 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-11T06:49:18.738 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fail_eio": "false", 2024-09-11T06:49:18.738 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-11T06:49:18.738 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-11T06:49:18.738 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-11T06:49:18.738 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-11T06:49:18.739 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-11T06:49:18.739 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-11T06:49:18.739 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_on_mount": "false", 2024-09-11T06:49:18.739 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-11T06:49:18.739 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_on_umount": "false", 2024-09-11T06:49:18.739 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-11T06:49:18.739 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-11T06:49:18.739 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-11T06:49:18.739 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-11T06:49:18.739 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-11T06:49:18.739 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-11T06:49:18.739 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-11T06:49:18.739 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-11T06:49:18.739 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_ignore_data_csum": "false", 2024-09-11T06:49:18.740 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-11T06:49:18.740 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-11T06:49:18.740 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-11T06:49:18.740 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-11T06:49:18.740 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_log_op_age": "5.000000", 2024-09-11T06:49:18.740 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_max_alloc_size": "0", 2024-09-11T06:49:18.740 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_max_blob_size": "0", 2024-09-11T06:49:18.740 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-11T06:49:18.741 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-11T06:49:18.741 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-11T06:49:18.741 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_max_deferred_txc": "32", 2024-09-11T06:49:18.741 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_min_alloc_size": "0", 2024-09-11T06:49:18.741 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-11T06:49:18.741 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-11T06:49:18.741 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_nid_prealloc": "1024", 2024-09-11T06:49:18.742 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-11T06:49:18.742 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-11T06:49:18.742 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-11T06:49:18.742 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-11T06:49:18.742 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_retry_disk_reads": "3", 2024-09-11T06:49:18.742 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_rocksdb_cf": "true", 2024-09-11T06:49:18.743 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-11T06:49:18.743 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-11T06:49:18.743 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-11T06:49:18.743 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-11T06:49:18.743 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-11T06:49:18.743 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-11T06:49:18.743 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-11T06:49:18.744 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-11T06:49:18.744 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_spdk_mem": "512", 2024-09-11T06:49:18.744 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-11T06:49:18.744 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-11T06:49:18.744 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-11T06:49:18.744 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-11T06:49:18.745 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-11T06:49:18.745 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-11T06:49:18.745 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-11T06:49:18.745 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_tracing": "false", 2024-09-11T06:49:18.745 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-11T06:49:18.745 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-11T06:49:18.745 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-11T06:49:18.746 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-11T06:49:18.746 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-11T06:49:18.746 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-11T06:49:18.746 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-11T06:49:18.746 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-11T06:49:18.746 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-11T06:49:18.746 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_zero_block_detection": "false", 2024-09-11T06:49:18.746 INFO:teuthology.orchestra.run.smithi071.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-11T06:49:18.746 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-11T06:49:18.746 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-11T06:49:18.746 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-11T06:49:18.747 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-11T06:49:18.747 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-11T06:49:18.747 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-11T06:49:18.747 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-11T06:49:18.747 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-11T06:49:18.747 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-11T06:49:18.747 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-11T06:49:18.747 INFO:teuthology.orchestra.run.smithi071.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-11T06:49:18.747 INFO:teuthology.orchestra.run.smithi071.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-11T06:49:18.747 INFO:teuthology.orchestra.run.smithi071.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-11T06:49:18.747 INFO:teuthology.orchestra.run.smithi071.stdout: "cephx_cluster_require_signatures": "false", 2024-09-11T06:49:18.748 INFO:teuthology.orchestra.run.smithi071.stdout: "cephx_cluster_require_version": "2", 2024-09-11T06:49:18.748 INFO:teuthology.orchestra.run.smithi071.stdout: "cephx_require_signatures": "false", 2024-09-11T06:49:18.748 INFO:teuthology.orchestra.run.smithi071.stdout: "cephx_require_version": "2", 2024-09-11T06:49:18.748 INFO:teuthology.orchestra.run.smithi071.stdout: "cephx_service_require_signatures": "false", 2024-09-11T06:49:18.748 INFO:teuthology.orchestra.run.smithi071.stdout: "cephx_service_require_version": "2", 2024-09-11T06:49:18.748 INFO:teuthology.orchestra.run.smithi071.stdout: "cephx_sign_messages": "true", 2024-09-11T06:49:18.748 INFO:teuthology.orchestra.run.smithi071.stdout: "chdir": "", 2024-09-11T06:49:18.748 INFO:teuthology.orchestra.run.smithi071.stdout: "client_acl_type": "", 2024-09-11T06:49:18.748 INFO:teuthology.orchestra.run.smithi071.stdout: "client_asio_thread_count": "2", 2024-09-11T06:49:18.748 INFO:teuthology.orchestra.run.smithi071.stdout: "client_cache_mid": "0.750000", 2024-09-11T06:49:18.748 INFO:teuthology.orchestra.run.smithi071.stdout: "client_cache_size": "16384", 2024-09-11T06:49:18.748 INFO:teuthology.orchestra.run.smithi071.stdout: "client_caps_release_delay": "5", 2024-09-11T06:49:18.749 INFO:teuthology.orchestra.run.smithi071.stdout: "client_check_pool_perm": "true", 2024-09-11T06:49:18.749 INFO:teuthology.orchestra.run.smithi071.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-11T06:49:18.749 INFO:teuthology.orchestra.run.smithi071.stdout: "client_debug_force_sync_read": "false", 2024-09-11T06:49:18.749 INFO:teuthology.orchestra.run.smithi071.stdout: "client_debug_getattr_caps": "false", 2024-09-11T06:49:18.749 INFO:teuthology.orchestra.run.smithi071.stdout: "client_debug_inject_features": "", 2024-09-11T06:49:18.749 INFO:teuthology.orchestra.run.smithi071.stdout: "client_debug_inject_tick_delay": "0", 2024-09-11T06:49:18.749 INFO:teuthology.orchestra.run.smithi071.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-11T06:49:18.749 INFO:teuthology.orchestra.run.smithi071.stdout: "client_die_on_failed_remount": "false", 2024-09-11T06:49:18.749 INFO:teuthology.orchestra.run.smithi071.stdout: "client_dirsize_rbytes": "true", 2024-09-11T06:49:18.749 INFO:teuthology.orchestra.run.smithi071.stdout: "client_force_lazyio": "false", 2024-09-11T06:49:18.749 INFO:teuthology.orchestra.run.smithi071.stdout: "client_fs": "", 2024-09-11T06:49:18.750 INFO:teuthology.orchestra.run.smithi071.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-11T06:49:18.750 INFO:teuthology.orchestra.run.smithi071.stdout: "client_inject_release_failure": "false", 2024-09-11T06:49:18.750 INFO:teuthology.orchestra.run.smithi071.stdout: "client_max_inline_size": "4096", 2024-09-11T06:49:18.750 INFO:teuthology.orchestra.run.smithi071.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-11T06:49:18.750 INFO:teuthology.orchestra.run.smithi071.stdout: "client_mds_namespace": "", 2024-09-11T06:49:18.750 INFO:teuthology.orchestra.run.smithi071.stdout: "client_metadata": "", 2024-09-11T06:49:18.750 INFO:teuthology.orchestra.run.smithi071.stdout: "client_mount_gid": "-1", 2024-09-11T06:49:18.750 INFO:teuthology.orchestra.run.smithi071.stdout: "client_mount_timeout": "300", 2024-09-11T06:49:18.750 INFO:teuthology.orchestra.run.smithi071.stdout: "client_mount_uid": "-1", 2024-09-11T06:49:18.750 INFO:teuthology.orchestra.run.smithi071.stdout: "client_mountpoint": "/", 2024-09-11T06:49:18.750 INFO:teuthology.orchestra.run.smithi071.stdout: "client_notify_timeout": "10", 2024-09-11T06:49:18.750 INFO:teuthology.orchestra.run.smithi071.stdout: "client_oc": "true", 2024-09-11T06:49:18.750 INFO:teuthology.orchestra.run.smithi071.stdout: "client_oc_max_dirty": "104857600", 2024-09-11T06:49:18.750 INFO:teuthology.orchestra.run.smithi071.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-11T06:49:18.751 INFO:teuthology.orchestra.run.smithi071.stdout: "client_oc_max_objects": "1000", 2024-09-11T06:49:18.751 INFO:teuthology.orchestra.run.smithi071.stdout: "client_oc_size": "209715200", 2024-09-11T06:49:18.751 INFO:teuthology.orchestra.run.smithi071.stdout: "client_oc_target_dirty": "8388608", 2024-09-11T06:49:18.751 INFO:teuthology.orchestra.run.smithi071.stdout: "client_permissions": "true", 2024-09-11T06:49:18.751 INFO:teuthology.orchestra.run.smithi071.stdout: "client_quota": "true", 2024-09-11T06:49:18.751 INFO:teuthology.orchestra.run.smithi071.stdout: "client_quota_df": "true", 2024-09-11T06:49:18.751 INFO:teuthology.orchestra.run.smithi071.stdout: "client_readahead_max_bytes": "0", 2024-09-11T06:49:18.751 INFO:teuthology.orchestra.run.smithi071.stdout: "client_readahead_max_periods": "4", 2024-09-11T06:49:18.751 INFO:teuthology.orchestra.run.smithi071.stdout: "client_readahead_min": "131072", 2024-09-11T06:49:18.751 INFO:teuthology.orchestra.run.smithi071.stdout: "client_reconnect_stale": "false", 2024-09-11T06:49:18.751 INFO:teuthology.orchestra.run.smithi071.stdout: "client_shutdown_timeout": "30", 2024-09-11T06:49:18.751 INFO:teuthology.orchestra.run.smithi071.stdout: "client_snapdir": ".snap", 2024-09-11T06:49:18.751 INFO:teuthology.orchestra.run.smithi071.stdout: "client_tick_interval": "1", 2024-09-11T06:49:18.752 INFO:teuthology.orchestra.run.smithi071.stdout: "client_trace": "", 2024-09-11T06:49:18.752 INFO:teuthology.orchestra.run.smithi071.stdout: "client_try_dentry_invalidate": "false", 2024-09-11T06:49:18.752 INFO:teuthology.orchestra.run.smithi071.stdout: "client_use_faked_inos": "false", 2024-09-11T06:49:18.752 INFO:teuthology.orchestra.run.smithi071.stdout: "client_use_random_mds": "false", 2024-09-11T06:49:18.752 INFO:teuthology.orchestra.run.smithi071.stdout: "clog_to_graylog": "false", 2024-09-11T06:49:18.752 INFO:teuthology.orchestra.run.smithi071.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-11T06:49:18.752 INFO:teuthology.orchestra.run.smithi071.stdout: "clog_to_graylog_port": "12201", 2024-09-11T06:49:18.752 INFO:teuthology.orchestra.run.smithi071.stdout: "clog_to_monitors": "default=true", 2024-09-11T06:49:18.752 INFO:teuthology.orchestra.run.smithi071.stdout: "clog_to_syslog": "false", 2024-09-11T06:49:18.752 INFO:teuthology.orchestra.run.smithi071.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-11T06:49:18.752 INFO:teuthology.orchestra.run.smithi071.stdout: "clog_to_syslog_level": "info", 2024-09-11T06:49:18.752 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "-", 2024-09-11T06:49:18.752 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_network": "", 2024-09-11T06:49:18.752 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_network_interface": "", 2024-09-11T06:49:18.753 INFO:teuthology.orchestra.run.smithi071.stdout: "compressor_zlib_isal": "false", 2024-09-11T06:49:18.753 INFO:teuthology.orchestra.run.smithi071.stdout: "compressor_zlib_level": "5", 2024-09-11T06:49:18.753 INFO:teuthology.orchestra.run.smithi071.stdout: "compressor_zlib_winsize": "-15", 2024-09-11T06:49:18.753 INFO:teuthology.orchestra.run.smithi071.stdout: "compressor_zstd_level": "1", 2024-09-11T06:49:18.753 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-11T06:49:18.753 INFO:teuthology.orchestra.run.smithi071.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-11T06:49:18.753 INFO:teuthology.orchestra.run.smithi071.stdout: "crimson_alien_op_num_threads": "6", 2024-09-11T06:49:18.753 INFO:teuthology.orchestra.run.smithi071.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-11T06:49:18.753 INFO:teuthology.orchestra.run.smithi071.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-11T06:49:18.753 INFO:teuthology.orchestra.run.smithi071.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-11T06:49:18.753 INFO:teuthology.orchestra.run.smithi071.stdout: "crimson_osd_stat_interval": "0", 2024-09-11T06:49:18.753 INFO:teuthology.orchestra.run.smithi071.stdout: "crimson_seastar_cpu_cores": "", 2024-09-11T06:49:18.753 INFO:teuthology.orchestra.run.smithi071.stdout: "crimson_seastar_num_threads": "0", 2024-09-11T06:49:18.754 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_location": "", 2024-09-11T06:49:18.754 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_location_hook": "", 2024-09-11T06:49:18.754 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_location_hook_timeout": "10", 2024-09-11T06:49:18.754 INFO:teuthology.orchestra.run.smithi071.stdout: "daemonize": "false", 2024-09-11T06:49:18.754 INFO:teuthology.orchestra.run.smithi071.stdout: "daos_pool": "tank", 2024-09-11T06:49:18.754 INFO:teuthology.orchestra.run.smithi071.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-11T06:49:18.754 INFO:teuthology.orchestra.run.smithi071.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-11T06:49:18.754 INFO:teuthology.orchestra.run.smithi071.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-11T06:49:18.754 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_alienstore": "0/5", 2024-09-11T06:49:18.755 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_allow_any_pool_priority": "false", 2024-09-11T06:49:18.755 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_asok": "1/5", 2024-09-11T06:49:18.755 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_asok_assert_abort": "false", 2024-09-11T06:49:18.755 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_asserts_on_shutdown": "true", 2024-09-11T06:49:18.755 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_auth": "20/20", 2024-09-11T06:49:18.755 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_bdev": "1/3", 2024-09-11T06:49:18.755 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_bluefs": "1/5", 2024-09-11T06:49:18.755 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_bluestore": "1/5", 2024-09-11T06:49:18.755 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_buffer": "0/1", 2024-09-11T06:49:18.756 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_ceph_exporter": "1/5", 2024-09-11T06:49:18.756 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_cephfs_mirror": "0/5", 2024-09-11T06:49:18.756 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_cephsqlite": "0/5", 2024-09-11T06:49:18.756 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_client": "0/5", 2024-09-11T06:49:18.756 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_compressor": "1/5", 2024-09-11T06:49:18.756 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_context": "0/1", 2024-09-11T06:49:18.756 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_crimson_interrupt": "0/5", 2024-09-11T06:49:18.756 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_crush": "1/1", 2024-09-11T06:49:18.757 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_crypto": "1/5", 2024-09-11T06:49:18.757 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_cyanstore": "0/5", 2024-09-11T06:49:18.757 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_deliberately_leak_memory": "false", 2024-09-11T06:49:18.757 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_disable_randomized_ping": "false", 2024-09-11T06:49:18.757 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_dpdk": "1/5", 2024-09-11T06:49:18.757 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_eventtrace": "1/5", 2024-09-11T06:49:18.757 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_filer": "0/1", 2024-09-11T06:49:18.757 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_filestore": "1/3", 2024-09-11T06:49:18.757 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_finisher": "1/1", 2024-09-11T06:49:18.758 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_fuse": "1/5", 2024-09-11T06:49:18.758 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_heartbeat_testing_span": "0", 2024-09-11T06:49:18.758 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_heartbeatmap": "1/5", 2024-09-11T06:49:18.758 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-11T06:49:18.758 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_javaclient": "1/5", 2024-09-11T06:49:18.758 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_journal": "1/3", 2024-09-11T06:49:18.758 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_journaler": "0/5", 2024-09-11T06:49:18.758 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_kstore": "1/5", 2024-09-11T06:49:18.758 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_lockdep": "0/1", 2024-09-11T06:49:18.759 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_mclock": "1/5", 2024-09-11T06:49:18.759 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_mds": "1/5", 2024-09-11T06:49:18.759 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_mds_balancer": "1/5", 2024-09-11T06:49:18.759 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_mds_locker": "1/5", 2024-09-11T06:49:18.759 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_mds_log": "1/5", 2024-09-11T06:49:18.759 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_mds_log_expire": "1/5", 2024-09-11T06:49:18.759 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_mds_migrator": "1/5", 2024-09-11T06:49:18.759 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_mds_quiesce": "3/5", 2024-09-11T06:49:18.760 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_memstore": "1/5", 2024-09-11T06:49:18.760 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_mgr": "2/5", 2024-09-11T06:49:18.760 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_mgrc": "1/5", 2024-09-11T06:49:18.760 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_mon": "20/20", 2024-09-11T06:49:18.760 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_monc": "0/10", 2024-09-11T06:49:18.760 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_ms": "1/1", 2024-09-11T06:49:18.760 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_none": "0/5", 2024-09-11T06:49:18.760 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_objclass": "0/5", 2024-09-11T06:49:18.761 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_objectcacher": "0/5", 2024-09-11T06:49:18.761 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_objecter": "0/1", 2024-09-11T06:49:18.761 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_optracker": "0/5", 2024-09-11T06:49:18.761 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_osd": "1/5", 2024-09-11T06:49:18.761 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_paxos": "20/20", 2024-09-11T06:49:18.761 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_perfcounter": "1/5", 2024-09-11T06:49:18.761 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_prioritycache": "1/5", 2024-09-11T06:49:18.761 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_rados": "0/5", 2024-09-11T06:49:18.762 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_rbd": "0/5", 2024-09-11T06:49:18.762 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_rbd_mirror": "0/5", 2024-09-11T06:49:18.762 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_rbd_pwl": "0/5", 2024-09-11T06:49:18.762 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_rbd_replay": "0/5", 2024-09-11T06:49:18.762 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_refs": "0/0", 2024-09-11T06:49:18.762 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_reserver": "1/1", 2024-09-11T06:49:18.762 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_rgw": "1/5", 2024-09-11T06:49:18.762 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_rgw_access": "1/5", 2024-09-11T06:49:18.763 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_rgw_datacache": "1/5", 2024-09-11T06:49:18.763 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_rgw_dbstore": "1/5", 2024-09-11T06:49:18.763 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_rgw_flight": "1/5", 2024-09-11T06:49:18.763 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-11T06:49:18.763 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_rgw_sync": "1/5", 2024-09-11T06:49:18.763 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_rocksdb": "4/5", 2024-09-11T06:49:18.763 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_seastore": "0/5", 2024-09-11T06:49:18.763 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_seastore_backref": "0/5", 2024-09-11T06:49:18.763 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_seastore_cache": "0/5", 2024-09-11T06:49:18.764 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_seastore_cleaner": "0/5", 2024-09-11T06:49:18.764 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_seastore_device": "0/5", 2024-09-11T06:49:18.764 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_seastore_epm": "0/5", 2024-09-11T06:49:18.764 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-11T06:49:18.764 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_seastore_journal": "0/5", 2024-09-11T06:49:18.764 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_seastore_lba": "0/5", 2024-09-11T06:49:18.764 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_seastore_odata": "0/5", 2024-09-11T06:49:18.764 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_seastore_omap": "0/5", 2024-09-11T06:49:18.765 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_seastore_onode": "0/5", 2024-09-11T06:49:18.765 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_seastore_t": "0/5", 2024-09-11T06:49:18.765 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_seastore_tm": "0/5", 2024-09-11T06:49:18.765 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_striper": "0/1", 2024-09-11T06:49:18.765 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_test": "0/5", 2024-09-11T06:49:18.765 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_throttle": "1/1", 2024-09-11T06:49:18.765 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_timer": "0/1", 2024-09-11T06:49:18.765 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_tp": "0/5", 2024-09-11T06:49:18.765 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_trace": "1/5", 2024-09-11T06:49:18.766 INFO:teuthology.orchestra.run.smithi071.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-11T06:49:18.766 INFO:teuthology.orchestra.run.smithi071.stdout: "device_failure_prediction_mode": "none", 2024-09-11T06:49:18.766 INFO:teuthology.orchestra.run.smithi071.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-11T06:49:18.766 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-11T06:49:18.766 INFO:teuthology.orchestra.run.smithi071.stdout: "err_to_graylog": "false", 2024-09-11T06:49:18.766 INFO:teuthology.orchestra.run.smithi071.stdout: "err_to_journald": "false", 2024-09-11T06:49:18.766 INFO:teuthology.orchestra.run.smithi071.stdout: "err_to_stderr": "true", 2024-09-11T06:49:18.766 INFO:teuthology.orchestra.run.smithi071.stdout: "err_to_syslog": "false", 2024-09-11T06:49:18.767 INFO:teuthology.orchestra.run.smithi071.stdout: "event_tracing": "false", 2024-09-11T06:49:18.767 INFO:teuthology.orchestra.run.smithi071.stdout: "exporter_addr": "0.0.0.0", 2024-09-11T06:49:18.767 INFO:teuthology.orchestra.run.smithi071.stdout: "exporter_http_port": "9926", 2024-09-11T06:49:18.767 INFO:teuthology.orchestra.run.smithi071.stdout: "exporter_prio_limit": "5", 2024-09-11T06:49:18.767 INFO:teuthology.orchestra.run.smithi071.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-11T06:49:18.767 INFO:teuthology.orchestra.run.smithi071.stdout: "exporter_sort_metrics": "true", 2024-09-11T06:49:18.767 INFO:teuthology.orchestra.run.smithi071.stdout: "exporter_stats_period": "5", 2024-09-11T06:49:18.767 INFO:teuthology.orchestra.run.smithi071.stdout: "fake_statfs_for_testing": "0", 2024-09-11T06:49:18.767 INFO:teuthology.orchestra.run.smithi071.stdout: "fatal_signal_handlers": "true", 2024-09-11T06:49:18.768 INFO:teuthology.orchestra.run.smithi071.stdout: "filer_max_purge_ops": "10", 2024-09-11T06:49:18.768 INFO:teuthology.orchestra.run.smithi071.stdout: "filer_max_truncate_ops": "128", 2024-09-11T06:49:18.768 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_apply_finisher_threads": "1", 2024-09-11T06:49:18.768 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_blackhole": "false", 2024-09-11T06:49:18.769 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_btrfs_clone_range": "true", 2024-09-11T06:49:18.769 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_btrfs_snap": "true", 2024-09-11T06:49:18.769 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_caller_concurrency": "10", 2024-09-11T06:49:18.769 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_collect_device_partition_information": "true", 2024-09-11T06:49:18.769 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_commit_timeout": "600.000000", 2024-09-11T06:49:18.769 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_debug_inject_read_err": "false", 2024-09-11T06:49:18.769 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_debug_omap_check": "false", 2024-09-11T06:49:18.769 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-11T06:49:18.769 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_debug_verify_split": "false", 2024-09-11T06:49:18.770 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_dump_file": "", 2024-09-11T06:49:18.770 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-11T06:49:18.770 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-11T06:49:18.770 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_fadvise": "true", 2024-09-11T06:49:18.770 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_fail_eio": "true", 2024-09-11T06:49:18.770 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_fd_cache_shards": "16", 2024-09-11T06:49:18.770 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_fd_cache_size": "128", 2024-09-11T06:49:18.770 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_fiemap": "false", 2024-09-11T06:49:18.770 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_fiemap_threshold": "4096", 2024-09-11T06:49:18.770 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-11T06:49:18.770 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-11T06:49:18.770 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_inject_stall": "0", 2024-09-11T06:49:18.770 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_journal_parallel": "false", 2024-09-11T06:49:18.771 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_journal_trailing": "false", 2024-09-11T06:49:18.771 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_journal_writeahead": "false", 2024-09-11T06:49:18.771 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_kill_at": "0", 2024-09-11T06:49:18.771 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-11T06:49:18.771 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-11T06:49:18.771 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-11T06:49:18.771 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-11T06:49:18.771 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-11T06:49:18.771 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_max_inline_xattrs": "0", 2024-09-11T06:49:18.771 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-11T06:49:18.771 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-11T06:49:18.772 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-11T06:49:18.772 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-11T06:49:18.772 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_max_xattr_value_size": "0", 2024-09-11T06:49:18.772 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-11T06:49:18.772 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-11T06:49:18.772 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-11T06:49:18.772 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_merge_threshold": "-10", 2024-09-11T06:49:18.772 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-11T06:49:18.772 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_odsync_write": "false", 2024-09-11T06:49:18.772 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_omap_backend": "rocksdb", 2024-09-11T06:49:18.772 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_omap_backend_path": "", 2024-09-11T06:49:18.772 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-11T06:49:18.772 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-11T06:49:18.772 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-11T06:49:18.773 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_op_thread_timeout": "60", 2024-09-11T06:49:18.773 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_op_threads": "2", 2024-09-11T06:49:18.773 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_punch_hole": "false", 2024-09-11T06:49:18.773 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-11T06:49:18.773 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-11T06:49:18.773 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-11T06:49:18.773 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-11T06:49:18.773 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-11T06:49:18.773 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-11T06:49:18.773 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-11T06:49:18.773 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-11T06:49:18.773 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-11T06:49:18.773 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_queue_max_ops": "50", 2024-09-11T06:49:18.773 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-11T06:49:18.774 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_seek_data_hole": "false", 2024-09-11T06:49:18.774 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_sloppy_crc": "false", 2024-09-11T06:49:18.774 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-11T06:49:18.774 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_splice": "false", 2024-09-11T06:49:18.774 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_split_multiple": "2", 2024-09-11T06:49:18.774 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_split_rand_factor": "20", 2024-09-11T06:49:18.774 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_update_to": "1000", 2024-09-11T06:49:18.774 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-11T06:49:18.774 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-11T06:49:18.774 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-11T06:49:18.774 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-11T06:49:18.775 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-11T06:49:18.775 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-11T06:49:18.775 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_wbthrottle_enable": "true", 2024-09-11T06:49:18.775 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-11T06:49:18.775 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-11T06:49:18.775 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-11T06:49:18.775 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-11T06:49:18.775 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-11T06:49:18.775 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-11T06:49:18.775 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_xfs_extsize": "false", 2024-09-11T06:49:18.775 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_zfs_snap": "false", 2024-09-11T06:49:18.775 INFO:teuthology.orchestra.run.smithi071.stdout: "fio_dir": "/tmp/fio", 2024-09-11T06:49:18.775 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "bd5dcb33-e9f7-4a5f-a26c-a451452e8aa2", 2024-09-11T06:49:18.775 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_allow_other": "true", 2024-09-11T06:49:18.776 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_atomic_o_trunc": "true", 2024-09-11T06:49:18.776 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_big_writes": "true", 2024-09-11T06:49:18.776 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_debug": "false", 2024-09-11T06:49:18.776 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_default_permissions": "false", 2024-09-11T06:49:18.776 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_disable_pagecache": "false", 2024-09-11T06:49:18.776 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_max_write": "0", 2024-09-11T06:49:18.776 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_multithreaded": "true", 2024-09-11T06:49:18.776 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_require_active_mds": "true", 2024-09-11T06:49:18.776 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_set_user_groups": "true", 2024-09-11T06:49:18.776 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_splice_move": "true", 2024-09-11T06:49:18.776 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_splice_read": "true", 2024-09-11T06:49:18.776 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_splice_write": "true", 2024-09-11T06:49:18.777 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-11T06:49:18.777 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_use_invalidate_cb": "true", 2024-09-11T06:49:18.777 INFO:teuthology.orchestra.run.smithi071.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-11T06:49:18.777 INFO:teuthology.orchestra.run.smithi071.stdout: "gss_target_name": "ceph", 2024-09-11T06:49:18.777 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_file": "", 2024-09-11T06:49:18.777 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_inject_failure": "0", 2024-09-11T06:49:18.777 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_interval": "5", 2024-09-11T06:49:18.777 INFO:teuthology.orchestra.run.smithi071.stdout: "host": "smithi071", 2024-09-11T06:49:18.777 INFO:teuthology.orchestra.run.smithi071.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-11T06:49:18.777 INFO:teuthology.orchestra.run.smithi071.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-11T06:49:18.777 INFO:teuthology.orchestra.run.smithi071.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-11T06:49:18.777 INFO:teuthology.orchestra.run.smithi071.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-11T06:49:18.777 INFO:teuthology.orchestra.run.smithi071.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-11T06:49:18.778 INFO:teuthology.orchestra.run.smithi071.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-11T06:49:18.778 INFO:teuthology.orchestra.run.smithi071.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-11T06:49:18.778 INFO:teuthology.orchestra.run.smithi071.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-11T06:49:18.778 INFO:teuthology.orchestra.run.smithi071.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-11T06:49:18.778 INFO:teuthology.orchestra.run.smithi071.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-11T06:49:18.778 INFO:teuthology.orchestra.run.smithi071.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-11T06:49:18.778 INFO:teuthology.orchestra.run.smithi071.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-11T06:49:18.778 INFO:teuthology.orchestra.run.smithi071.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-11T06:49:18.778 INFO:teuthology.orchestra.run.smithi071.stdout: "inject_early_sigterm": "false", 2024-09-11T06:49:18.778 INFO:teuthology.orchestra.run.smithi071.stdout: "jaeger_agent_port": "6799", 2024-09-11T06:49:18.778 INFO:teuthology.orchestra.run.smithi071.stdout: "jaeger_tracing_enable": "false", 2024-09-11T06:49:18.778 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_aio": "true", 2024-09-11T06:49:18.778 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_align_min_size": "65536", 2024-09-11T06:49:18.779 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_block_align": "true", 2024-09-11T06:49:18.779 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_block_size": "4096", 2024-09-11T06:49:18.779 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_dio": "true", 2024-09-11T06:49:18.779 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_discard": "false", 2024-09-11T06:49:18.779 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_force_aio": "false", 2024-09-11T06:49:18.779 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_ignore_corruption": "false", 2024-09-11T06:49:18.779 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_max_write_bytes": "10485760", 2024-09-11T06:49:18.779 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_max_write_entries": "100", 2024-09-11T06:49:18.779 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_replay_from": "0", 2024-09-11T06:49:18.779 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-11T06:49:18.779 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-11T06:49:18.779 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-11T06:49:18.779 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-11T06:49:18.780 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_write_header_frequency": "0", 2024-09-11T06:49:18.780 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_zero_on_create": "false", 2024-09-11T06:49:18.780 INFO:teuthology.orchestra.run.smithi071.stdout: "journaler_prefetch_periods": "10", 2024-09-11T06:49:18.780 INFO:teuthology.orchestra.run.smithi071.stdout: "journaler_prezero_periods": "5", 2024-09-11T06:49:18.780 INFO:teuthology.orchestra.run.smithi071.stdout: "journaler_write_head_interval": "15", 2024-09-11T06:49:18.780 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "", 2024-09-11T06:49:18.780 INFO:teuthology.orchestra.run.smithi071.stdout: "keyfile": "", 2024-09-11T06:49:18.780 INFO:teuthology.orchestra.run.smithi071.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-11T06:49:18.780 INFO:teuthology.orchestra.run.smithi071.stdout: "kstore_backend": "rocksdb", 2024-09-11T06:49:18.780 INFO:teuthology.orchestra.run.smithi071.stdout: "kstore_default_stripe_size": "65536", 2024-09-11T06:49:18.780 INFO:teuthology.orchestra.run.smithi071.stdout: "kstore_fsck_on_mount": "false", 2024-09-11T06:49:18.780 INFO:teuthology.orchestra.run.smithi071.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-11T06:49:18.780 INFO:teuthology.orchestra.run.smithi071.stdout: "kstore_max_bytes": "67108864", 2024-09-11T06:49:18.781 INFO:teuthology.orchestra.run.smithi071.stdout: "kstore_max_ops": "512", 2024-09-11T06:49:18.781 INFO:teuthology.orchestra.run.smithi071.stdout: "kstore_nid_prealloc": "1024", 2024-09-11T06:49:18.781 INFO:teuthology.orchestra.run.smithi071.stdout: "kstore_onode_map_size": "1024", 2024-09-11T06:49:18.781 INFO:teuthology.orchestra.run.smithi071.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-11T06:49:18.781 INFO:teuthology.orchestra.run.smithi071.stdout: "kstore_sync_submit_transaction": "false", 2024-09-11T06:49:18.781 INFO:teuthology.orchestra.run.smithi071.stdout: "kstore_sync_transaction": "false", 2024-09-11T06:49:18.781 INFO:teuthology.orchestra.run.smithi071.stdout: "librados_thread_count": "2", 2024-09-11T06:49:18.781 INFO:teuthology.orchestra.run.smithi071.stdout: "lockdep": "false", 2024-09-11T06:49:18.781 INFO:teuthology.orchestra.run.smithi071.stdout: "lockdep_force_backtrace": "false", 2024-09-11T06:49:18.781 INFO:teuthology.orchestra.run.smithi071.stdout: "log_coarse_timestamps": "true", 2024-09-11T06:49:18.781 INFO:teuthology.orchestra.run.smithi071.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-11T06:49:18.781 INFO:teuthology.orchestra.run.smithi071.stdout: "log_flush_on_exit": "false", 2024-09-11T06:49:18.781 INFO:teuthology.orchestra.run.smithi071.stdout: "log_graylog_host": "127.0.0.1", 2024-09-11T06:49:18.782 INFO:teuthology.orchestra.run.smithi071.stdout: "log_graylog_port": "12201", 2024-09-11T06:49:18.782 INFO:teuthology.orchestra.run.smithi071.stdout: "log_max_new": "1000", 2024-09-11T06:49:18.782 INFO:teuthology.orchestra.run.smithi071.stdout: "log_max_recent": "10000", 2024-09-11T06:49:18.782 INFO:teuthology.orchestra.run.smithi071.stdout: "log_stderr_prefix": "", 2024-09-11T06:49:18.782 INFO:teuthology.orchestra.run.smithi071.stdout: "log_stop_at_utilization": "0.970000", 2024-09-11T06:49:18.782 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": "true", 2024-09-11T06:49:18.782 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_graylog": "false", 2024-09-11T06:49:18.782 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_journald": "false", 2024-09-11T06:49:18.782 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_stderr": "false", 2024-09-11T06:49:18.782 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_syslog": "false", 2024-09-11T06:49:18.782 INFO:teuthology.orchestra.run.smithi071.stdout: "max_rotating_auth_attempts": "10", 2024-09-11T06:49:18.782 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-11T06:49:18.782 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_action_on_write_error": "1", 2024-09-11T06:49:18.782 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_alternate_name_max": "8192", 2024-09-11T06:49:18.783 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_asio_thread_count": "2", 2024-09-11T06:49:18.783 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_export_pin": "true", 2024-09-11T06:49:18.783 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_fragment_dirs": "true", 2024-09-11T06:49:18.783 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-11T06:49:18.783 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_fragment_interval": "5", 2024-09-11T06:49:18.783 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-11T06:49:18.783 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-11T06:49:18.783 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_interval": "10", 2024-09-11T06:49:18.783 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_max": "-1", 2024-09-11T06:49:18.783 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_max_until": "-1", 2024-09-11T06:49:18.783 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_merge_size": "50", 2024-09-11T06:49:18.783 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_midchunk": "0.300000", 2024-09-11T06:49:18.783 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-11T06:49:18.783 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_min_start": "0.200000", 2024-09-11T06:49:18.784 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_minchunk": "0.001000", 2024-09-11T06:49:18.784 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_mode": "0", 2024-09-11T06:49:18.784 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_need_max": "1.200000", 2024-09-11T06:49:18.784 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_need_min": "0.800000", 2024-09-11T06:49:18.784 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_overload_epochs": "2", 2024-09-11T06:49:18.784 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-11T06:49:18.784 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-11T06:49:18.784 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_split_bits": "3", 2024-09-11T06:49:18.784 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-11T06:49:18.784 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_split_size": "10000", 2024-09-11T06:49:18.784 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-11T06:49:18.784 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_target_decay": "10.000000", 2024-09-11T06:49:18.784 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-11T06:49:18.785 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_beacon_grace": "15.000000", 2024-09-11T06:49:18.785 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_beacon_interval": "4.000000", 2024-09-11T06:49:18.785 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-11T06:49:18.785 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-11T06:49:18.785 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache_mid": "0.700000", 2024-09-11T06:49:18.785 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-11T06:49:18.785 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache_quiesce_delay": "0", 2024-09-11T06:49:18.785 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-11T06:49:18.785 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-11T06:49:18.785 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-11T06:49:18.785 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache_release_free_interval": "10", 2024-09-11T06:49:18.785 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache_reservation": "0.050000", 2024-09-11T06:49:18.785 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-11T06:49:18.786 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache_trim_interval": "1", 2024-09-11T06:49:18.786 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache_trim_threshold": "262144", 2024-09-11T06:49:18.786 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-11T06:49:18.786 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-11T06:49:18.786 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-11T06:49:18.786 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_client_prealloc_inos": "1000", 2024-09-11T06:49:18.786 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-11T06:49:18.786 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_connect_bootstrapping": "false", 2024-09-11T06:49:18.786 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_damage_table_max_entries": "10000", 2024-09-11T06:49:18.786 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-11T06:49:18.786 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_debug_auth_pins": "false", 2024-09-11T06:49:18.786 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_debug_frag": "false", 2024-09-11T06:49:18.786 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_debug_scatterstat": "false", 2024-09-11T06:49:18.786 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_debug_subtrees": "false", 2024-09-11T06:49:18.787 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_decay_halflife": "5.000000", 2024-09-11T06:49:18.787 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_default_dir_hash": "2", 2024-09-11T06:49:18.787 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_defer_session_stale": "true", 2024-09-11T06:49:18.787 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_deny_all_reconnect": "false", 2024-09-11T06:49:18.787 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_dir_keys_per_op": "16384", 2024-09-11T06:49:18.787 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_dir_max_commit_size": "10", 2024-09-11T06:49:18.787 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_dir_max_entries": "0", 2024-09-11T06:49:18.787 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_dir_prefetch": "true", 2024-09-11T06:49:18.787 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-11T06:49:18.787 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-11T06:49:18.787 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_dump_cache_on_map": "false", 2024-09-11T06:49:18.787 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-11T06:49:18.787 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-11T06:49:18.787 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_early_reply": "true", 2024-09-11T06:49:18.788 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_enable_op_tracker": "true", 2024-09-11T06:49:18.788 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_enforce_unique_name": "true", 2024-09-11T06:49:18.788 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-11T06:49:18.788 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-11T06:49:18.788 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_export_ephemeral_random": "true", 2024-09-11T06:49:18.788 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-11T06:49:18.788 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-11T06:49:18.788 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-11T06:49:18.788 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-11T06:49:18.788 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-11T06:49:18.788 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-11T06:49:18.788 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-11T06:49:18.789 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-11T06:49:18.789 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_health_summarize_threshold": "10", 2024-09-11T06:49:18.789 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-11T06:49:18.789 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-11T06:49:18.789 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_inject_health_dummy": "false", 2024-09-11T06:49:18.789 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-11T06:49:18.789 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_inject_migrator_session_race": "false", 2024-09-11T06:49:18.789 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-11T06:49:18.789 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-11T06:49:18.789 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-11T06:49:18.789 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_join_fs": "", 2024-09-11T06:49:18.789 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_journal_format": "1", 2024-09-11T06:49:18.789 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-11T06:49:18.789 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_kill_create_at": "0", 2024-09-11T06:49:18.790 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_kill_export_at": "0", 2024-09-11T06:49:18.790 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_kill_import_at": "0", 2024-09-11T06:49:18.790 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_kill_journal_at": "0", 2024-09-11T06:49:18.790 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_kill_journal_expire_at": "0", 2024-09-11T06:49:18.790 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_kill_journal_replay_at": "0", 2024-09-11T06:49:18.790 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_kill_link_at": "0", 2024-09-11T06:49:18.790 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_kill_mdstable_at": "0", 2024-09-11T06:49:18.790 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_kill_openc_at": "0", 2024-09-11T06:49:18.790 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_kill_rename_at": "0", 2024-09-11T06:49:18.790 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_kill_shutdown_at": "0", 2024-09-11T06:49:18.790 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log_event_large_threshold": "524288", 2024-09-11T06:49:18.790 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log_events_per_segment": "1024", 2024-09-11T06:49:18.790 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-11T06:49:18.791 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log_max_events": "-1", 2024-09-11T06:49:18.791 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log_max_segments": "128", 2024-09-11T06:49:18.791 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log_pause": "false", 2024-09-11T06:49:18.791 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log_segment_size": "0", 2024-09-11T06:49:18.791 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-11T06:49:18.791 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-11T06:49:18.791 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-11T06:49:18.791 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log_trim_threshold": "128", 2024-09-11T06:49:18.791 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-11T06:49:18.791 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log_warn_factor": "2.000000", 2024-09-11T06:49:18.791 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_max_caps_per_client": "1048576", 2024-09-11T06:49:18.791 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_max_completed_flushes": "100000", 2024-09-11T06:49:18.791 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_max_completed_requests": "100000", 2024-09-11T06:49:18.791 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_max_export_size": "20971520", 2024-09-11T06:49:18.792 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_max_file_recover": "32", 2024-09-11T06:49:18.792 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_max_purge_files": "64", 2024-09-11T06:49:18.792 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_max_purge_ops": "8192", 2024-09-11T06:49:18.792 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-11T06:49:18.792 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-11T06:49:18.792 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_max_snaps_per_dir": "100", 2024-09-11T06:49:18.792 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_metrics_update_interval": "2", 2024-09-11T06:49:18.792 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_min_caps_per_client": "100", 2024-09-11T06:49:18.792 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_min_caps_working_set": "10000", 2024-09-11T06:49:18.792 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-11T06:49:18.792 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_numa_node": "-1", 2024-09-11T06:49:18.792 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-11T06:49:18.792 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_op_complaint_time": "30.000000", 2024-09-11T06:49:18.793 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_op_history_duration": "600", 2024-09-11T06:49:18.793 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_op_history_size": "20", 2024-09-11T06:49:18.793 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_op_history_slow_op_size": "20", 2024-09-11T06:49:18.793 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-11T06:49:18.793 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_op_log_threshold": "5", 2024-09-11T06:49:18.793 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_ping_grace": "15", 2024-09-11T06:49:18.793 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_ping_interval": "5", 2024-09-11T06:49:18.793 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-11T06:49:18.793 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-11T06:49:18.793 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_recall_max_caps": "30000", 2024-09-11T06:49:18.793 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-11T06:49:18.793 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-11T06:49:18.794 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-11T06:49:18.794 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_recall_warning_threshold": "262144", 2024-09-11T06:49:18.794 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-11T06:49:18.794 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_replay_interval": "1.000000", 2024-09-11T06:49:18.794 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-11T06:49:18.794 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-11T06:49:18.794 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_root_ino_gid": "0", 2024-09-11T06:49:18.794 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_root_ino_uid": "0", 2024-09-11T06:49:18.794 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-11T06:49:18.794 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-11T06:49:18.794 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-11T06:49:18.794 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-11T06:49:18.794 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-11T06:49:18.794 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-11T06:49:18.795 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-11T06:49:18.795 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-11T06:49:18.795 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-11T06:49:18.795 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-11T06:49:18.795 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_shutdown_check": "0", 2024-09-11T06:49:18.795 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_skip_ino": "0", 2024-09-11T06:49:18.795 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-11T06:49:18.795 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_snap_max_uid": "4294967294", 2024-09-11T06:49:18.795 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_snap_min_uid": "0", 2024-09-11T06:49:18.795 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_snap_rstat": "false", 2024-09-11T06:49:18.795 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_standby_replay_damaged": "false", 2024-09-11T06:49:18.795 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_symlink_recovery": "true", 2024-09-11T06:49:18.795 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-11T06:49:18.795 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_thrash_exports": "0", 2024-09-11T06:49:18.796 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_thrash_fragments": "0", 2024-09-11T06:49:18.796 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_tick_interval": "5.000000", 2024-09-11T06:49:18.796 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_valgrind_exit": "false", 2024-09-11T06:49:18.796 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_verify_backtrace": "1", 2024-09-11T06:49:18.796 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_verify_scatter": "false", 2024-09-11T06:49:18.796 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-11T06:49:18.796 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_wipe_sessions": "false", 2024-09-11T06:49:18.796 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool_debug": "false", 2024-09-11T06:49:18.796 INFO:teuthology.orchestra.run.smithi071.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-11T06:49:18.796 INFO:teuthology.orchestra.run.smithi071.stdout: "memstore_device_bytes": "1073741824", 2024-09-11T06:49:18.796 INFO:teuthology.orchestra.run.smithi071.stdout: "memstore_page_set": "false", 2024-09-11T06:49:18.796 INFO:teuthology.orchestra.run.smithi071.stdout: "memstore_page_size": "65536", 2024-09-11T06:49:18.796 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_client_bytes": "134217728", 2024-09-11T06:49:18.797 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_client_messages": "512", 2024-09-11T06:49:18.797 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-11T06:49:18.797 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-11T06:49:18.797 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-11T06:49:18.797 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-11T06:49:18.797 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_disabled_modules": "", 2024-09-11T06:49:18.797 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_enable_op_tracker": "true", 2024-09-11T06:49:18.797 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-11T06:49:18.797 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_max_pg_num_change": "128", 2024-09-11T06:49:18.797 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_mds_bytes": "134217728", 2024-09-11T06:49:18.797 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_mds_messages": "128", 2024-09-11T06:49:18.797 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-11T06:49:18.797 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_mon_bytes": "134217728", 2024-09-11T06:49:18.798 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_mon_messages": "128", 2024-09-11T06:49:18.798 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-11T06:49:18.798 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-11T06:49:18.798 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_op_history_duration": "600", 2024-09-11T06:49:18.798 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_op_history_size": "20", 2024-09-11T06:49:18.798 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-11T06:49:18.798 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-11T06:49:18.798 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_op_log_threshold": "5", 2024-09-11T06:49:18.798 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_osd_bytes": "536870912", 2024-09-11T06:49:18.798 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_osd_messages": "8192", 2024-09-11T06:49:18.798 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_pool": "true", 2024-09-11T06:49:18.799 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-11T06:49:18.799 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_standby_modules": "true", 2024-09-11T06:49:18.799 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_stats_period": "5", 2024-09-11T06:49:18.799 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_stats_threshold": "5", 2024-09-11T06:49:18.799 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_tick_period": "2", 2024-09-11T06:49:18.799 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-11T06:49:18.799 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-11T06:49:18.799 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_allow_pool_delete": "true", 2024-09-11T06:49:18.799 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_allow_pool_size_one": "true", 2024-09-11T06:49:18.799 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_auth_validate_all_caps": "true", 2024-09-11T06:49:18.800 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-11T06:49:18.800 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-11T06:49:18.800 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_client_bytes": "104857600", 2024-09-11T06:49:18.800 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_client_directed_command_retry": "5", 2024-09-11T06:49:18.800 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-11T06:49:18.800 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-11T06:49:18.800 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-11T06:49:18.800 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-11T06:49:18.800 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_client_hunt_parallel": "3", 2024-09-11T06:49:18.800 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_client_log_interval": "1.000000", 2024-09-11T06:49:18.801 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-11T06:49:18.801 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_client_ping_interval": "10.000000", 2024-09-11T06:49:18.801 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-11T06:49:18.801 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_client_target_rank": "-1", 2024-09-11T06:49:18.801 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-11T06:49:18.801 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-11T06:49:18.801 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-11T06:49:18.801 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_cluster_log_file_level": "debug", 2024-09-11T06:49:18.801 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_cluster_log_to_file": "true", 2024-09-11T06:49:18.801 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-11T06:49:18.802 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-11T06:49:18.802 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-11T06:49:18.802 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_cluster_log_to_journald": "false", 2024-09-11T06:49:18.802 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-11T06:49:18.802 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-11T06:49:18.802 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-11T06:49:18.802 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-11T06:49:18.802 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_compact_on_bootstrap": "false", 2024-09-11T06:49:18.802 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_compact_on_start": "false", 2024-09-11T06:49:18.802 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_compact_on_trim": "true", 2024-09-11T06:49:18.802 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-11T06:49:18.802 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-11T06:49:18.802 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-11T06:49:18.803 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_cpu_threads": "4", 2024-09-11T06:49:18.803 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_crush_min_required_version": "hammer", 2024-09-11T06:49:18.803 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_daemon_bytes": "419430400", 2024-09-11T06:49:18.803 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-11T06:49:18.803 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_data_avail_crit": "5", 2024-09-11T06:49:18.803 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_data_avail_warn": "5", 2024-09-11T06:49:18.803 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_data_size_warn": "16106127360", 2024-09-11T06:49:18.803 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-11T06:49:18.803 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-11T06:49:18.803 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_debug_dump_json": "false", 2024-09-11T06:49:18.803 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-11T06:49:18.803 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_debug_dump_transactions": "false", 2024-09-11T06:49:18.803 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_debug_extra_checks": "false", 2024-09-11T06:49:18.803 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-11T06:49:18.804 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-11T06:49:18.804 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_debug_no_require_reef": "false", 2024-09-11T06:49:18.804 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_debug_no_require_squid": "false", 2024-09-11T06:49:18.804 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-11T06:49:18.804 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-11T06:49:18.804 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-11T06:49:18.804 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_down_mkfs_grace": "120", 2024-09-11T06:49:18.804 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_down_uptime_grace": "60", 2024-09-11T06:49:18.804 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_election_default_strategy": "1", 2024-09-11T06:49:18.804 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_election_timeout": "5.000000", 2024-09-11T06:49:18.804 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-11T06:49:18.804 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_elector_ping_divisor": "2", 2024-09-11T06:49:18.804 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-11T06:49:18.805 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_enable_op_tracker": "true", 2024-09-11T06:49:18.805 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_fake_pool_delete": "false", 2024-09-11T06:49:18.805 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_force_quorum_join": "false", 2024-09-11T06:49:18.805 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-11T06:49:18.805 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_globalid_prealloc": "10000", 2024-09-11T06:49:18.805 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_health_detail_to_clog": "false", 2024-09-11T06:49:18.805 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_health_log_update_period": "5", 2024-09-11T06:49:18.805 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_health_max_detail": "50", 2024-09-11T06:49:18.805 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_health_to_clog": "true", 2024-09-11T06:49:18.805 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_health_to_clog_interval": "600", 2024-09-11T06:49:18.805 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-11T06:49:18.806 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_host": "172.21.15.71,[v2:172.21.15.71:3301,v1:172.21.15.71:6790],172.21.15.89", 2024-09-11T06:49:18.806 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_host_override": "", 2024-09-11T06:49:18.806 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_initial_members": "", 2024-09-11T06:49:18.806 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-11T06:49:18.806 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-11T06:49:18.806 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-11T06:49:18.806 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-11T06:49:18.806 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-11T06:49:18.806 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_lease": "5.000000", 2024-09-11T06:49:18.806 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-11T06:49:18.806 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-11T06:49:18.806 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_log_full_interval": "50", 2024-09-11T06:49:18.807 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_log_max": "10000", 2024-09-11T06:49:18.807 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_log_max_summary": "50", 2024-09-11T06:49:18.807 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-11T06:49:18.807 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_max_log_epochs": "500", 2024-09-11T06:49:18.807 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-11T06:49:18.807 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-11T06:49:18.808 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_max_osd": "10000", 2024-09-11T06:49:18.808 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_max_pg_per_osd": "10000", 2024-09-11T06:49:18.808 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_max_pool_pg_num": "65536", 2024-09-11T06:49:18.808 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-11T06:49:18.808 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-11T06:49:18.808 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_mds_force_trim_to": "0", 2024-09-11T06:49:18.809 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_mds_skip_sanity": "false", 2024-09-11T06:49:18.809 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_memory_autotune": "true", 2024-09-11T06:49:18.809 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_memory_target": "2147483648", 2024-09-11T06:49:18.809 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_mgr_beacon_grace": "30", 2024-09-11T06:49:18.809 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-11T06:49:18.809 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_mgr_digest_period": "5", 2024-09-11T06:49:18.809 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_mgr_inactive_grace": "60", 2024-09-11T06:49:18.810 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-11T06:49:18.810 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-11T06:49:18.810 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_min_osdmap_epochs": "500", 2024-09-11T06:49:18.810 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_op_complaint_time": "30", 2024-09-11T06:49:18.810 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_op_history_duration": "600", 2024-09-11T06:49:18.810 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_op_history_size": "20", 2024-09-11T06:49:18.810 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_op_history_slow_op_size": "20", 2024-09-11T06:49:18.811 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-11T06:49:18.811 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_op_log_threshold": "5", 2024-09-11T06:49:18.811 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-11T06:49:18.811 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-11T06:49:18.811 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-11T06:49:18.811 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_auto_mark_in": "false", 2024-09-11T06:49:18.812 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-11T06:49:18.812 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-11T06:49:18.812 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-11T06:49:18.812 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_cache_size": "500", 2024-09-11T06:49:18.812 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-11T06:49:18.812 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-11T06:49:18.812 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-11T06:49:18.812 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_down_out_interval": "600", 2024-09-11T06:49:18.812 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-11T06:49:18.812 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-11T06:49:18.812 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_force_trim_to": "0", 2024-09-11T06:49:18.813 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-11T06:49:18.813 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-11T06:49:18.813 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-11T06:49:18.813 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-11T06:49:18.813 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-11T06:49:18.813 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-11T06:49:18.813 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-11T06:49:18.813 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-11T06:49:18.813 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_min_down_reporters": "2", 2024-09-11T06:49:18.813 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-11T06:49:18.813 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-11T06:49:18.813 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-11T06:49:18.813 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-11T06:49:18.814 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-11T06:49:18.814 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-11T06:49:18.814 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_report_timeout": "900", 2024-09-11T06:49:18.814 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-11T06:49:18.814 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-11T06:49:18.814 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-11T06:49:18.814 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-11T06:49:18.814 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-11T06:49:18.814 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-11T06:49:18.814 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-11T06:49:18.814 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-11T06:49:18.814 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-11T06:49:18.814 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_pg_stuck_threshold": "60", 2024-09-11T06:49:18.814 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-11T06:49:18.815 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-11T06:49:18.815 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-11T06:49:18.815 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-11T06:49:18.815 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-11T06:49:18.815 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-11T06:49:18.815 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_probe_timeout": "2.000000", 2024-09-11T06:49:18.815 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_reweight_max_change": "0.050000", 2024-09-11T06:49:18.815 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_reweight_max_osds": "4", 2024-09-11T06:49:18.815 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-11T06:49:18.815 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-11T06:49:18.815 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-11T06:49:18.815 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-11T06:49:18.815 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-11T06:49:18.815 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_scrub_interval": "86400", 2024-09-11T06:49:18.816 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_scrub_max_keys": "100", 2024-09-11T06:49:18.816 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_scrub_timeout": "300", 2024-09-11T06:49:18.816 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_session_timeout": "300", 2024-09-11T06:49:18.816 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_smart_report_timeout": "5", 2024-09-11T06:49:18.816 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_stat_smooth_intervals": "6", 2024-09-11T06:49:18.816 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-11T06:49:18.816 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_stretch_pool_min_size": "2", 2024-09-11T06:49:18.816 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_stretch_pool_size": "4", 2024-09-11T06:49:18.816 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-11T06:49:18.816 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-11T06:49:18.816 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_sync_debug": "false", 2024-09-11T06:49:18.816 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-11T06:49:18.816 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-11T06:49:18.816 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_sync_provider_kill_at": "0", 2024-09-11T06:49:18.817 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_sync_requester_kill_at": "0", 2024-09-11T06:49:18.817 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_sync_timeout": "60.000000", 2024-09-11T06:49:18.817 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_target_pg_per_osd": "100", 2024-09-11T06:49:18.817 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_tick_interval": "5", 2024-09-11T06:49:18.817 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_timecheck_interval": "300.000000", 2024-09-11T06:49:18.817 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-11T06:49:18.817 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_use_min_delay_socket": "false", 2024-09-11T06:49:18.817 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_older_version_delay": "604800", 2024-09-11T06:49:18.817 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-11T06:49:18.817 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-11T06:49:18.817 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-11T06:49:18.818 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-11T06:49:18.818 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-11T06:49:18.818 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-11T06:49:18.818 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-11T06:49:18.818 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_misplaced": "false", 2024-09-11T06:49:18.818 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-11T06:49:18.818 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_older_version": "true", 2024-09-11T06:49:18.818 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-11T06:49:18.818 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-11T06:49:18.818 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-11T06:49:18.818 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-11T06:49:18.818 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-11T06:49:18.818 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-11T06:49:18.819 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-11T06:49:18.819 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-11T06:49:18.819 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-11T06:49:18.819 INFO:teuthology.orchestra.run.smithi071.stdout: "monmap": "", 2024-09-11T06:49:18.819 INFO:teuthology.orchestra.run.smithi071.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-11T06:49:18.819 INFO:teuthology.orchestra.run.smithi071.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-11T06:49:18.819 INFO:teuthology.orchestra.run.smithi071.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-11T06:49:18.819 INFO:teuthology.orchestra.run.smithi071.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-11T06:49:18.819 INFO:teuthology.orchestra.run.smithi071.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-11T06:49:18.819 INFO:teuthology.orchestra.run.smithi071.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-11T06:49:18.819 INFO:teuthology.orchestra.run.smithi071.stdout: "motr_tracing_enabled": "false", 2024-09-11T06:49:18.819 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_op_threads": "3", 2024-09-11T06:49:18.819 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-11T06:49:18.819 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_cm": "false", 2024-09-11T06:49:18.820 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_device_name": "", 2024-09-11T06:49:18.820 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_dscp": "96", 2024-09-11T06:49:18.820 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-11T06:49:18.820 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-11T06:49:18.820 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_local_gid": "", 2024-09-11T06:49:18.820 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-11T06:49:18.820 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_port_num": "1", 2024-09-11T06:49:18.820 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-11T06:49:18.820 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-11T06:49:18.820 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-11T06:49:18.820 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-11T06:49:18.821 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_sl": "3", 2024-09-11T06:49:18.821 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_support_srq": "true", 2024-09-11T06:49:18.821 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_type": "ib", 2024-09-11T06:49:18.821 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_reap_threshold": "5", 2024-09-11T06:49:18.821 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_bind_before_connect": "false", 2024-09-11T06:49:18.821 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_bind_ipv4": "true", 2024-09-11T06:49:18.821 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_bind_ipv6": "false", 2024-09-11T06:49:18.821 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_bind_msgr1": "true", 2024-09-11T06:49:18.821 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_bind_msgr2": "true", 2024-09-11T06:49:18.821 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_bind_port_max": "7568", 2024-09-11T06:49:18.821 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_bind_port_min": "6800", 2024-09-11T06:49:18.821 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-11T06:49:18.821 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_bind_retry_count": "3", 2024-09-11T06:49:18.822 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_bind_retry_delay": "5", 2024-09-11T06:49:18.822 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_blackhole_client": "false", 2024-09-11T06:49:18.822 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_blackhole_mds": "false", 2024-09-11T06:49:18.822 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_blackhole_mgr": "false", 2024-09-11T06:49:18.822 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_blackhole_mon": "false", 2024-09-11T06:49:18.822 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_blackhole_osd": "false", 2024-09-11T06:49:18.822 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_client_mode": "crc secure", 2024-09-11T06:49:18.822 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_cluster_mode": "crc secure", 2024-09-11T06:49:18.822 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_cluster_type": "", 2024-09-11T06:49:18.822 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_compress_secure": "false", 2024-09-11T06:49:18.822 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_connection_idle_timeout": "900", 2024-09-11T06:49:18.822 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_connection_ready_timeout": "10", 2024-09-11T06:49:18.823 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_crc_data": "true", 2024-09-11T06:49:18.823 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_crc_header": "true", 2024-09-11T06:49:18.823 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_die_on_bad_msg": "false", 2024-09-11T06:49:18.823 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_die_on_bug": "true", 2024-09-11T06:49:18.823 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_die_on_old_message": "true", 2024-09-11T06:49:18.823 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_die_on_skipped_message": "false", 2024-09-11T06:49:18.823 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-11T06:49:18.823 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-11T06:49:18.823 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_coremask": "0xF", 2024-09-11T06:49:18.823 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-11T06:49:18.823 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-11T06:49:18.823 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_enable_tso": "true", 2024-09-11T06:49:18.823 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-11T06:49:18.823 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-11T06:49:18.823 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_hugepages": "", 2024-09-11T06:49:18.824 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-11T06:49:18.824 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-11T06:49:18.824 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_lro": "true", 2024-09-11T06:49:18.824 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_memory_channel": "4", 2024-09-11T06:49:18.824 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-11T06:49:18.824 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_pmd": "", 2024-09-11T06:49:18.824 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_port_id": "0", 2024-09-11T06:49:18.824 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-11T06:49:18.824 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-11T06:49:18.824 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dump_on_send": "false", 2024-09-11T06:49:18.824 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_initial_backoff": "0.200000", 2024-09-11T06:49:18.824 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_inject_delay_max": "1.000000", 2024-09-11T06:49:18.825 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-11T06:49:18.825 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_inject_delay_type": "", 2024-09-11T06:49:18.825 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-11T06:49:18.825 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_inject_network_congestion": "0", 2024-09-11T06:49:18.825 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_inject_socket_failures": "5000", 2024-09-11T06:49:18.825 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_learn_addr_from_peer": "true", 2024-09-11T06:49:18.825 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_max_accept_failures": "4", 2024-09-11T06:49:18.825 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_max_backoff": "15.000000", 2024-09-11T06:49:18.825 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_mon_client_mode": "secure crc", 2024-09-11T06:49:18.825 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-11T06:49:18.825 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_mon_service_mode": "secure crc", 2024-09-11T06:49:18.825 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_osd_compress_min_size": "1024", 2024-09-11T06:49:18.825 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_osd_compress_mode": "none", 2024-09-11T06:49:18.825 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-11T06:49:18.826 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-11T06:49:18.826 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_pq_min_cost": "65536", 2024-09-11T06:49:18.826 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_public_type": "", 2024-09-11T06:49:18.826 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_service_mode": "crc secure", 2024-09-11T06:49:18.826 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_tcp_listen_backlog": "512", 2024-09-11T06:49:18.826 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_tcp_nodelay": "true", 2024-09-11T06:49:18.826 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-11T06:49:18.826 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_tcp_rcvbuf": "0", 2024-09-11T06:49:18.826 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_type": "async+posix", 2024-09-11T06:49:18.826 INFO:teuthology.orchestra.run.smithi071.stdout: "no_config_file": "false", 2024-09-11T06:49:18.826 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter_completion_locks_per_session": "32", 2024-09-11T06:49:18.826 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-11T06:49:18.826 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-11T06:49:18.826 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter_inflight_ops": "1024", 2024-09-11T06:49:18.827 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-11T06:49:18.827 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-11T06:49:18.827 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter_tick_interval": "5.000000", 2024-09-11T06:49:18.827 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter_timeout": "10.000000", 2024-09-11T06:49:18.827 INFO:teuthology.orchestra.run.smithi071.stdout: "objectstore_blackhole": "false", 2024-09-11T06:49:18.827 INFO:teuthology.orchestra.run.smithi071.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-11T06:49:18.827 INFO:teuthology.orchestra.run.smithi071.stdout: "openssl_engine_opts": "", 2024-09-11T06:49:18.827 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_agent_delay_time": "5.000000", 2024-09-11T06:49:18.827 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_agent_hist_halflife": "1000", 2024-09-11T06:49:18.827 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_agent_max_low_ops": "2", 2024-09-11T06:49:18.827 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_agent_max_ops": "4", 2024-09-11T06:49:18.827 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-11T06:49:18.827 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-11T06:49:18.827 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_agent_slop": "0.020000", 2024-09-11T06:49:18.827 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-11T06:49:18.828 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-11T06:49:18.828 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_asio_thread_count": "2", 2024-09-11T06:49:18.828 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_async_recovery_min_cost": "100", 2024-09-11T06:49:18.828 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-11T06:49:18.828 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-11T06:49:18.828 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_backfill_scan_max": "512", 2024-09-11T06:49:18.828 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_backfill_scan_min": "64", 2024-09-11T06:49:18.828 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_backoff_on_degraded": "false", 2024-09-11T06:49:18.828 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_backoff_on_peering": "false", 2024-09-11T06:49:18.828 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_backoff_on_unfound": "true", 2024-09-11T06:49:18.828 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_beacon_report_interval": "300", 2024-09-11T06:49:18.828 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bench_duration": "30", 2024-09-11T06:49:18.828 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-11T06:49:18.829 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bench_max_block_size": "67108864", 2024-09-11T06:49:18.829 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-11T06:49:18.829 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_blkin_trace_all": "false", 2024-09-11T06:49:18.829 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-11T06:49:18.829 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-11T06:49:18.829 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-11T06:49:18.829 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-11T06:49:18.829 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_check_for_log_corruption": "false", 2024-09-11T06:49:18.829 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-11T06:49:18.829 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-11T06:49:18.829 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-11T06:49:18.830 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-11T06:49:18.830 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_class_update_on_start": "true", 2024-09-11T06:49:18.830 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_client_message_cap": "256", 2024-09-11T06:49:18.830 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_client_message_size_cap": "524288000", 2024-09-11T06:49:18.830 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_client_op_priority": "63", 2024-09-11T06:49:18.830 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_client_watch_timeout": "30", 2024-09-11T06:49:18.830 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_command_max_records": "256", 2024-09-11T06:49:18.830 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_compact_on_start": "false", 2024-09-11T06:49:18.830 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-11T06:49:18.830 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-11T06:49:18.830 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-11T06:49:18.830 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_crush_update_on_start": "true", 2024-09-11T06:49:18.830 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_crush_update_weight_set": "true", 2024-09-11T06:49:18.831 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-11T06:49:18.831 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-11T06:49:18.831 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-11T06:49:18.831 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-11T06:49:18.831 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-11T06:49:18.831 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_feed_pullee": "-1", 2024-09-11T06:49:18.831 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-11T06:49:18.831 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-11T06:49:18.831 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-11T06:49:18.831 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_misdirected_ops": "false", 2024-09-11T06:49:18.831 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_no_acting_change": "false", 2024-09-11T06:49:18.831 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_no_purge_strays": "false", 2024-09-11T06:49:18.831 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_op_order": "false", 2024-09-11T06:49:18.831 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-11T06:49:18.832 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-11T06:49:18.832 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-11T06:49:18.832 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-11T06:49:18.832 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_shutdown": "false", 2024-09-11T06:49:18.832 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-11T06:49:18.832 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-11T06:49:18.832 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_trim_objects": "false", 2024-09-11T06:49:18.833 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-11T06:49:18.833 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-11T06:49:18.833 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_verify_snaps": "false", 2024-09-11T06:49:18.833 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-11T06:49:18.833 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-11T06:49:18.833 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_deep_scrub_keys": "1024", 2024-09-11T06:49:18.834 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-11T06:49:18.834 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-11T06:49:18.834 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-11T06:49:18.834 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_deep_scrub_stride": "524288", 2024-09-11T06:49:18.834 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-11T06:49:18.834 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-11T06:49:18.835 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_default_notify_timeout": "30", 2024-09-11T06:49:18.835 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_delete_sleep": "0.000000", 2024-09-11T06:49:18.835 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-11T06:49:18.835 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-11T06:49:18.835 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-11T06:49:18.835 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_discard_disconnected_ops": "true", 2024-09-11T06:49:18.836 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_enable_op_tracker": "true", 2024-09-11T06:49:18.836 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-11T06:49:18.836 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-11T06:49:18.836 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-11T06:49:18.836 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-11T06:49:18.836 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_fast_info": "true", 2024-09-11T06:49:18.836 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_fast_shutdown": "true", 2024-09-11T06:49:18.836 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-11T06:49:18.837 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-11T06:49:18.837 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-11T06:49:18.837 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-11T06:49:18.837 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-11T06:49:18.837 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_function_tracing": "false", 2024-09-11T06:49:18.837 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_heartbeat_grace": "20", 2024-09-11T06:49:18.837 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_heartbeat_interval": "6", 2024-09-11T06:49:18.837 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-11T06:49:18.837 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_heartbeat_min_peers": "10", 2024-09-11T06:49:18.837 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_heartbeat_min_size": "2000", 2024-09-11T06:49:18.837 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_heartbeat_stale": "600", 2024-09-11T06:49:18.837 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-11T06:49:18.837 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_hit_set_max_size": "100000", 2024-09-11T06:49:18.837 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_hit_set_min_size": "1000", 2024-09-11T06:49:18.838 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-11T06:49:18.838 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-11T06:49:18.838 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-11T06:49:18.838 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-11T06:49:18.838 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-11T06:49:18.838 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-11T06:49:18.838 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_journal_size": "5120", 2024-09-11T06:49:18.838 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_kill_backfill_at": "0", 2024-09-11T06:49:18.838 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-11T06:49:18.838 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_map_cache_size": "50", 2024-09-11T06:49:18.838 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_map_dedup": "true", 2024-09-11T06:49:18.838 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_map_message_max": "40", 2024-09-11T06:49:18.838 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-11T06:49:18.838 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_map_share_max_epochs": "40", 2024-09-11T06:49:18.839 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_attr_name_len": "100", 2024-09-11T06:49:18.839 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_attr_size": "0", 2024-09-11T06:49:18.839 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_backfills": "1", 2024-09-11T06:49:18.839 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_markdown_count": "5", 2024-09-11T06:49:18.839 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_markdown_period": "600", 2024-09-11T06:49:18.839 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_object_name_len": "2048", 2024-09-11T06:49:18.839 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_object_namespace_len": "256", 2024-09-11T06:49:18.839 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_object_size": "134217728", 2024-09-11T06:49:18.839 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-11T06:49:18.839 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-11T06:49:18.839 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_pg_blocked_by": "16", 2024-09-11T06:49:18.839 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_pg_log_entries": "10000", 2024-09-11T06:49:18.839 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-11T06:49:18.839 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_pgls": "1024", 2024-09-11T06:49:18.840 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_push_cost": "8388608", 2024-09-11T06:49:18.840 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_push_objects": "10", 2024-09-11T06:49:18.840 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_scrubs": "3", 2024-09-11T06:49:18.840 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-11T06:49:18.840 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_trimming_pgs": "2", 2024-09-11T06:49:18.840 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_write_op_reply_len": "64", 2024-09-11T06:49:18.840 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_write_size": "90", 2024-09-11T06:49:18.840 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-11T06:49:18.840 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-11T06:49:18.840 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-11T06:49:18.840 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-11T06:49:18.841 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-11T06:49:18.841 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-11T06:49:18.841 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-11T06:49:18.841 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-11T06:49:18.841 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_profile": "balanced", 2024-09-11T06:49:18.841 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-11T06:49:18.841 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-11T06:49:18.841 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-11T06:49:18.841 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-11T06:49:18.841 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-11T06:49:18.841 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-11T06:49:18.841 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-11T06:49:18.842 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-11T06:49:18.842 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-11T06:49:18.842 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-11T06:49:18.842 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-11T06:49:18.842 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_memory_base": "805306368", 2024-09-11T06:49:18.842 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_memory_cache_min": "134217728", 2024-09-11T06:49:18.842 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-11T06:49:18.842 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-11T06:49:18.842 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_memory_target": "4294967296", 2024-09-11T06:49:18.842 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_memory_target_autotune": "false", 2024-09-11T06:49:18.842 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-11T06:49:18.842 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_min_pg_log_entries": "250", 2024-09-11T06:49:18.842 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_min_recovery_priority": "0", 2024-09-11T06:49:18.842 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-11T06:49:18.843 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-11T06:49:18.843 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mon_report_interval": "5", 2024-09-11T06:49:18.843 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-11T06:49:18.843 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-11T06:49:18.843 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_num_cache_shards": "32", 2024-09-11T06:49:18.843 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_num_op_tracker_shard": "32", 2024-09-11T06:49:18.843 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_numa_auto_affinity": "true", 2024-09-11T06:49:18.843 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_numa_node": "-1", 2024-09-11T06:49:18.843 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_numa_prefer_iface": "true", 2024-09-11T06:49:18.843 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-11T06:49:18.843 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_objecter_finishers": "1", 2024-09-11T06:49:18.843 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_objectstore": "bluestore", 2024-09-11T06:49:18.843 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_objectstore_fuse": "false", 2024-09-11T06:49:18.843 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_objectstore_tracing": "false", 2024-09-11T06:49:18.844 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_complaint_time": "30.000000", 2024-09-11T06:49:18.844 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_history_duration": "600", 2024-09-11T06:49:18.844 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_history_size": "20", 2024-09-11T06:49:18.844 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_history_slow_op_size": "20", 2024-09-11T06:49:18.844 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-11T06:49:18.844 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_log_threshold": "5", 2024-09-11T06:49:18.844 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_num_shards": "0", 2024-09-11T06:49:18.844 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_num_shards_hdd": "5", 2024-09-11T06:49:18.844 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_num_shards_ssd": "8", 2024-09-11T06:49:18.844 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-11T06:49:18.844 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-11T06:49:18.844 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-11T06:49:18.844 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-11T06:49:18.845 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_pq_min_cost": "65536", 2024-09-11T06:49:18.845 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-11T06:49:18.845 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_queue_cut_off": "high", 2024-09-11T06:49:18.845 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-11T06:49:18.845 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_thread_timeout": "15", 2024-09-11T06:49:18.845 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_open_classes_on_start": "true", 2024-09-11T06:49:18.845 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_os_flags": "0", 2024-09-11T06:49:18.845 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_peering_op_priority": "255", 2024-09-11T06:49:18.845 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pg_delete_cost": "1048576", 2024-09-11T06:49:18.845 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pg_delete_priority": "5", 2024-09-11T06:49:18.845 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-11T06:49:18.846 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-11T06:49:18.846 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-11T06:49:18.846 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pg_log_trim_max": "10000", 2024-09-11T06:49:18.846 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pg_log_trim_min": "100", 2024-09-11T06:49:18.846 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-11T06:49:18.846 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-11T06:49:18.846 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-11T06:49:18.846 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-11T06:49:18.846 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-11T06:49:18.846 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-11T06:49:18.846 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-11T06:49:18.847 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-11T06:49:18.847 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-11T06:49:18.847 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-11T06:49:18.847 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_crimson": "false", 2024-09-11T06:49:18.847 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-11T06:49:18.847 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-11T06:49:18.847 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-11T06:49:18.847 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-11T06:49:18.847 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-11T06:49:18.847 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-11T06:49:18.847 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-11T06:49:18.847 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-11T06:49:18.847 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_flags": "0", 2024-09-11T06:49:18.848 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-11T06:49:18.848 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_min_size": "0", 2024-09-11T06:49:18.848 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-11T06:49:18.848 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_pg_num": "32", 2024-09-11T06:49:18.848 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_pgp_num": "0", 2024-09-11T06:49:18.848 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-11T06:49:18.848 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_read_ratio": "70", 2024-09-11T06:49:18.848 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_size": "2", 2024-09-11T06:49:18.848 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_type": "replicated", 2024-09-11T06:49:18.848 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-11T06:49:18.848 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-11T06:49:18.848 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_push_per_object_cost": "1000", 2024-09-11T06:49:18.849 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-11T06:49:18.849 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recover_clone_overlap": "true", 2024-09-11T06:49:18.849 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-11T06:49:18.849 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_cost": "20971520", 2024-09-11T06:49:18.849 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-11T06:49:18.849 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_max_active": "0", 2024-09-11T06:49:18.849 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-11T06:49:18.849 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-11T06:49:18.849 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-11T06:49:18.849 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-11T06:49:18.849 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_max_single_start": "1", 2024-09-11T06:49:18.849 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_op_priority": "3", 2024-09-11T06:49:18.850 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-11T06:49:18.850 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_priority": "5", 2024-09-11T06:49:18.850 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-11T06:49:18.850 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_sleep": "0.000000", 2024-09-11T06:49:18.850 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-11T06:49:18.850 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-11T06:49:18.850 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-11T06:49:18.850 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_repair_during_recovery": "false", 2024-09-11T06:49:18.850 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_requested_scrub_priority": "120", 2024-09-11T06:49:18.850 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-11T06:49:18.850 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-11T06:49:18.850 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_auto_repair": "false", 2024-09-11T06:49:18.851 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-11T06:49:18.851 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-11T06:49:18.851 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_begin_hour": "0", 2024-09-11T06:49:18.851 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_begin_week_day": "0", 2024-09-11T06:49:18.851 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_chunk_max": "25", 2024-09-11T06:49:18.851 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_chunk_min": "5", 2024-09-11T06:49:18.851 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_cost": "52428800", 2024-09-11T06:49:18.851 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-11T06:49:18.851 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_during_recovery": "false", 2024-09-11T06:49:18.851 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_end_hour": "0", 2024-09-11T06:49:18.852 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_end_week_day": "0", 2024-09-11T06:49:18.852 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_event_cost": "4096", 2024-09-11T06:49:18.852 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-11T06:49:18.852 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-11T06:49:18.852 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_invalid_stats": "true", 2024-09-11T06:49:18.852 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-11T06:49:18.852 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-11T06:49:18.852 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_max_preemptions": "5", 2024-09-11T06:49:18.852 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-11T06:49:18.852 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_priority": "5", 2024-09-11T06:49:18.852 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_sleep": "0.000000", 2024-09-11T06:49:18.852 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-11T06:49:18.852 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-11T06:49:18.853 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-11T06:49:18.853 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_skip_data_digest": "false", 2024-09-11T06:49:18.853 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_smart_report_timeout": "5", 2024-09-11T06:49:18.853 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_snap_trim_cost": "1048576", 2024-09-11T06:49:18.853 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_snap_trim_priority": "5", 2024-09-11T06:49:18.853 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-11T06:49:18.853 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-11T06:49:18.853 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-11T06:49:18.853 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-11T06:49:18.853 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-11T06:49:18.853 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-11T06:49:18.853 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-11T06:49:18.853 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_target_transaction_size": "30", 2024-09-11T06:49:18.853 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-11T06:49:18.854 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-11T06:49:18.854 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-11T06:49:18.854 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-11T06:49:18.854 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-11T06:49:18.854 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-11T06:49:18.854 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-11T06:49:18.854 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-11T06:49:18.854 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-11T06:49:18.854 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-11T06:49:18.854 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_tracing": "false", 2024-09-11T06:49:18.854 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_use_stale_snap": "false", 2024-09-11T06:49:18.854 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-11T06:49:18.854 INFO:teuthology.orchestra.run.smithi071.stdout: "osdc_blkin_trace_all": "false", 2024-09-11T06:49:18.854 INFO:teuthology.orchestra.run.smithi071.stdout: "paxos_kill_at": "0", 2024-09-11T06:49:18.855 INFO:teuthology.orchestra.run.smithi071.stdout: "paxos_max_join_drift": "10", 2024-09-11T06:49:18.855 INFO:teuthology.orchestra.run.smithi071.stdout: "paxos_min": "500", 2024-09-11T06:49:18.855 INFO:teuthology.orchestra.run.smithi071.stdout: "paxos_min_wait": "0.050000", 2024-09-11T06:49:18.855 INFO:teuthology.orchestra.run.smithi071.stdout: "paxos_propose_interval": "1.000000", 2024-09-11T06:49:18.855 INFO:teuthology.orchestra.run.smithi071.stdout: "paxos_service_trim_max": "500", 2024-09-11T06:49:18.855 INFO:teuthology.orchestra.run.smithi071.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-11T06:49:18.855 INFO:teuthology.orchestra.run.smithi071.stdout: "paxos_service_trim_min": "250", 2024-09-11T06:49:18.855 INFO:teuthology.orchestra.run.smithi071.stdout: "paxos_stash_full_interval": "25", 2024-09-11T06:49:18.855 INFO:teuthology.orchestra.run.smithi071.stdout: "paxos_trim_max": "500", 2024-09-11T06:49:18.855 INFO:teuthology.orchestra.run.smithi071.stdout: "paxos_trim_min": "250", 2024-09-11T06:49:18.855 INFO:teuthology.orchestra.run.smithi071.stdout: "perf": "true", 2024-09-11T06:49:18.855 INFO:teuthology.orchestra.run.smithi071.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-11T06:49:18.856 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-11T06:49:18.856 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-11T06:49:18.856 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "-", 2024-09-11T06:49:18.856 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrv": "", 2024-09-11T06:49:18.856 INFO:teuthology.orchestra.run.smithi071.stdout: "public_bind_addr": "-", 2024-09-11T06:49:18.856 INFO:teuthology.orchestra.run.smithi071.stdout: "public_network": "", 2024-09-11T06:49:18.856 INFO:teuthology.orchestra.run.smithi071.stdout: "public_network_interface": "", 2024-09-11T06:49:18.856 INFO:teuthology.orchestra.run.smithi071.stdout: "qat_compressor_busy_polling": "false", 2024-09-11T06:49:18.856 INFO:teuthology.orchestra.run.smithi071.stdout: "qat_compressor_enabled": "false", 2024-09-11T06:49:18.856 INFO:teuthology.orchestra.run.smithi071.stdout: "qat_compressor_session_max_number": "256", 2024-09-11T06:49:18.856 INFO:teuthology.orchestra.run.smithi071.stdout: "rados_mon_op_timeout": "0", 2024-09-11T06:49:18.856 INFO:teuthology.orchestra.run.smithi071.stdout: "rados_osd_op_timeout": "0", 2024-09-11T06:49:18.856 INFO:teuthology.orchestra.run.smithi071.stdout: "rados_tracing": "false", 2024-09-11T06:49:18.856 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_atime_update_interval": "60", 2024-09-11T06:49:18.857 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-11T06:49:18.857 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_balance_parent_reads": "false", 2024-09-11T06:49:18.857 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_balance_snap_reads": "false", 2024-09-11T06:49:18.857 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_blkin_trace_all": "false", 2024-09-11T06:49:18.857 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-11T06:49:18.857 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-11T06:49:18.857 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_cache": "true", 2024-09-11T06:49:18.857 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-11T06:49:18.857 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-11T06:49:18.857 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-11T06:49:18.857 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-11T06:49:18.857 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_cache_policy": "writearound", 2024-09-11T06:49:18.857 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_cache_size": "33554432", 2024-09-11T06:49:18.858 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-11T06:49:18.858 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-11T06:49:18.858 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_clone_copy_on_read": "false", 2024-09-11T06:49:18.858 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_compression_hint": "none", 2024-09-11T06:49:18.858 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_concurrent_management_ops": "10", 2024-09-11T06:49:18.858 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-11T06:49:18.858 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_default_clone_format": "auto", 2024-09-11T06:49:18.858 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_default_data_pool": "", 2024-09-11T06:49:18.858 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_default_features": "61", 2024-09-11T06:49:18.858 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_default_format": "2", 2024-09-11T06:49:18.858 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_default_map_options": "", 2024-09-11T06:49:18.858 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_default_order": "22", 2024-09-11T06:49:18.858 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_default_pool": "rbd", 2024-09-11T06:49:18.858 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-11T06:49:18.859 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_default_stripe_count": "0", 2024-09-11T06:49:18.859 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_default_stripe_unit": "0", 2024-09-11T06:49:18.859 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-11T06:49:18.859 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-11T06:49:18.859 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-11T06:49:18.859 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_enable_alloc_hint": "true", 2024-09-11T06:49:18.859 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-11T06:49:18.859 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_io_scheduler": "simple", 2024-09-11T06:49:18.859 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-11T06:49:18.859 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-11T06:49:18.859 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-11T06:49:18.859 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-11T06:49:18.859 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-11T06:49:18.859 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-11T06:49:18.860 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-11T06:49:18.860 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-11T06:49:18.860 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-11T06:49:18.860 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_journal_order": "24", 2024-09-11T06:49:18.860 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_journal_pool": "", 2024-09-11T06:49:18.860 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_journal_splay_width": "4", 2024-09-11T06:49:18.860 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_localize_parent_reads": "false", 2024-09-11T06:49:18.860 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_localize_snap_reads": "false", 2024-09-11T06:49:18.860 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-11T06:49:18.860 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-11T06:49:18.860 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-11T06:49:18.860 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-11T06:49:18.860 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-11T06:49:18.860 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-11T06:49:18.861 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-11T06:49:18.861 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-11T06:49:18.861 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-11T06:49:18.861 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-11T06:49:18.861 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-11T06:49:18.861 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-11T06:49:18.861 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-11T06:49:18.861 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-11T06:49:18.861 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-11T06:49:18.861 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-11T06:49:18.861 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-11T06:49:18.861 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-11T06:49:18.861 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-11T06:49:18.862 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-11T06:49:18.862 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-11T06:49:18.862 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-11T06:49:18.862 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-11T06:49:18.862 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-11T06:49:18.862 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-11T06:49:18.862 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-11T06:49:18.862 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-11T06:49:18.862 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-11T06:49:18.863 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-11T06:49:18.863 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-11T06:49:18.863 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-11T06:49:18.863 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mtime_update_interval": "60", 2024-09-11T06:49:18.863 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_non_blocking_aio": "true", 2024-09-11T06:49:18.863 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_op_thread_timeout": "60", 2024-09-11T06:49:18.863 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_op_threads": "1", 2024-09-11T06:49:18.863 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_parent_cache_enabled": "false", 2024-09-11T06:49:18.863 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-11T06:49:18.863 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-11T06:49:18.864 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-11T06:49:18.864 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_plugins": "", 2024-09-11T06:49:18.864 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_bps_burst": "0", 2024-09-11T06:49:18.864 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-11T06:49:18.864 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_bps_limit": "0", 2024-09-11T06:49:18.864 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_exclude_ops": "0", 2024-09-11T06:49:18.864 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_iops_burst": "0", 2024-09-11T06:49:18.864 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-11T06:49:18.864 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_iops_limit": "0", 2024-09-11T06:49:18.864 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-11T06:49:18.864 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-11T06:49:18.864 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-11T06:49:18.864 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-11T06:49:18.865 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-11T06:49:18.865 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-11T06:49:18.865 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-11T06:49:18.865 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-11T06:49:18.865 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-11T06:49:18.865 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-11T06:49:18.865 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-11T06:49:18.865 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-11T06:49:18.865 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-11T06:49:18.865 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-11T06:49:18.865 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_read_from_replica_policy": "default", 2024-09-11T06:49:18.865 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-11T06:49:18.865 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-11T06:49:18.865 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-11T06:49:18.866 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-11T06:49:18.866 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_skip_partial_discard": "true", 2024-09-11T06:49:18.866 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-11T06:49:18.866 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_tracing": "false", 2024-09-11T06:49:18.866 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_validate_names": "true", 2024-09-11T06:49:18.866 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_validate_pool": "true", 2024-09-11T06:49:18.866 INFO:teuthology.orchestra.run.smithi071.stdout: "restapi_base_url": "", 2024-09-11T06:49:18.866 INFO:teuthology.orchestra.run.smithi071.stdout: "restapi_log_level": "", 2024-09-11T06:49:18.866 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-11T06:49:18.866 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-11T06:49:18.866 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_acl_grants_max_num": "100", 2024-09-11T06:49:18.866 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_admin_entry": "admin", 2024-09-11T06:49:18.866 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-11T06:49:18.866 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_backend_store": "rados", 2024-09-11T06:49:18.867 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_barbican_url": "", 2024-09-11T06:49:18.867 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_beast_enable_async": "true", 2024-09-11T06:49:18.867 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_bucket_counters_cache": "false", 2024-09-11T06:49:18.867 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-11T06:49:18.867 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-11T06:49:18.867 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-11T06:49:18.867 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-11T06:49:18.867 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-11T06:49:18.867 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-11T06:49:18.867 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-11T06:49:18.867 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-11T06:49:18.867 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_cache_enabled": "true", 2024-09-11T06:49:18.868 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_cache_expiry_interval": "900", 2024-09-11T06:49:18.868 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_cache_lru_size": "25000", 2024-09-11T06:49:18.868 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_config_store": "rados", 2024-09-11T06:49:18.868 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_content_length_compat": "false", 2024-09-11T06:49:18.868 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_copy_obj_progress": "true", 2024-09-11T06:49:18.868 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-11T06:49:18.868 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_cors_rules_max_num": "100", 2024-09-11T06:49:18.868 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_cross_domain_policy": "", 2024-09-11T06:49:18.868 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-11T06:49:18.868 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_kmip_addr": "", 2024-09-11T06:49:18.868 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-11T06:49:18.868 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-11T06:49:18.868 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-11T06:49:18.869 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-11T06:49:18.869 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_kmip_password": "", 2024-09-11T06:49:18.869 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-11T06:49:18.869 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_kmip_username": "", 2024-09-11T06:49:18.869 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_require_ssl": "true", 2024-09-11T06:49:18.869 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-11T06:49:18.869 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-11T06:49:18.869 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-11T06:49:18.869 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-11T06:49:18.869 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-11T06:49:18.870 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-11T06:49:18.870 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-11T06:49:18.870 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-11T06:49:18.870 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-11T06:49:18.870 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-11T06:49:18.870 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-11T06:49:18.870 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-11T06:49:18.870 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-11T06:49:18.870 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-11T06:49:18.870 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-11T06:49:18.870 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_vault_addr": "", 2024-09-11T06:49:18.870 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_vault_auth": "token", 2024-09-11T06:49:18.870 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_vault_namespace": "", 2024-09-11T06:49:18.871 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_vault_prefix": "", 2024-09-11T06:49:18.871 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-11T06:49:18.871 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-11T06:49:18.871 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-11T06:49:18.871 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-11T06:49:18.871 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_vault_token_file": "", 2024-09-11T06:49:18.871 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-11T06:49:18.871 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_curl_buffersize": "524288", 2024-09-11T06:49:18.871 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-11T06:49:18.871 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_curl_low_speed_time": "300", 2024-09-11T06:49:18.871 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-11T06:49:18.871 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-11T06:49:18.872 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-11T06:49:18.872 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-11T06:49:18.872 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-11T06:49:18.872 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-11T06:49:18.872 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-11T06:49:18.872 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-11T06:49:18.872 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-11T06:49:18.872 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-11T06:49:18.873 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-11T06:49:18.873 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_d4n_port": "6379", 2024-09-11T06:49:18.873 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-11T06:49:18.873 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_data_log_changes_size": "1000", 2024-09-11T06:49:18.873 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_data_log_num_shards": "128", 2024-09-11T06:49:18.873 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-11T06:49:18.873 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_data_log_window": "30", 2024-09-11T06:49:18.873 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-11T06:49:18.873 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-11T06:49:18.873 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-11T06:49:18.873 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-11T06:49:18.873 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-11T06:49:18.873 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-11T06:49:18.873 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-11T06:49:18.874 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-11T06:49:18.874 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-11T06:49:18.874 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-11T06:49:18.874 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-11T06:49:18.874 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-11T06:49:18.874 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-11T06:49:18.874 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-11T06:49:18.874 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-11T06:49:18.874 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-11T06:49:18.874 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-11T06:49:18.874 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-11T06:49:18.874 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-11T06:49:18.874 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-11T06:49:18.874 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-11T06:49:18.875 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-11T06:49:18.875 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-11T06:49:18.875 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-11T06:49:18.875 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-11T06:49:18.875 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dns_name": "", 2024-09-11T06:49:18.875 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dns_s3website_name": "", 2024-09-11T06:49:18.875 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dynamic_resharding": "true", 2024-09-11T06:49:18.875 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-11T06:49:18.875 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_enable_gc_threads": "true", 2024-09-11T06:49:18.875 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_enable_lc_threads": "true", 2024-09-11T06:49:18.876 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_enable_ops_log": "false", 2024-09-11T06:49:18.876 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_enable_quota_threads": "true", 2024-09-11T06:49:18.876 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_enable_static_website": "false", 2024-09-11T06:49:18.876 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_enable_usage_log": "false", 2024-09-11T06:49:18.876 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_enforce_swift_acls": "true", 2024-09-11T06:49:18.876 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_exit_timeout_secs": "120", 2024-09-11T06:49:18.876 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_expose_bucket": "false", 2024-09-11T06:49:18.876 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_extended_http_attrs": "", 2024-09-11T06:49:18.876 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_filter": "none", 2024-09-11T06:49:18.876 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-11T06:49:18.876 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_frontends": "beast port=7480", 2024-09-11T06:49:18.876 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-11T06:49:18.876 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_gc_max_deferred": "50", 2024-09-11T06:49:18.876 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-11T06:49:18.877 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_gc_max_objs": "32", 2024-09-11T06:49:18.877 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-11T06:49:18.877 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-11T06:49:18.877 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-11T06:49:18.877 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-11T06:49:18.877 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_gc_processor_period": "3600", 2024-09-11T06:49:18.877 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-11T06:49:18.877 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-11T06:49:18.877 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-11T06:49:18.877 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-11T06:49:18.877 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_init_timeout": "300", 2024-09-11T06:49:18.877 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-11T06:49:18.877 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-11T06:49:18.878 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_kafka_connection_idle": "300", 2024-09-11T06:49:18.878 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-11T06:49:18.878 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-11T06:49:18.878 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-11T06:49:18.878 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-11T06:49:18.878 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-11T06:49:18.878 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_admin_domain": "", 2024-09-11T06:49:18.878 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_admin_password": "", 2024-09-11T06:49:18.878 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_admin_password_path": "", 2024-09-11T06:49:18.878 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_admin_project": "", 2024-09-11T06:49:18.878 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_admin_tenant": "", 2024-09-11T06:49:18.878 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_admin_token": "", 2024-09-11T06:49:18.878 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_admin_token_path": "", 2024-09-11T06:49:18.879 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_admin_user": "", 2024-09-11T06:49:18.879 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_api_version": "2", 2024-09-11T06:49:18.879 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_barbican_domain": "", 2024-09-11T06:49:18.879 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_barbican_password": "", 2024-09-11T06:49:18.879 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_barbican_project": "", 2024-09-11T06:49:18.879 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-11T06:49:18.879 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_barbican_user": "", 2024-09-11T06:49:18.879 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-11T06:49:18.879 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-11T06:49:18.879 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-11T06:49:18.879 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-11T06:49:18.879 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-11T06:49:18.879 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_url": "", 2024-09-11T06:49:18.879 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-11T06:49:18.879 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_lc_debug_interval": "-1", 2024-09-11T06:49:18.880 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_lc_lock_max_time": "90", 2024-09-11T06:49:18.880 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_lc_max_objs": "32", 2024-09-11T06:49:18.880 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_lc_max_rules": "1000", 2024-09-11T06:49:18.880 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_lc_max_worker": "3", 2024-09-11T06:49:18.880 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-11T06:49:18.880 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_lc_thread_delay": "0", 2024-09-11T06:49:18.880 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-11T06:49:18.880 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_ldap_dnattr": "uid", 2024-09-11T06:49:18.880 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-11T06:49:18.880 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_ldap_searchfilter": "", 2024-09-11T06:49:18.880 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-11T06:49:18.880 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_ldap_uri": "", 2024-09-11T06:49:18.880 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-11T06:49:18.881 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-11T06:49:18.881 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-11T06:49:18.881 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_log_http_headers": "", 2024-09-11T06:49:18.881 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-11T06:49:18.881 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-11T06:49:18.881 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_log_object_name_utc": "false", 2024-09-11T06:49:18.881 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-11T06:49:18.881 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-11T06:49:18.881 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_max_attr_name_len": "0", 2024-09-11T06:49:18.881 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_max_attr_size": "0", 2024-09-11T06:49:18.881 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-11T06:49:18.881 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_max_chunk_size": "4194304", 2024-09-11T06:49:18.881 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-11T06:49:18.881 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-11T06:49:18.882 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-11T06:49:18.882 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_max_listing_results": "1000", 2024-09-11T06:49:18.882 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_max_notify_retries": "10", 2024-09-11T06:49:18.882 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-11T06:49:18.882 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_max_put_param_size": "1048576", 2024-09-11T06:49:18.882 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_max_put_size": "5368709120", 2024-09-11T06:49:18.882 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_max_slo_entries": "1000", 2024-09-11T06:49:18.882 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_md_log_max_shards": "64", 2024-09-11T06:49:18.882 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-11T06:49:18.882 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-11T06:49:18.882 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-11T06:49:18.882 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-11T06:49:18.882 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_mp_lock_max_time": "600", 2024-09-11T06:49:18.883 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-11T06:49:18.883 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-11T06:49:18.883 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-11T06:49:18.883 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-11T06:49:18.883 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-11T06:49:18.883 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-11T06:49:18.883 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-11T06:49:18.883 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-11T06:49:18.883 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_nfs_max_gc": "300", 2024-09-11T06:49:18.884 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-11T06:49:18.884 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-11T06:49:18.884 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-11T06:49:18.884 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-11T06:49:18.884 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-11T06:49:18.884 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-11T06:49:18.884 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-11T06:49:18.884 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_num_async_rados_threads": "32", 2024-09-11T06:49:18.884 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_num_control_oids": "8", 2024-09-11T06:49:18.884 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_numa_node": "-1", 2024-09-11T06:49:18.884 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-11T06:49:18.884 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-11T06:49:18.884 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_objexp_chunk_size": "100", 2024-09-11T06:49:18.884 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_objexp_gc_interval": "600", 2024-09-11T06:49:18.885 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-11T06:49:18.885 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-11T06:49:18.885 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-11T06:49:18.885 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_op_thread_timeout": "600", 2024-09-11T06:49:18.885 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_op_tracing": "false", 2024-09-11T06:49:18.885 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_opa_token": "", 2024-09-11T06:49:18.885 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_opa_url": "", 2024-09-11T06:49:18.885 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_opa_verify_ssl": "true", 2024-09-11T06:49:18.885 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-11T06:49:18.885 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-11T06:49:18.885 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_ops_log_rados": "false", 2024-09-11T06:49:18.885 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_ops_log_socket_path": "", 2024-09-11T06:49:18.885 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-11T06:49:18.885 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-11T06:49:18.886 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-11T06:49:18.886 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_period_push_interval": "2.000000", 2024-09-11T06:49:18.886 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-11T06:49:18.886 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-11T06:49:18.886 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-11T06:49:18.886 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-11T06:49:18.886 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_posix_cache_lanes": "3", 2024-09-11T06:49:18.886 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-11T06:49:18.886 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-11T06:49:18.886 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_posix_cache_partitions": "3", 2024-09-11T06:49:18.886 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-11T06:49:18.886 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_print_continue": "true", 2024-09-11T06:49:18.886 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-11T06:49:18.887 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-11T06:49:18.887 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-11T06:49:18.887 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-11T06:49:18.887 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-11T06:49:18.887 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_rados_tracing": "false", 2024-09-11T06:49:18.887 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_realm": "", 2024-09-11T06:49:18.887 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_realm_id": "", 2024-09-11T06:49:18.887 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-11T06:49:18.887 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_region": "", 2024-09-11T06:49:18.887 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-11T06:49:18.887 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-11T06:49:18.887 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-11T06:49:18.887 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-11T06:49:18.887 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_request_uri": "", 2024-09-11T06:49:18.888 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_reshard_batch_size": "64", 2024-09-11T06:49:18.888 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-11T06:49:18.888 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_reshard_max_aio": "128", 2024-09-11T06:49:18.888 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_reshard_num_logs": "16", 2024-09-11T06:49:18.888 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_reshard_thread_interval": "600", 2024-09-11T06:49:18.888 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_resolve_cname": "false", 2024-09-11T06:49:18.888 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-11T06:49:18.888 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_run_sync_thread": "true", 2024-09-11T06:49:18.888 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-11T06:49:18.888 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-11T06:49:18.888 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-11T06:49:18.888 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-11T06:49:18.888 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-11T06:49:18.888 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-11T06:49:18.889 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-11T06:49:18.889 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-11T06:49:18.889 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-11T06:49:18.889 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_scheduler_type": "throttler", 2024-09-11T06:49:18.889 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_script_uri": "", 2024-09-11T06:49:18.889 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_service_provider_name": "", 2024-09-11T06:49:18.889 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-11T06:49:18.889 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sts_client_id": "", 2024-09-11T06:49:18.889 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sts_client_secret": "", 2024-09-11T06:49:18.889 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sts_entry": "sts", 2024-09-11T06:49:18.889 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sts_key": "sts", 2024-09-11T06:49:18.889 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-11T06:49:18.889 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sts_min_session_duration": "900", 2024-09-11T06:49:18.889 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sts_token_introspection_url": "", 2024-09-11T06:49:18.890 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_swift_account_in_url": "false", 2024-09-11T06:49:18.890 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_swift_auth_entry": "auth", 2024-09-11T06:49:18.890 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_swift_auth_url": "", 2024-09-11T06:49:18.890 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_swift_custom_header": "", 2024-09-11T06:49:18.890 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-11T06:49:18.890 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_swift_need_stats": "true", 2024-09-11T06:49:18.890 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_swift_tenant_name": "", 2024-09-11T06:49:18.890 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_swift_token_expiration": "86400", 2024-09-11T06:49:18.890 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_swift_url": "", 2024-09-11T06:49:18.890 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_swift_url_prefix": "swift", 2024-09-11T06:49:18.890 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-11T06:49:18.890 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-11T06:49:18.890 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-11T06:49:18.890 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sync_lease_period": "120", 2024-09-11T06:49:18.891 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-11T06:49:18.891 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-11T06:49:18.891 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-11T06:49:18.891 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-11T06:49:18.891 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-11T06:49:18.891 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-11T06:49:18.891 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-11T06:49:18.891 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-11T06:49:18.891 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-11T06:49:18.891 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_thread_pool_size": "512", 2024-09-11T06:49:18.891 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-11T06:49:18.891 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-11T06:49:18.891 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-11T06:49:18.891 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-11T06:49:18.892 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_torrent_comment": "", 2024-09-11T06:49:18.892 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_torrent_createby": "", 2024-09-11T06:49:18.892 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_torrent_encoding": "", 2024-09-11T06:49:18.892 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_torrent_flag": "false", 2024-09-11T06:49:18.892 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-11T06:49:18.892 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_torrent_origin": "", 2024-09-11T06:49:18.892 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-11T06:49:18.892 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_torrent_tracker": "", 2024-09-11T06:49:18.892 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_trust_forwarded_https": "false", 2024-09-11T06:49:18.892 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-11T06:49:18.892 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-11T06:49:18.892 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_usage_max_shards": "32", 2024-09-11T06:49:18.892 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_usage_max_user_shards": "1", 2024-09-11T06:49:18.892 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_use_opa_authz": "false", 2024-09-11T06:49:18.893 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_user_counters_cache": "false", 2024-09-11T06:49:18.893 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-11T06:49:18.893 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-11T06:49:18.893 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-11T06:49:18.893 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_user_max_buckets": "1000", 2024-09-11T06:49:18.893 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_user_policies_max_num": "100", 2024-09-11T06:49:18.893 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-11T06:49:18.893 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-11T06:49:18.893 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-11T06:49:18.893 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-11T06:49:18.894 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_user_unique_email": "true", 2024-09-11T06:49:18.894 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_verify_ssl": "true", 2024-09-11T06:49:18.894 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-11T06:49:18.894 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_zone": "", 2024-09-11T06:49:18.894 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_zone_id": "", 2024-09-11T06:49:18.894 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-11T06:49:18.894 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_zonegroup": "", 2024-09-11T06:49:18.894 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_zonegroup_id": "", 2024-09-11T06:49:18.894 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-11T06:49:18.894 INFO:teuthology.orchestra.run.smithi071.stdout: "rgwlc_auto_session_clear": "true", 2024-09-11T06:49:18.894 INFO:teuthology.orchestra.run.smithi071.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-11T06:49:18.894 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_block_size": "4096", 2024-09-11T06:49:18.895 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-11T06:49:18.895 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-11T06:49:18.895 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-11T06:49:18.895 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-11T06:49:18.895 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-11T06:49:18.895 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_cache_size": "536870912", 2024-09-11T06:49:18.895 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-11T06:49:18.895 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-11T06:49:18.895 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-11T06:49:18.895 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-11T06:49:18.895 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-11T06:49:18.895 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-11T06:49:18.895 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-11T06:49:18.895 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-11T06:49:18.896 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_index_type": "binary_search", 2024-09-11T06:49:18.896 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-11T06:49:18.896 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-11T06:49:18.896 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_partition_filters": "false", 2024-09-11T06:49:18.896 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_perf": "false", 2024-09-11T06:49:18.896 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-11T06:49:18.896 INFO:teuthology.orchestra.run.smithi071.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-11T06:49:18.896 INFO:teuthology.orchestra.run.smithi071.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-11T06:49:18.896 INFO:teuthology.orchestra.run.smithi071.stdout: "run_dir": "/var/run/ceph", 2024-09-11T06:49:18.896 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_block_create": "true", 2024-09-11T06:49:18.896 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_cache_lru_size": "67108864", 2024-09-11T06:49:18.896 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-11T06:49:18.896 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-11T06:49:18.896 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_default_max_object_size": "16777216", 2024-09-11T06:49:18.897 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-11T06:49:18.897 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_device_size": "53687091200", 2024-09-11T06:49:18.897 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-11T06:49:18.897 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_full_integrity_check": "false", 2024-09-11T06:49:18.897 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_journal_batch_capacity": "16", 2024-09-11T06:49:18.897 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-11T06:49:18.897 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-11T06:49:18.897 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-11T06:49:18.897 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_main_device_type": "SSD", 2024-09-11T06:49:18.897 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-11T06:49:18.897 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-11T06:49:18.897 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-11T06:49:18.897 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-11T06:49:18.897 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-11T06:49:18.898 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-11T06:49:18.898 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_segment_size": "67108864", 2024-09-11T06:49:18.898 INFO:teuthology.orchestra.run.smithi071.stdout: "set_keepcaps": "false", 2024-09-11T06:49:18.898 INFO:teuthology.orchestra.run.smithi071.stdout: "setgroup": "", 2024-09-11T06:49:18.898 INFO:teuthology.orchestra.run.smithi071.stdout: "setuser": "", 2024-09-11T06:49:18.898 INFO:teuthology.orchestra.run.smithi071.stdout: "setuser_match_path": "", 2024-09-11T06:49:18.898 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-11T06:49:18.898 INFO:teuthology.orchestra.run.smithi071.stdout: "thp": "false", 2024-09-11T06:49:18.898 INFO:teuthology.orchestra.run.smithi071.stdout: "threadpool_default_timeout": "60", 2024-09-11T06:49:18.898 INFO:teuthology.orchestra.run.smithi071.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-11T06:49:18.898 INFO:teuthology.orchestra.run.smithi071.stdout: "throttler_perf_counter": "true" 2024-09-11T06:49:18.898 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:18.901 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-11T06:49:19.036 INFO:teuthology.orchestra.run.smithi071.stdout:mon.a: {} 2024-09-11T06:49:19.037 INFO:teuthology.orchestra.run.smithi071.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T06:49:19.048 INFO:teuthology.orchestra.run.smithi071.stdout:mon.b: {} 2024-09-11T06:49:19.049 INFO:teuthology.orchestra.run.smithi071.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T06:49:19.057 INFO:teuthology.orchestra.run.smithi071.stdout:mon.c: {} 2024-09-11T06:49:19.057 INFO:teuthology.orchestra.run.smithi071.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T06:49:19.072 DEBUG:teuthology.orchestra.run.smithi089:> type ceph-objectstore-tool 2024-09-11T06:49:19.081 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-11T06:49:19.081 DEBUG:teuthology.orchestra.run.smithi071:> type ceph-objectstore-tool 2024-09-11T06:49:19.086 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-11T06:49:19.086 INFO:teuthology.run_tasks:Running task workunit... 2024-09-11T06:49:19.105 INFO:tasks.workunit:Pulling workunits from ref daaf78de334647fc0c800f4ac495cc9475a26bae 2024-09-11T06:49:19.105 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-11T06:49:19.106 DEBUG:teuthology.orchestra.run.smithi176:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-11T06:49:19.107 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-11T06:49:19.107 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T06:49:19.107 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T06:49:19.107 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [0, 1, 2, 3, 4, 5] 2024-09-11T06:49:19.108 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-11T06:49:19.108 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-11T06:49:19.109 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-11T06:49:19.109 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:49:19.109 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-11T06:49:19.109 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:49:19.109 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-11T06:49:19.110 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T06:49:19.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T06:49:19.116 INFO:teuthology.orchestra.run.smithi176.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-11T06:49:19.116 DEBUG:teuthology.orchestra.run.smithi176:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-09-11T06:49:19.123 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:19.122+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:19.162 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-09-11T06:49:19.163 DEBUG:teuthology.orchestra.run.smithi176:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-09-11T06:49:19.211 INFO:tasks.workunit:timeout=3h 2024-09-11T06:49:19.211 INFO:tasks.workunit:cleanup=True 2024-09-11T06:49:19.211 DEBUG:teuthology.orchestra.run.smithi176:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout daaf78de334647fc0c800f4ac495cc9475a26bae 2024-09-11T06:49:19.212 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:19.206+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:19.241 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:19.241 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:19.241 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:49:19.241 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:19.257 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:49:19.260 INFO:tasks.workunit.client.0.smithi176.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-11T06:49:19.264 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:49:19.264 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:49:19.294 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:49:19.295 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:49:19.312 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:19.310+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:19.324 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:49:19.324 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:49:19.358 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:49:19.358 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:49:19.372 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:19.372 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:19.386 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:49:19.392 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:49:19.393 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:49:19.413 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:19.414+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:19.426 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:49:19.426 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:49:19.499 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:19.499 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:19.506 INFO:tasks.ceph.mon.a.smithi071.stderr:2024-09-11T06:49:19.502+0000 7f1296554640 -1 mon.a@0(leader).osd e19 definitely_dead 0 2024-09-11T06:49:19.511 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:49:19.513 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:19.510+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:19.614 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:19.610+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:19.621 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:19.621 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:19.634 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:49:19.715 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:19.714+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:19.786 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:19.786 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:19.799 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:49:19.816 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:19.814+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:19.917 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:19.917 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:19.918 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:19.914+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:19.928 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:49:20.019 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:20.018+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:20.046 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:20.046 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:20.058 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:49:20.119 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:20.118+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:20.177 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:20.177 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:20.191 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:49:20.220 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:20.218+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:20.270 INFO:teuthology.orchestra.run.smithi071.stderr:marked out osd.1. 2024-09-11T06:49:20.270 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is set 2024-09-11T06:49:20.321 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:20.318+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:20.367 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:20.367 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:20.381 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:49:20.429 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:20.426+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:20.512 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:20.512 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:20.523 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:20.522+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:20.526 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:49:20.624 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:20.622+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:20.692 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:20.692 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:20.705 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:49:20.751 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:20.750+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:20.821 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:20.821 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:20.826 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:20.822+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:20.838 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:49:20.927 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:20.926+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:21.006 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:21.006 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:21.021 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:49:21.063 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:21.062+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:21.128 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:21.126+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:21.186 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:21.186 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:21.200 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:49:21.245 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:21.242+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:21.329 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:21.326+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:21.370 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:21.370 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:21.384 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:49:21.431 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:21.430+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:21.448 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:49:21.506 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:21.506 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:21.520 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:49:21.563 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:21.562+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:21.601 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:49:21.601 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:49:21.617 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:49:21.617 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:49:21.631 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:21.630+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:21.633 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:49:21.633 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:49:21.656 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:49:21.657 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:49:21.673 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:49:21.673 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:49:21.686 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:21.687 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:21.694 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:49:21.694 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:49:21.701 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:49:21.751 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:21.750+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:21.820 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:21.820 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:49:21.821 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:49:21.821 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:49:21.821 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.821 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:49:21.821 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:49:21.821 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 10.590924255999999, 2024-09-11T06:49:21.821 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010553425999999999, 2024-09-11T06:49:21.821 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:21.821 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:21.821 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:21.821 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:21.821 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:49:21.821 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:49:21.822 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:49:21.822 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.822 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:21.822 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.822 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:21.822 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:49:21.822 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:21.822 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.822 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.822 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:21.822 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:49:21.822 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:21.822 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.822 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.823 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:21.823 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230274+0000", 2024-09-11T06:49:21.823 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.533e-06 2024-09-11T06:49:21.823 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.823 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.823 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:21.823 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230285+0000", 2024-09-11T06:49:21.823 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0540000000000001e-05 2024-09-11T06:49:21.823 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.823 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.823 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:21.823 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230309+0000", 2024-09-11T06:49:21.823 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4223000000000002e-05 2024-09-11T06:49:21.824 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.824 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.824 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:21.824 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230327+0000", 2024-09-11T06:49:21.824 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:49:21.824 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.824 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.824 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:21.824 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230478+0000", 2024-09-11T06:49:21.824 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000151459 2024-09-11T06:49:21.824 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.824 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.824 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:49:21.825 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230490+0000", 2024-09-11T06:49:21.825 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.147e-05 2024-09-11T06:49:21.825 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.825 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.825 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:21.825 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231803+0000", 2024-09-11T06:49:21.825 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001313158 2024-09-11T06:49:21.825 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.825 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.825 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:49:21.825 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231817+0000", 2024-09-11T06:49:21.825 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4197e-05 2024-09-11T06:49:21.825 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.825 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.826 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:21.826 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.235945+0000", 2024-09-11T06:49:21.826 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0041282480000000002 2024-09-11T06:49:21.826 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.826 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.826 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:21.826 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236312+0000", 2024-09-11T06:49:21.826 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036652700000000001 2024-09-11T06:49:21.826 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.826 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.826 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:49:21.826 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236689+0000", 2024-09-11T06:49:21.826 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000377117 2024-09-11T06:49:21.826 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.827 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.827 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:21.827 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238460+0000", 2024-09-11T06:49:21.827 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001770918 2024-09-11T06:49:21.827 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.827 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.827 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:21.827 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240627+0000", 2024-09-11T06:49:21.827 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021670779999999998 2024-09-11T06:49:21.827 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.827 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.827 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:21.827 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240751+0000", 2024-09-11T06:49:21.828 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012384900000000001 2024-09-11T06:49:21.828 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.828 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.828 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:21.828 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240821+0000", 2024-09-11T06:49:21.828 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0516e-05 2024-09-11T06:49:21.828 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:21.828 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:21.828 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:21.828 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.828 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.828 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop_reply(client.4230.0:1 2.3 e16/15 ondisk, result = 0)", 2024-09-11T06:49:21.828 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.238282+0000", 2024-09-11T06:49:21.829 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 10.582909931, 2024-09-11T06:49:21.829 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018970799999999999, 2024-09-11T06:49:21.829 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:21.829 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:21.829 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:49:21.829 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:21.829 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.829 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:21.829 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238282+0000", 2024-09-11T06:49:21.829 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:21.829 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.829 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.829 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:21.829 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238282+0000", 2024-09-11T06:49:21.830 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:21.830 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.830 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.830 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:21.830 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238284+0000", 2024-09-11T06:49:21.830 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:49:21.830 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.830 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.830 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:21.830 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238292+0000", 2024-09-11T06:49:21.830 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2390000000000002e-06 2024-09-11T06:49:21.830 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.830 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.831 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:21.831 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238314+0000", 2024-09-11T06:49:21.831 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1776999999999998e-05 2024-09-11T06:49:21.831 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.831 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.831 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:21.831 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238339+0000", 2024-09-11T06:49:21.831 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5119000000000001e-05 2024-09-11T06:49:21.831 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.831 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.831 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:21.831 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238438+0000", 2024-09-11T06:49:21.831 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9553e-05 2024-09-11T06:49:21.832 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.832 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.832 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:21.832 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238451+0000", 2024-09-11T06:49:21.832 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2286000000000001e-05 2024-09-11T06:49:21.832 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.832 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.832 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:21.832 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238472+0000", 2024-09-11T06:49:21.832 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1166999999999999e-05 2024-09-11T06:49:21.832 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:21.832 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:21.833 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:21.833 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.833 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.833 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:49:21.833 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.244627+0000", 2024-09-11T06:49:21.833 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 10.576565233, 2024-09-11T06:49:21.833 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00046141999999999998, 2024-09-11T06:49:21.833 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:21.833 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:21.833 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:49:21.833 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:21.833 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:49:21.833 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:49:21.833 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 2 2024-09-11T06:49:21.834 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.834 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:21.834 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.834 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:21.834 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244627+0000", 2024-09-11T06:49:21.834 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:21.834 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.834 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.834 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:21.834 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244627+0000", 2024-09-11T06:49:21.834 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:21.834 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.834 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.834 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:21.835 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244629+0000", 2024-09-11T06:49:21.835 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.95e-06 2024-09-11T06:49:21.835 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.835 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.835 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:21.835 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244640+0000", 2024-09-11T06:49:21.835 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1494e-05 2024-09-11T06:49:21.835 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.835 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.835 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:21.835 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244664+0000", 2024-09-11T06:49:21.835 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4006e-05 2024-09-11T06:49:21.835 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.836 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.836 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:21.836 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244684+0000", 2024-09-11T06:49:21.836 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.961e-05 2024-09-11T06:49:21.836 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.836 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.836 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:21.836 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244862+0000", 2024-09-11T06:49:21.836 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017863500000000001 2024-09-11T06:49:21.836 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.836 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.836 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:21.836 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245088+0000", 2024-09-11T06:49:21.837 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000225725 2024-09-11T06:49:21.837 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:21.837 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:21.837 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:21.837 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.837 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.837 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:49:21.837 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.245492+0000", 2024-09-11T06:49:21.837 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 10.57570029, 2024-09-11T06:49:21.837 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00042092600000000001, 2024-09-11T06:49:21.837 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:21.837 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:21.837 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:49:21.837 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:21.838 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:49:21.838 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:49:21.838 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 3 2024-09-11T06:49:21.838 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.838 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:21.838 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.838 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:21.838 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245492+0000", 2024-09-11T06:49:21.838 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:21.838 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.839 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.839 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:21.839 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245492+0000", 2024-09-11T06:49:21.839 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:21.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.839 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.839 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:21.839 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245493+0000", 2024-09-11T06:49:21.839 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.657e-06 2024-09-11T06:49:21.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.839 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.839 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:21.840 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245501+0000", 2024-09-11T06:49:21.840 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2490000000000004e-06 2024-09-11T06:49:21.840 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.840 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.840 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:21.840 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245523+0000", 2024-09-11T06:49:21.840 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2016999999999999e-05 2024-09-11T06:49:21.840 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.840 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.840 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:21.840 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245548+0000", 2024-09-11T06:49:21.840 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5060000000000001e-05 2024-09-11T06:49:21.840 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.840 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.841 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:21.841 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245720+0000", 2024-09-11T06:49:21.841 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017190200000000001 2024-09-11T06:49:21.841 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.841 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.841 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:21.841 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245913+0000", 2024-09-11T06:49:21.841 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193041 2024-09-11T06:49:21.841 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:21.841 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:21.841 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:21.841 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.841 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-11T06:49:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:49:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 9.5887534129999992, 2024-09-11T06:49:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069904770000000001, 2024-09-11T06:49:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:49:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:49:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 5 2024-09-11T06:49:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:21.843 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:49:21.843 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:21.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.843 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.843 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:21.843 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:49:21.843 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:21.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.843 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.843 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:21.843 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232442+0000", 2024-09-11T06:49:21.843 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.044e-06 2024-09-11T06:49:21.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.844 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.844 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:21.844 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232460+0000", 2024-09-11T06:49:21.844 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7863000000000002e-05 2024-09-11T06:49:21.844 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.844 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.844 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:21.844 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232492+0000", 2024-09-11T06:49:21.844 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2812999999999999e-05 2024-09-11T06:49:21.844 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.844 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.844 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:21.844 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232510+0000", 2024-09-11T06:49:21.845 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7552999999999999e-05 2024-09-11T06:49:21.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.845 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.845 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:21.845 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232640+0000", 2024-09-11T06:49:21.845 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013029900000000001 2024-09-11T06:49:21.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.845 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.845 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:49:21.845 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232650+0000", 2024-09-11T06:49:21.845 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0023999999999999e-05 2024-09-11T06:49:21.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.845 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.846 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:21.846 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235641+0000", 2024-09-11T06:49:21.846 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00299089 2024-09-11T06:49:21.846 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.846 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.846 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:21.846 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235980+0000", 2024-09-11T06:49:21.846 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000338907 2024-09-11T06:49:21.846 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.846 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.846 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:49:21.846 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.236351+0000", 2024-09-11T06:49:21.846 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037087699999999998 2024-09-11T06:49:21.846 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.846 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.847 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:21.847 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238074+0000", 2024-09-11T06:49:21.847 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001722859 2024-09-11T06:49:21.847 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.847 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.847 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:21.847 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239160+0000", 2024-09-11T06:49:21.847 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001086074 2024-09-11T06:49:21.847 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.847 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.847 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:21.847 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239299+0000", 2024-09-11T06:49:21.847 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013933500000000001 2024-09-11T06:49:21.848 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.848 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.848 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:21.848 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239429+0000", 2024-09-11T06:49:21.848 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129939 2024-09-11T06:49:21.848 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:21.848 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:21.848 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:21.848 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.848 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.848 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop_reply(client.4230.0:5 2.2 e16/15 ondisk, result = 0)", 2024-09-11T06:49:21.848 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:12.238911+0000", 2024-09-11T06:49:21.848 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 9.5822811249999997, 2024-09-11T06:49:21.848 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00053624900000000004, 2024-09-11T06:49:21.849 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:21.849 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:21.849 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:49:21.849 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:21.849 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.849 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:21.849 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238911+0000", 2024-09-11T06:49:21.849 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:21.849 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.849 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.849 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:21.849 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238911+0000", 2024-09-11T06:49:21.850 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:21.850 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.850 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.850 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:21.850 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238913+0000", 2024-09-11T06:49:21.850 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2069999999999998e-06 2024-09-11T06:49:21.850 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.850 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.850 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:21.850 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238923+0000", 2024-09-11T06:49:21.850 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0360000000000001e-05 2024-09-11T06:49:21.850 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.851 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.851 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:21.851 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238953+0000", 2024-09-11T06:49:21.851 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9258999999999999e-05 2024-09-11T06:49:21.851 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.851 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.851 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:21.851 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238967+0000", 2024-09-11T06:49:21.851 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3797e-05 2024-09-11T06:49:21.851 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.851 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.852 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:21.852 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239117+0000", 2024-09-11T06:49:21.852 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000150765 2024-09-11T06:49:21.852 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.852 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.852 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:21.852 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239143+0000", 2024-09-11T06:49:21.852 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5514e-05 2024-09-11T06:49:21.852 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.852 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.852 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:21.852 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239447+0000", 2024-09-11T06:49:21.852 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00030434700000000001 2024-09-11T06:49:21.853 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:21.853 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:21.853 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:21.853 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.853 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.853 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-11T06:49:21.853 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:49:21.853 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 8.5746654610000004, 2024-09-11T06:49:21.853 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069253250000000004, 2024-09-11T06:49:21.853 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:21.853 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:21.853 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:21.853 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:21.854 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:49:21.854 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:49:21.854 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 7 2024-09-11T06:49:21.854 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.854 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:21.854 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.854 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:21.854 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:49:21.854 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:21.854 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.854 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.854 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:21.854 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:49:21.854 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:21.855 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.855 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.855 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:21.855 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246530+0000", 2024-09-11T06:49:21.855 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9469999999999999e-06 2024-09-11T06:49:21.855 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.855 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.855 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:21.855 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246545+0000", 2024-09-11T06:49:21.855 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5973000000000001e-05 2024-09-11T06:49:21.855 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.855 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.855 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:21.855 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246578+0000", 2024-09-11T06:49:21.856 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2759999999999998e-05 2024-09-11T06:49:21.856 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.856 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.856 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:21.856 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246595+0000", 2024-09-11T06:49:21.856 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6939000000000002e-05 2024-09-11T06:49:21.856 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.856 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.856 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:21.856 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246770+0000", 2024-09-11T06:49:21.856 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000174723 2024-09-11T06:49:21.856 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.856 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.856 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:49:21.857 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246789+0000", 2024-09-11T06:49:21.857 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9159000000000001e-05 2024-09-11T06:49:21.857 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.857 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.857 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:21.857 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250337+0000", 2024-09-11T06:49:21.857 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035480759999999998 2024-09-11T06:49:21.857 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.857 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.857 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:21.857 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250549+0000", 2024-09-11T06:49:21.858 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021192199999999999 2024-09-11T06:49:21.858 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.858 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.858 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:49:21.858 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250750+0000", 2024-09-11T06:49:21.858 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020104199999999999 2024-09-11T06:49:21.858 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.858 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.858 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:21.858 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.252455+0000", 2024-09-11T06:49:21.858 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001705249 2024-09-11T06:49:21.858 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.859 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.859 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:21.859 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253305+0000", 2024-09-11T06:49:21.859 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00084941200000000002 2024-09-11T06:49:21.859 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.859 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.859 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:21.859 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253388+0000", 2024-09-11T06:49:21.859 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3642999999999999e-05 2024-09-11T06:49:21.859 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.859 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.859 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:21.859 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253452+0000", 2024-09-11T06:49:21.860 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3479999999999995e-05 2024-09-11T06:49:21.860 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:21.860 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:21.860 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:21.860 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.860 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.860 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop_reply(client.4230.0:7 2.2 e18/15 ondisk, result = 0)", 2024-09-11T06:49:21.860 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:13.253128+0000", 2024-09-11T06:49:21.860 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 8.5680638869999992, 2024-09-11T06:49:21.860 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034023700000000002, 2024-09-11T06:49:21.860 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:21.860 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:21.860 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:49:21.860 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:21.861 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.861 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:21.861 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253128+0000", 2024-09-11T06:49:21.861 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:21.861 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.861 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.861 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:21.864 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253128+0000", 2024-09-11T06:49:21.864 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:21.864 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.865 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.865 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:21.865 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253135+0000", 2024-09-11T06:49:21.865 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.657e-06 2024-09-11T06:49:21.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.865 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.865 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:21.865 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253144+0000", 2024-09-11T06:49:21.865 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0869999999999996e-06 2024-09-11T06:49:21.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.865 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.865 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:21.865 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253167+0000", 2024-09-11T06:49:21.865 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3252e-05 2024-09-11T06:49:21.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.866 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:21.866 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253186+0000", 2024-09-11T06:49:21.866 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9239999999999999e-05 2024-09-11T06:49:21.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.866 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:21.866 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253282+0000", 2024-09-11T06:49:21.866 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.6093000000000003e-05 2024-09-11T06:49:21.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.866 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:21.866 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253294+0000", 2024-09-11T06:49:21.867 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1986e-05 2024-09-11T06:49:21.867 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:21.867 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:21.867 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:21.867 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253468+0000", 2024-09-11T06:49:21.867 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017392199999999999 2024-09-11T06:49:21.867 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:21.867 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:21.867 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:21.867 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:21.867 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:21.867 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:21.869 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:21.866+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:21.869 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:49:21.969 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:21.966+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:21.995 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:21.995 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:21.995 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:49:21.995 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:22.009 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:49:22.070 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:22.066+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:22.126 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:22.127 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:22.127 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:49:22.127 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:49:22.127 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:22.140 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:49:22.171 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:22.170+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:22.265 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:22.265 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:49:22.265 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:49:22.265 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:49:22.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.265 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:54 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'32, mlcod=13'29)", 2024-09-11T06:49:22.265 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:49:22.265 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 16.054482290999999, 2024-09-11T06:49:22.266 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034271670000000001, 2024-09-11T06:49:22.266 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:22.266 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:22.266 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:22.266 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:22.266 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.266 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:22.266 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:49:22.266 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.266 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.266 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:22.266 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:49:22.267 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.267 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.267 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:22.267 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210361+0000", 2024-09-11T06:49:22.267 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1260000000000001e-06 2024-09-11T06:49:22.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.267 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.267 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:22.267 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210370+0000", 2024-09-11T06:49:22.267 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8200000000000003e-06 2024-09-11T06:49:22.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.267 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.267 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:22.267 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210390+0000", 2024-09-11T06:49:22.268 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0126e-05 2024-09-11T06:49:22.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.268 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:22.268 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210404+0000", 2024-09-11T06:49:22.268 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4589999999999999e-05 2024-09-11T06:49:22.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.268 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:22.268 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210538+0000", 2024-09-11T06:49:22.268 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000133719 2024-09-11T06:49:22.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.268 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:22.269 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210593+0000", 2024-09-11T06:49:22.269 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5016999999999999e-05 2024-09-11T06:49:22.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.273 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:22.273 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213695+0000", 2024-09-11T06:49:22.273 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003101866 2024-09-11T06:49:22.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.273 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:22.273 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213787+0000", 2024-09-11T06:49:22.274 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1903000000000004e-05 2024-09-11T06:49:22.274 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.274 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:22.274 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.274 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.274 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.274 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:55 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'33, mlcod=13'29)", 2024-09-11T06:49:22.274 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:49:22.274 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 16.053463053000002, 2024-09-11T06:49:22.274 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.005086091, 2024-09-11T06:49:22.274 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:22.274 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:22.275 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:22.275 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:22.275 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.275 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:22.275 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:49:22.275 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.275 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.275 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.275 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:22.275 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:49:22.275 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.275 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.276 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.276 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:22.276 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211380+0000", 2024-09-11T06:49:22.276 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.097e-06 2024-09-11T06:49:22.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.276 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.276 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:22.276 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211537+0000", 2024-09-11T06:49:22.276 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000157369 2024-09-11T06:49:22.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.276 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.276 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:22.276 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211559+0000", 2024-09-11T06:49:22.277 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000001e-05 2024-09-11T06:49:22.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.277 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:22.277 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211573+0000", 2024-09-11T06:49:22.277 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4946000000000001e-05 2024-09-11T06:49:22.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.277 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:22.277 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211676+0000", 2024-09-11T06:49:22.277 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0001026 2024-09-11T06:49:22.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.278 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:22.278 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211709+0000", 2024-09-11T06:49:22.278 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3071999999999998e-05 2024-09-11T06:49:22.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.278 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.278 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:22.278 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216379+0000", 2024-09-11T06:49:22.278 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0046693780000000001 2024-09-11T06:49:22.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.278 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.278 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:22.278 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216465+0000", 2024-09-11T06:49:22.278 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6469000000000002e-05 2024-09-11T06:49:22.278 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.279 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:22.279 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.279 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.279 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.279 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:56 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'34, mlcod=13'29)", 2024-09-11T06:49:22.279 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:49:22.279 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 16.052029193999999, 2024-09-11T06:49:22.279 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0060310629999999997, 2024-09-11T06:49:22.279 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:22.279 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:22.279 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:22.279 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:22.279 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.279 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:22.280 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:49:22.280 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.280 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.280 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:22.280 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:49:22.280 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.280 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.280 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:22.280 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212814+0000", 2024-09-11T06:49:22.280 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0869999999999999e-06 2024-09-11T06:49:22.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.281 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.281 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:22.281 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212822+0000", 2024-09-11T06:49:22.281 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0870000000000007e-06 2024-09-11T06:49:22.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.281 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.281 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:22.281 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212840+0000", 2024-09-11T06:49:22.281 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:49:22.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.281 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.281 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:22.281 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212854+0000", 2024-09-11T06:49:22.281 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4003e-05 2024-09-11T06:49:22.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.282 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:22.282 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213026+0000", 2024-09-11T06:49:22.282 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017207499999999999 2024-09-11T06:49:22.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.282 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:22.282 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213063+0000", 2024-09-11T06:49:22.282 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7187000000000002e-05 2024-09-11T06:49:22.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.282 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:22.282 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218762+0000", 2024-09-11T06:49:22.283 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0056993290000000004 2024-09-11T06:49:22.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.283 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:22.283 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218844+0000", 2024-09-11T06:49:22.283 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1701999999999998e-05 2024-09-11T06:49:22.283 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.283 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:22.283 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.283 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:60 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'38, mlcod=13'36)", 2024-09-11T06:49:22.283 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:49:22.283 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 16.038087845, 2024-09-11T06:49:22.283 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039254270000000004, 2024-09-11T06:49:22.284 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:22.284 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:22.284 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:22.284 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:22.284 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.284 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:22.284 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:49:22.284 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.284 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.284 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:22.284 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:49:22.284 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.285 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:22.285 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226756+0000", 2024-09-11T06:49:22.285 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-11T06:49:22.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.285 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:22.285 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226768+0000", 2024-09-11T06:49:22.285 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1919999999999999e-05 2024-09-11T06:49:22.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.285 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:22.285 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226797+0000", 2024-09-11T06:49:22.286 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9573e-05 2024-09-11T06:49:22.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.286 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:22.286 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226818+0000", 2024-09-11T06:49:22.286 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0432999999999999e-05 2024-09-11T06:49:22.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.286 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:22.286 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226909+0000", 2024-09-11T06:49:22.286 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0920000000000004e-05 2024-09-11T06:49:22.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.286 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:22.286 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226936+0000", 2024-09-11T06:49:22.286 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7472999999999999e-05 2024-09-11T06:49:22.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.287 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.287 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:22.287 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230596+0000", 2024-09-11T06:49:22.287 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0036602520000000001 2024-09-11T06:49:22.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.287 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.287 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:22.287 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230680+0000", 2024-09-11T06:49:22.287 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3106e-05 2024-09-11T06:49:22.287 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.287 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:22.287 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.287 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.288 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:107 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'69, mlcod=13'64)", 2024-09-11T06:49:22.288 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:49:22.288 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.821640388, 2024-09-11T06:49:22.288 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040728830000000002, 2024-09-11T06:49:22.288 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:22.288 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:22.288 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:22.288 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:22.288 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.288 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:22.288 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:49:22.288 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.288 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.289 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:22.289 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:49:22.289 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.289 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.289 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.289 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:22.289 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443203+0000", 2024-09-11T06:49:22.289 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1480000000000001e-06 2024-09-11T06:49:22.289 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.289 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.289 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:22.289 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443222+0000", 2024-09-11T06:49:22.289 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9040000000000001e-05 2024-09-11T06:49:22.289 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.290 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.290 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:22.290 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443246+0000", 2024-09-11T06:49:22.290 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4606000000000001e-05 2024-09-11T06:49:22.290 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.290 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.290 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:22.290 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443277+0000", 2024-09-11T06:49:22.290 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0207999999999999e-05 2024-09-11T06:49:22.290 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.290 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.290 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:22.290 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443340+0000", 2024-09-11T06:49:22.290 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3602000000000005e-05 2024-09-11T06:49:22.290 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.291 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:22.291 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443363+0000", 2024-09-11T06:49:22.291 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2331e-05 2024-09-11T06:49:22.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.291 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:22.291 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447216+0000", 2024-09-11T06:49:22.291 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0038539690000000001 2024-09-11T06:49:22.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.291 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:22.291 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447274+0000", 2024-09-11T06:49:22.292 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7979000000000002e-05 2024-09-11T06:49:22.292 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.292 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:22.292 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.292 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.292 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.292 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:111 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'73, mlcod=13'67)", 2024-09-11T06:49:22.292 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:49:22.292 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.815128758, 2024-09-11T06:49:22.292 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034372360000000002, 2024-09-11T06:49:22.292 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:22.292 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:22.292 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:22.292 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:22.293 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.293 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:22.293 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:49:22.293 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.293 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.293 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:22.293 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:49:22.293 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.293 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.293 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:22.293 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449714+0000", 2024-09-11T06:49:22.293 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5000000000000001e-07 2024-09-11T06:49:22.294 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.294 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.294 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:22.294 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449721+0000", 2024-09-11T06:49:22.294 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8530000000000003e-06 2024-09-11T06:49:22.294 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.294 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.294 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:22.294 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449736+0000", 2024-09-11T06:49:22.294 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5114e-05 2024-09-11T06:49:22.294 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.294 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.294 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:22.295 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449743+0000", 2024-09-11T06:49:22.295 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0210000000000002e-06 2024-09-11T06:49:22.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.295 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.295 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:22.295 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449804+0000", 2024-09-11T06:49:22.295 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0479999999999997e-05 2024-09-11T06:49:22.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.295 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.295 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:22.295 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449826+0000", 2024-09-11T06:49:22.295 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1922000000000001e-05 2024-09-11T06:49:22.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.296 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.296 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:22.296 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453104+0000", 2024-09-11T06:49:22.296 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003278652 2024-09-11T06:49:22.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.296 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.296 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:22.296 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453150+0000", 2024-09-11T06:49:22.296 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6244e-05 2024-09-11T06:49:22.296 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.296 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:22.296 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.297 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.297 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:114 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'76, mlcod=13'71)", 2024-09-11T06:49:22.297 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:49:22.297 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.810450686999999, 2024-09-11T06:49:22.297 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0042576660000000002, 2024-09-11T06:49:22.297 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:22.297 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:22.297 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:22.297 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:22.297 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.297 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:22.297 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:49:22.297 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.298 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.298 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.298 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:22.298 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:49:22.298 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.298 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.298 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.298 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:22.298 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454392+0000", 2024-09-11T06:49:22.298 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5999999999999991e-07 2024-09-11T06:49:22.298 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.298 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.298 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:22.299 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454399+0000", 2024-09-11T06:49:22.299 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.878e-06 2024-09-11T06:49:22.299 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.299 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.299 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:22.299 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454415+0000", 2024-09-11T06:49:22.299 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5659999999999999e-05 2024-09-11T06:49:22.299 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.299 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.299 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:22.299 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454422+0000", 2024-09-11T06:49:22.299 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:49:22.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.300 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.300 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:22.300 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454474+0000", 2024-09-11T06:49:22.300 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1860999999999997e-05 2024-09-11T06:49:22.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.300 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.300 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:22.300 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454497+0000", 2024-09-11T06:49:22.300 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3686e-05 2024-09-11T06:49:22.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.300 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.300 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:22.301 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458552+0000", 2024-09-11T06:49:22.301 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040544550000000002 2024-09-11T06:49:22.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.301 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.301 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:22.301 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458649+0000", 2024-09-11T06:49:22.301 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7005999999999999e-05 2024-09-11T06:49:22.301 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.301 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:22.301 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.301 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.301 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:115 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'77, mlcod=13'72)", 2024-09-11T06:49:22.301 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:22.302 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.809110075, 2024-09-11T06:49:22.302 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003355584, 2024-09-11T06:49:22.302 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:22.302 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:22.302 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:22.302 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:22.302 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.302 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:22.302 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:22.302 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.302 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.302 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.302 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:22.302 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:22.303 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.303 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.303 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.303 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:22.303 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:22.303 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5999999999999999e-07 2024-09-11T06:49:22.303 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.303 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.303 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:22.303 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455803+0000", 2024-09-11T06:49:22.303 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0310999999999998e-05 2024-09-11T06:49:22.303 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.303 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.303 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:22.304 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455815+0000", 2024-09-11T06:49:22.304 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2787e-05 2024-09-11T06:49:22.304 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.304 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.304 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:22.304 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455823+0000", 2024-09-11T06:49:22.304 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5959999999999998e-06 2024-09-11T06:49:22.304 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.304 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.304 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:22.304 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455875+0000", 2024-09-11T06:49:22.304 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2054e-05 2024-09-11T06:49:22.304 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.304 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.305 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:22.305 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455896+0000", 2024-09-11T06:49:22.305 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0554999999999999e-05 2024-09-11T06:49:22.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.305 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.305 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:22.305 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459000+0000", 2024-09-11T06:49:22.305 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003104842 2024-09-11T06:49:22.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.305 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.305 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:22.305 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459087+0000", 2024-09-11T06:49:22.305 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6978999999999995e-05 2024-09-11T06:49:22.305 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.306 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:22.306 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.306 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:120 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'82, mlcod=13'78)", 2024-09-11T06:49:22.306 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:49:22.306 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.79843279, 2024-09-11T06:49:22.306 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0064612549999999999, 2024-09-11T06:49:22.306 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:22.306 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:22.306 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:22.306 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:22.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.306 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:22.307 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:49:22.307 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.307 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.307 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.307 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:22.307 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:49:22.307 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.307 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.307 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.307 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:22.307 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466410+0000", 2024-09-11T06:49:22.307 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0389999999999999e-06 2024-09-11T06:49:22.307 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.308 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.308 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:22.308 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466496+0000", 2024-09-11T06:49:22.308 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5519000000000006e-05 2024-09-11T06:49:22.308 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.308 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.308 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:22.308 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466510+0000", 2024-09-11T06:49:22.308 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3842999999999999e-05 2024-09-11T06:49:22.308 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.308 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.308 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:22.308 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466517+0000", 2024-09-11T06:49:22.308 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5229999999999998e-06 2024-09-11T06:49:22.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.309 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.309 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:22.309 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466595+0000", 2024-09-11T06:49:22.309 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8019e-05 2024-09-11T06:49:22.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.309 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.309 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:22.309 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466612+0000", 2024-09-11T06:49:22.309 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7013000000000001e-05 2024-09-11T06:49:22.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.309 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.309 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:22.310 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472787+0000", 2024-09-11T06:49:22.310 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0061750269999999996 2024-09-11T06:49:22.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.310 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.310 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:22.310 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472870+0000", 2024-09-11T06:49:22.310 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3271999999999998e-05 2024-09-11T06:49:22.310 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.310 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:22.310 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.310 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.311 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:121 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'83, mlcod=13'78)", 2024-09-11T06:49:22.311 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:49:22.311 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.797207994000001, 2024-09-11T06:49:22.311 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055613069999999997, 2024-09-11T06:49:22.311 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:22.311 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:22.311 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:22.311 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:22.311 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.311 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:22.311 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:49:22.311 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.311 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.311 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.312 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:22.312 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:49:22.312 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.312 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.312 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.312 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:22.312 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467635+0000", 2024-09-11T06:49:22.312 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6899999999999996e-07 2024-09-11T06:49:22.312 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.312 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.312 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:22.312 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467727+0000", 2024-09-11T06:49:22.312 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1800999999999997e-05 2024-09-11T06:49:22.313 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.313 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.313 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:22.313 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467740+0000", 2024-09-11T06:49:22.313 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3747999999999999e-05 2024-09-11T06:49:22.313 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.313 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.313 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:22.313 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467753+0000", 2024-09-11T06:49:22.313 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2807e-05 2024-09-11T06:49:22.313 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.313 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.313 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:22.314 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467812+0000", 2024-09-11T06:49:22.314 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8381999999999998e-05 2024-09-11T06:49:22.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.314 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.314 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:22.314 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467828+0000", 2024-09-11T06:49:22.314 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6728e-05 2024-09-11T06:49:22.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.314 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.314 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:22.314 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473112+0000", 2024-09-11T06:49:22.314 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052833619999999998 2024-09-11T06:49:22.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.315 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:22.315 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473195+0000", 2024-09-11T06:49:22.315 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3609999999999994e-05 2024-09-11T06:49:22.315 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.315 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:22.315 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.315 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:131 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'89, mlcod=13'85)", 2024-09-11T06:49:22.315 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:49:22.315 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.445802279, 2024-09-11T06:49:22.315 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052492260000000001, 2024-09-11T06:49:22.316 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:22.316 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:22.316 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:22.316 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:22.316 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.316 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:22.316 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:49:22.316 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.316 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.316 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.316 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:22.316 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:49:22.316 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.316 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.317 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.317 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:22.317 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819041+0000", 2024-09-11T06:49:22.317 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4360000000000001e-06 2024-09-11T06:49:22.317 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.317 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.317 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:22.317 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819100+0000", 2024-09-11T06:49:22.317 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9225999999999998e-05 2024-09-11T06:49:22.317 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.317 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.317 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:22.317 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819128+0000", 2024-09-11T06:49:22.317 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.779e-05 2024-09-11T06:49:22.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.318 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.318 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:22.318 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819142+0000", 2024-09-11T06:49:22.318 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3573e-05 2024-09-11T06:49:22.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.318 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.318 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:22.318 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819263+0000", 2024-09-11T06:49:22.318 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000121086 2024-09-11T06:49:22.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.318 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.318 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:22.319 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819292+0000", 2024-09-11T06:49:22.319 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9363e-05 2024-09-11T06:49:22.319 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.319 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.319 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:22.319 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824185+0000", 2024-09-11T06:49:22.319 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0048924620000000002 2024-09-11T06:49:22.319 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.319 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.319 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:22.319 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824289+0000", 2024-09-11T06:49:22.319 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010429000000000001 2024-09-11T06:49:22.319 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.319 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:22.320 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.320 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:132 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'90, mlcod=13'85)", 2024-09-11T06:49:22.320 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:49:22.320 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.445027589, 2024-09-11T06:49:22.320 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.004532342, 2024-09-11T06:49:22.320 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:22.320 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:22.320 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:22.320 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:22.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.320 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:22.320 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:49:22.321 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.321 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:22.321 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:49:22.321 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.321 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:22.321 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819816+0000", 2024-09-11T06:49:22.321 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.61e-06 2024-09-11T06:49:22.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.322 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:22.322 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819827+0000", 2024-09-11T06:49:22.322 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1e-05 2024-09-11T06:49:22.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.322 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.322 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:22.322 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819851+0000", 2024-09-11T06:49:22.322 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4267e-05 2024-09-11T06:49:22.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.322 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.322 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:22.322 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819864+0000", 2024-09-11T06:49:22.322 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2767e-05 2024-09-11T06:49:22.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.323 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.323 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:22.323 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820071+0000", 2024-09-11T06:49:22.323 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000207061 2024-09-11T06:49:22.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.323 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.323 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:22.323 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820103+0000", 2024-09-11T06:49:22.323 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2326999999999997e-05 2024-09-11T06:49:22.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.323 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.323 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:22.323 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824302+0000", 2024-09-11T06:49:22.323 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0041986000000000002 2024-09-11T06:49:22.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.324 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:22.324 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824347+0000", 2024-09-11T06:49:22.324 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4709999999999997e-05 2024-09-11T06:49:22.324 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.324 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:22.324 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.324 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:133 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'91, mlcod=13'86)", 2024-09-11T06:49:22.324 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:49:22.324 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.444395930000001, 2024-09-11T06:49:22.324 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039421459999999997, 2024-09-11T06:49:22.325 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:22.325 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:22.325 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:22.325 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:22.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.325 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:22.325 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:49:22.325 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.325 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:22.325 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:49:22.325 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.326 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:22.326 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820448+0000", 2024-09-11T06:49:22.326 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.84e-06 2024-09-11T06:49:22.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.326 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:22.326 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820463+0000", 2024-09-11T06:49:22.326 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5356e-05 2024-09-11T06:49:22.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.326 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:22.326 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820499+0000", 2024-09-11T06:49:22.326 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.625e-05 2024-09-11T06:49:22.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.327 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.327 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:22.327 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820546+0000", 2024-09-11T06:49:22.327 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7017000000000001e-05 2024-09-11T06:49:22.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.327 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.327 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:22.327 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820882+0000", 2024-09-11T06:49:22.327 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033512 2024-09-11T06:49:22.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.327 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.327 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:22.327 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820941+0000", 2024-09-11T06:49:22.328 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8922999999999998e-05 2024-09-11T06:49:22.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.328 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.328 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:22.328 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824351+0000", 2024-09-11T06:49:22.328 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034101940000000001 2024-09-11T06:49:22.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.328 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.328 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:22.328 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824388+0000", 2024-09-11T06:49:22.328 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7446e-05 2024-09-11T06:49:22.329 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.329 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:22.329 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.329 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.329 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:134 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'92, mlcod=13'86)", 2024-09-11T06:49:22.329 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:49:22.329 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.443736811999999, 2024-09-11T06:49:22.329 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033248179999999998, 2024-09-11T06:49:22.329 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:22.329 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:22.329 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:22.329 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:22.329 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.330 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:22.330 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:49:22.330 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.330 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.330 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:22.330 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:49:22.330 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.330 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.330 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:22.330 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821107+0000", 2024-09-11T06:49:22.330 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7430000000000001e-06 2024-09-11T06:49:22.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.331 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.331 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:22.331 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821205+0000", 2024-09-11T06:49:22.331 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7666999999999998e-05 2024-09-11T06:49:22.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.331 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.331 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:22.331 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821234+0000", 2024-09-11T06:49:22.331 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9756000000000001e-05 2024-09-11T06:49:22.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.331 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.331 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:22.331 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821254+0000", 2024-09-11T06:49:22.332 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9446e-05 2024-09-11T06:49:22.332 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.332 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.332 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:22.332 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821846+0000", 2024-09-11T06:49:22.332 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00059187599999999997 2024-09-11T06:49:22.332 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.332 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.332 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:22.332 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821894+0000", 2024-09-11T06:49:22.332 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8766e-05 2024-09-11T06:49:22.332 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.332 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.332 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:22.333 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824392+0000", 2024-09-11T06:49:22.333 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0024977509999999999 2024-09-11T06:49:22.333 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.333 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.333 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:22.333 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824430+0000", 2024-09-11T06:49:22.333 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7812999999999999e-05 2024-09-11T06:49:22.333 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.333 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:22.333 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.333 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.333 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.333 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:137 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'95, mlcod=13'86)", 2024-09-11T06:49:22.333 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:49:22.334 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.441274671, 2024-09-11T06:49:22.334 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003084299, 2024-09-11T06:49:22.334 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:22.334 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:22.334 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:22.334 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:22.334 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.334 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:22.334 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:49:22.334 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.334 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.334 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.334 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:22.335 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:49:22.335 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.335 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.335 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.335 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:22.335 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823568+0000", 2024-09-11T06:49:22.335 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1030000000000001e-06 2024-09-11T06:49:22.335 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.335 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.335 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:22.335 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823662+0000", 2024-09-11T06:49:22.335 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3529999999999994e-05 2024-09-11T06:49:22.335 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.335 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.336 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:22.336 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823686+0000", 2024-09-11T06:49:22.336 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4358999999999998e-05 2024-09-11T06:49:22.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.336 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.336 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:22.336 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823701+0000", 2024-09-11T06:49:22.336 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4847e-05 2024-09-11T06:49:22.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.336 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.336 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:22.336 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824518+0000", 2024-09-11T06:49:22.336 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00081716700000000005 2024-09-11T06:49:22.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.337 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.337 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:22.337 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824546+0000", 2024-09-11T06:49:22.337 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7860000000000001e-05 2024-09-11T06:49:22.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.337 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.337 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:22.337 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826558+0000", 2024-09-11T06:49:22.337 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0020120009999999998 2024-09-11T06:49:22.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.337 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.337 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:22.337 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826652+0000", 2024-09-11T06:49:22.338 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3431999999999996e-05 2024-09-11T06:49:22.338 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.338 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:22.338 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.338 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:139 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'97, mlcod=13'86)", 2024-09-11T06:49:22.338 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:49:22.338 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.438985951999999, 2024-09-11T06:49:22.338 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003732221, 2024-09-11T06:49:22.338 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:22.338 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:22.338 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:22.338 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:22.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.339 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:22.339 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:49:22.339 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.339 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:22.339 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:49:22.339 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.339 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:22.339 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825858+0000", 2024-09-11T06:49:22.339 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5829999999999999e-06 2024-09-11T06:49:22.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.340 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.340 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:22.340 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825874+0000", 2024-09-11T06:49:22.340 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5967000000000001e-05 2024-09-11T06:49:22.340 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.340 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.340 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:22.340 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825909+0000", 2024-09-11T06:49:22.340 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5692999999999999e-05 2024-09-11T06:49:22.340 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.340 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.340 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:22.340 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825923+0000", 2024-09-11T06:49:22.340 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4080000000000001e-05 2024-09-11T06:49:22.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.341 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.341 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:22.341 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826010+0000", 2024-09-11T06:49:22.341 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6248999999999994e-05 2024-09-11T06:49:22.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.341 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.341 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:22.341 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826038+0000", 2024-09-11T06:49:22.341 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8010000000000001e-05 2024-09-11T06:49:22.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.341 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.342 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:22.342 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829499+0000", 2024-09-11T06:49:22.342 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003461783 2024-09-11T06:49:22.342 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.342 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.342 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:22.342 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829588+0000", 2024-09-11T06:49:22.342 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8856000000000004e-05 2024-09-11T06:49:22.342 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.342 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:22.342 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.342 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.342 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.342 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:140 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'98, mlcod=13'93)", 2024-09-11T06:49:22.343 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:49:22.343 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.437135223, 2024-09-11T06:49:22.343 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0057995929999999996, 2024-09-11T06:49:22.343 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:22.343 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:22.343 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:22.343 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:22.343 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.343 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:22.343 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:49:22.343 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.343 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.344 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:22.344 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:49:22.344 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.344 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.344 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:22.344 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827708+0000", 2024-09-11T06:49:22.344 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.547e-06 2024-09-11T06:49:22.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.344 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.344 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:22.344 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827765+0000", 2024-09-11T06:49:22.344 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6266e-05 2024-09-11T06:49:22.345 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.345 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:22.345 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827791+0000", 2024-09-11T06:49:22.345 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6098999999999999e-05 2024-09-11T06:49:22.345 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.345 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:22.345 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827806+0000", 2024-09-11T06:49:22.345 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5537e-05 2024-09-11T06:49:22.345 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.345 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:22.346 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827928+0000", 2024-09-11T06:49:22.346 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000122012 2024-09-11T06:49:22.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.346 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.346 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:22.346 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827961+0000", 2024-09-11T06:49:22.346 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2663999999999998e-05 2024-09-11T06:49:22.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.346 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.346 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:22.346 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833409+0000", 2024-09-11T06:49:22.346 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054479350000000001 2024-09-11T06:49:22.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.346 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.347 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:22.347 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833506+0000", 2024-09-11T06:49:22.347 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7533000000000003e-05 2024-09-11T06:49:22.347 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.347 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:22.347 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.347 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.347 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:141 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'99, mlcod=13'93)", 2024-09-11T06:49:22.347 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:49:22.347 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.436241839999999, 2024-09-11T06:49:22.347 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055394709999999998, 2024-09-11T06:49:22.347 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:22.348 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:22.348 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:22.348 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:22.348 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.348 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:22.348 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:49:22.348 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.348 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.348 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.348 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:22.348 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:49:22.348 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.349 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.349 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:22.349 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828602+0000", 2024-09-11T06:49:22.349 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.46e-06 2024-09-11T06:49:22.349 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.349 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:22.349 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828618+0000", 2024-09-11T06:49:22.349 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6592999999999999e-05 2024-09-11T06:49:22.349 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.349 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:22.349 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828646+0000", 2024-09-11T06:49:22.350 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7552999999999999e-05 2024-09-11T06:49:22.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.350 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:22.350 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828663+0000", 2024-09-11T06:49:22.350 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.719e-05 2024-09-11T06:49:22.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.350 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:22.350 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829023+0000", 2024-09-11T06:49:22.350 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036054799999999998 2024-09-11T06:49:22.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.351 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:22.351 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829062+0000", 2024-09-11T06:49:22.351 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8865999999999997e-05 2024-09-11T06:49:22.351 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.351 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.351 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:22.351 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834078+0000", 2024-09-11T06:49:22.351 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050157480000000004 2024-09-11T06:49:22.351 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.351 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.351 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:22.351 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834140+0000", 2024-09-11T06:49:22.351 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1513000000000005e-05 2024-09-11T06:49:22.352 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.352 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:22.352 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.352 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.352 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.352 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:144 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'102, mlcod=13'98)", 2024-09-11T06:49:22.352 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:49:22.352 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.427081227, 2024-09-11T06:49:22.352 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003210348, 2024-09-11T06:49:22.352 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:22.352 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:22.352 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:22.352 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:22.352 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.353 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:22.353 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:49:22.353 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.353 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.353 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:22.353 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:49:22.353 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.353 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.353 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:22.353 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837768+0000", 2024-09-11T06:49:22.353 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9330000000000004e-06 2024-09-11T06:49:22.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.354 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.354 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:22.354 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837858+0000", 2024-09-11T06:49:22.354 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0185999999999993e-05 2024-09-11T06:49:22.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.354 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.354 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:22.354 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837890+0000", 2024-09-11T06:49:22.354 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2437000000000001e-05 2024-09-11T06:49:22.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.354 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.354 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:22.354 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837905+0000", 2024-09-11T06:49:22.354 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:49:22.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.355 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:22.355 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838019+0000", 2024-09-11T06:49:22.355 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000114336 2024-09-11T06:49:22.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.355 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:22.355 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838052+0000", 2024-09-11T06:49:22.355 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3139000000000002e-05 2024-09-11T06:49:22.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.355 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:22.355 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840914+0000", 2024-09-11T06:49:22.355 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028612780000000001 2024-09-11T06:49:22.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.356 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:22.356 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840971+0000", 2024-09-11T06:49:22.356 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7402999999999999e-05 2024-09-11T06:49:22.356 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.356 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:22.356 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.356 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:145 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'103, mlcod=13'98)", 2024-09-11T06:49:22.356 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:49:22.356 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.426289343000001, 2024-09-11T06:49:22.356 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054455409999999999, 2024-09-11T06:49:22.356 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:22.357 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:22.357 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:22.357 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:22.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.357 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:22.357 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:49:22.357 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.357 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:22.357 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:49:22.357 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.357 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:22.358 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838554+0000", 2024-09-11T06:49:22.358 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3430000000000001e-06 2024-09-11T06:49:22.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.358 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:22.358 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838640+0000", 2024-09-11T06:49:22.358 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5569999999999996e-05 2024-09-11T06:49:22.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.358 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:22.358 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838660+0000", 2024-09-11T06:49:22.358 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0576000000000001e-05 2024-09-11T06:49:22.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.358 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:22.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838674+0000", 2024-09-11T06:49:22.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3529999999999999e-05 2024-09-11T06:49:22.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.359 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:22.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838838+0000", 2024-09-11T06:49:22.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016468799999999999 2024-09-11T06:49:22.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.359 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:22.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838885+0000", 2024-09-11T06:49:22.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6884e-05 2024-09-11T06:49:22.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.360 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:22.360 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843901+0000", 2024-09-11T06:49:22.360 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050162310000000003 2024-09-11T06:49:22.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.360 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:22.360 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843998+0000", 2024-09-11T06:49:22.361 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.6718999999999993e-05 2024-09-11T06:49:22.361 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.361 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:22.361 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.361 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.361 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:22.361 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:22.362 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T06:49:22.363 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:49:22.364 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:22.358+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:22.464 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:22.462+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:22.539 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:22.539 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:22.539 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:49:22.539 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:22.554 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:49:22.565 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:22.562+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:22.666 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:22.662+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:22.708 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:22.708 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:22.708 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:49:22.708 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:49:22.709 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:22.723 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:49:22.766 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:22.766+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:22.867 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:22.862+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:22.884 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:22.884 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:49:22.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:49:22.885 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:49:22.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.885 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:5 2.2 e17/15 2:5cea7035:::rbd_info:head v 17'1, mlcod=0'0)", 2024-09-11T06:49:22.885 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:49:22.885 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 10.646792292000001, 2024-09-11T06:49:22.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0019446719999999999, 2024-09-11T06:49:22.885 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:22.885 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:22.885 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:22.885 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:22.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:22.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:49:22.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:22.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:49:22.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:22.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236085+0000", 2024-09-11T06:49:22.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6000000000000003e-07 2024-09-11T06:49:22.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:22.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236108+0000", 2024-09-11T06:49:22.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3113e-05 2024-09-11T06:49:22.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.887 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:22.887 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236133+0000", 2024-09-11T06:49:22.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5723000000000001e-05 2024-09-11T06:49:22.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.887 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:22.887 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236154+0000", 2024-09-11T06:49:22.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0333999999999999e-05 2024-09-11T06:49:22.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.888 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:22.888 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236297+0000", 2024-09-11T06:49:22.888 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014278500000000001 2024-09-11T06:49:22.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.888 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:22.888 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236383+0000", 2024-09-11T06:49:22.888 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5927999999999995e-05 2024-09-11T06:49:22.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.888 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:22.888 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.237950+0000", 2024-09-11T06:49:22.888 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015673690000000001 2024-09-11T06:49:22.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.889 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:22.889 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.238029+0000", 2024-09-11T06:49:22.889 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8659999999999996e-05 2024-09-11T06:49:22.889 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.889 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:22.889 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.889 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:7 2.2 e18/15 2:5cea7035:::rbd_info:head v 18'2, mlcod=0'0)", 2024-09-11T06:49:22.889 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:49:22.889 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.632467085, 2024-09-11T06:49:22.889 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001907192, 2024-09-11T06:49:22.889 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:22.890 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:22.890 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:22.890 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:22.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:22.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:49:22.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:22.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:49:22.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:22.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:22.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250410+0000", 2024-09-11T06:49:22.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7700000000000004e-07 2024-09-11T06:49:22.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:22.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250445+0000", 2024-09-11T06:49:22.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4749000000000002e-05 2024-09-11T06:49:22.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:22.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250478+0000", 2024-09-11T06:49:22.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3269999999999998e-05 2024-09-11T06:49:22.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:22.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250501+0000", 2024-09-11T06:49:22.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3632999999999999e-05 2024-09-11T06:49:22.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:22.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250633+0000", 2024-09-11T06:49:22.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013143200000000001 2024-09-11T06:49:22.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:22.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250700+0000", 2024-09-11T06:49:22.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7002999999999995e-05 2024-09-11T06:49:22.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.893 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:22.893 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.252172+0000", 2024-09-11T06:49:22.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014719550000000001 2024-09-11T06:49:22.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:22.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:22.893 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:22.893 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.252316+0000", 2024-09-11T06:49:22.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144473 2024-09-11T06:49:22.893 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.893 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:22.893 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.893 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:22.894 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:22.894 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:22.896 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:49:22.968 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:22.966+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:23.059 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:23.059 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:23.059 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:49:23.059 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:23.069 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:23.066+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:23.073 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:49:23.170 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:23.166+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:23.240 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:23.241 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:23.241 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:49:23.241 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:49:23.241 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:23.255 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:49:23.271 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:23.270+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:23.293 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is set 2024-09-11T06:49:23.372 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:23.370+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:23.374 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:23.374 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:49:23.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:49:23.374 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:49:23.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:23.374 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:1 2.3 e16/15 2:c4c92e5a:::rbd_trash:head v 16'1, mlcod=0'0)", 2024-09-11T06:49:23.374 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:49:23.374 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.138830691000001, 2024-09-11T06:49:23.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001234761, 2024-09-11T06:49:23.374 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:23.374 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:23.374 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:23.374 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:23.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:23.375 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:23.375 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:49:23.375 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:23.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:23.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:23.375 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:23.375 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:49:23.375 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:23.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:23.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:23.375 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:23.375 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:49:23.375 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6000000000000003e-07 2024-09-11T06:49:23.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:23.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:23.376 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:23.376 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236296+0000", 2024-09-11T06:49:23.376 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6393999999999999e-05 2024-09-11T06:49:23.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:23.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:23.376 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:23.376 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236317+0000", 2024-09-11T06:49:23.376 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1243e-05 2024-09-11T06:49:23.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:23.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:23.377 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:23.377 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236333+0000", 2024-09-11T06:49:23.377 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.558e-05 2024-09-11T06:49:23.377 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:23.377 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:23.377 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:23.377 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236435+0000", 2024-09-11T06:49:23.377 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000102803 2024-09-11T06:49:23.377 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:23.377 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:23.377 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:23.377 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236474+0000", 2024-09-11T06:49:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8383999999999997e-05 2024-09-11T06:49:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.237432+0000", 2024-09-11T06:49:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00095849300000000004 2024-09-11T06:49:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.237513+0000", 2024-09-11T06:49:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1204000000000004e-05 2024-09-11T06:49:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:23.379 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:23.379 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:23.388 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:49:23.472 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:23.470+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:23.516 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:23.516 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:23.516 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:49:23.516 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:23.530 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:49:23.575 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:23.574+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:23.674 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:23.670+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:23.695 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:23.695 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:23.696 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:49:23.696 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:49:23.696 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:23.710 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:49:23.711 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:49:23.774 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:23.770+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:23.833 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:23.833 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:49:23.833 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:49:23.833 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:49:23.833 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.833 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:23.833 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:49:23.834 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.65981116, 2024-09-11T06:49:23.834 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011420669, 2024-09-11T06:49:23.834 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:23.834 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:23.834 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:23.834 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:23.834 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:23.834 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:23.834 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 37 2024-09-11T06:49:23.834 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.834 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:23.834 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.834 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:23.835 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:49:23.835 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.835 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.835 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.835 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:23.835 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:49:23.835 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.835 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.835 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.835 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:23.835 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170509+0000", 2024-09-11T06:49:23.835 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.13e-06 2024-09-11T06:49:23.835 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.836 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.836 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:23.836 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170521+0000", 2024-09-11T06:49:23.836 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1776999999999999e-05 2024-09-11T06:49:23.836 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.836 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.836 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:23.836 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170538+0000", 2024-09-11T06:49:23.837 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7413000000000001e-05 2024-09-11T06:49:23.837 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.837 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.837 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:23.837 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170570+0000", 2024-09-11T06:49:23.837 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2313e-05 2024-09-11T06:49:23.837 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.837 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.837 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:23.838 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179768+0000", 2024-09-11T06:49:23.838 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0091978010000000002 2024-09-11T06:49:23.838 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.838 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.838 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:23.838 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179951+0000", 2024-09-11T06:49:23.838 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018227900000000001 2024-09-11T06:49:23.838 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.838 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.838 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:23.838 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.180175+0000", 2024-09-11T06:49:23.839 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022432800000000001 2024-09-11T06:49:23.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.839 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.839 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:23.839 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181112+0000", 2024-09-11T06:49:23.839 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00093712500000000002 2024-09-11T06:49:23.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.839 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.840 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:23.840 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181789+0000", 2024-09-11T06:49:23.840 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00067705100000000002 2024-09-11T06:49:23.840 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.840 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.840 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:23.840 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181871+0000", 2024-09-11T06:49:23.840 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1816000000000004e-05 2024-09-11T06:49:23.840 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.840 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.840 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:23.840 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181929+0000", 2024-09-11T06:49:23.841 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7636000000000002e-05 2024-09-11T06:49:23.841 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.841 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:23.841 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.841 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.841 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.841 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:23.841 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:49:23.841 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.558552199000001, 2024-09-11T06:49:23.841 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283336, 2024-09-11T06:49:23.841 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:23.841 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:23.841 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:23.842 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:23.842 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:23.842 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:23.842 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 87 2024-09-11T06:49:23.842 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.842 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:23.842 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.842 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:23.843 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:49:23.843 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.843 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.843 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:23.843 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:49:23.843 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.843 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.843 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:23.843 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271768+0000", 2024-09-11T06:49:23.843 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7599999999999996e-07 2024-09-11T06:49:23.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.844 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.844 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:23.844 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271860+0000", 2024-09-11T06:49:23.844 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2639999999999994e-05 2024-09-11T06:49:23.844 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.844 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.844 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:23.844 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271873+0000", 2024-09-11T06:49:23.844 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3139000000000001e-05 2024-09-11T06:49:23.844 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.844 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.844 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:23.844 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271882+0000", 2024-09-11T06:49:23.845 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2369999999999992e-06 2024-09-11T06:49:23.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.845 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.845 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:23.845 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277150+0000", 2024-09-11T06:49:23.845 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052684419999999999 2024-09-11T06:49:23.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.845 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.845 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:23.845 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277273+0000", 2024-09-11T06:49:23.845 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012278899999999999 2024-09-11T06:49:23.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.845 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.846 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:23.846 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277466+0000", 2024-09-11T06:49:23.846 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193578 2024-09-11T06:49:23.846 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.846 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.846 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:23.846 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.280808+0000", 2024-09-11T06:49:23.846 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003341971 2024-09-11T06:49:23.846 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.846 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.846 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:23.846 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284419+0000", 2024-09-11T06:49:23.846 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003610982 2024-09-11T06:49:23.846 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.847 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.847 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:23.847 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284531+0000", 2024-09-11T06:49:23.847 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011121299999999999 2024-09-11T06:49:23.847 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.847 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.848 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:23.848 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284600+0000", 2024-09-11T06:49:23.848 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9592999999999996e-05 2024-09-11T06:49:23.848 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.848 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:23.848 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.849 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.849 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.849 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:23.849 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:49:23.849 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.558421946999999, 2024-09-11T06:49:23.849 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283314, 2024-09-11T06:49:23.849 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:23.849 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:23.850 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:23.850 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:23.850 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:23.850 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:23.850 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 88 2024-09-11T06:49:23.850 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.850 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:23.850 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.850 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:23.850 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:49:23.851 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.851 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.851 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.851 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:23.851 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:49:23.851 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.851 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.851 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.851 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:23.851 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271898+0000", 2024-09-11T06:49:23.851 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2999999999999999e-07 2024-09-11T06:49:23.851 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.851 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.852 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:23.852 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271904+0000", 2024-09-11T06:49:23.852 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.767e-06 2024-09-11T06:49:23.852 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.852 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.852 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:23.852 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271913+0000", 2024-09-11T06:49:23.852 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7200000000000001e-06 2024-09-11T06:49:23.852 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.852 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.852 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:23.852 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271920+0000", 2024-09-11T06:49:23.852 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7100000000000001e-06 2024-09-11T06:49:23.853 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.853 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.853 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:23.853 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278684+0000", 2024-09-11T06:49:23.853 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0067642650000000002 2024-09-11T06:49:23.853 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.853 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.853 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:23.853 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278838+0000", 2024-09-11T06:49:23.853 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015371499999999999 2024-09-11T06:49:23.853 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.853 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.854 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:23.854 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.279040+0000", 2024-09-11T06:49:23.854 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020207499999999999 2024-09-11T06:49:23.854 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.854 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.854 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:23.854 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.281448+0000", 2024-09-11T06:49:23.854 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002407686 2024-09-11T06:49:23.854 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.854 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.854 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:23.854 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284615+0000", 2024-09-11T06:49:23.854 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031675499999999999 2024-09-11T06:49:23.855 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.855 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.855 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:23.855 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284682+0000", 2024-09-11T06:49:23.855 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7032000000000005e-05 2024-09-11T06:49:23.855 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.855 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.855 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:23.855 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284730+0000", 2024-09-11T06:49:23.855 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7790000000000002e-05 2024-09-11T06:49:23.855 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.855 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:23.855 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.855 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.856 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.856 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:23.856 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:49:23.856 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.390520725000002, 2024-09-11T06:49:23.856 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011114877, 2024-09-11T06:49:23.856 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:23.856 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:23.856 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:23.856 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:23.856 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:23.856 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:23.856 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 107 2024-09-11T06:49:23.856 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.857 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:23.857 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.857 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:23.857 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:49:23.857 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.857 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.857 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.857 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:23.857 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:49:23.858 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.858 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.858 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.858 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:23.858 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439799+0000", 2024-09-11T06:49:23.858 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.16e-07 2024-09-11T06:49:23.858 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.858 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.858 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:23.858 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439806+0000", 2024-09-11T06:49:23.859 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3640000000000004e-06 2024-09-11T06:49:23.859 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.859 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.859 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:23.859 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439816+0000", 2024-09-11T06:49:23.859 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0485999999999999e-05 2024-09-11T06:49:23.859 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.859 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.859 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:23.860 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439824+0000", 2024-09-11T06:49:23.860 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.6699999999999994e-06 2024-09-11T06:49:23.860 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.860 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.860 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:23.860 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443328+0000", 2024-09-11T06:49:23.861 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035044329999999999 2024-09-11T06:49:23.861 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.861 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.861 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:23.861 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443461+0000", 2024-09-11T06:49:23.862 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013272199999999999 2024-09-11T06:49:23.862 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.862 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.862 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:23.862 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443678+0000", 2024-09-11T06:49:23.862 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021674500000000001 2024-09-11T06:49:23.863 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.863 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.863 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:23.863 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449635+0000", 2024-09-11T06:49:23.863 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0059572119999999999 2024-09-11T06:49:23.863 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.864 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.864 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:23.864 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450750+0000", 2024-09-11T06:49:23.864 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001115654 2024-09-11T06:49:23.864 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.864 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.864 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:23.865 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450832+0000", 2024-09-11T06:49:23.865 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1188999999999998e-05 2024-09-11T06:49:23.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.865 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.865 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:23.865 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450913+0000", 2024-09-11T06:49:23.865 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1485999999999999e-05 2024-09-11T06:49:23.866 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.866 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:23.866 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.866 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:23.867 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:49:23.867 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.38992975, 2024-09-11T06:49:23.867 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011236556, 2024-09-11T06:49:23.867 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:23.867 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:23.867 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:23.868 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:23.868 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:23.868 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:23.868 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 109 2024-09-11T06:49:23.868 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.868 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:23.869 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.869 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:23.869 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:49:23.869 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.869 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.869 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.869 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:23.870 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:49:23.870 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.870 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.870 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.870 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:23.870 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440391+0000", 2024-09-11T06:49:23.870 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8360000000000001e-06 2024-09-11T06:49:23.871 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.871 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.871 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:23.871 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440560+0000", 2024-09-11T06:49:23.871 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000169085 2024-09-11T06:49:23.871 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.871 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.871 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:23.872 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440585+0000", 2024-09-11T06:49:23.872 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4530000000000001e-05 2024-09-11T06:49:23.872 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.872 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.872 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:23.872 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440606+0000", 2024-09-11T06:49:23.872 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1277e-05 2024-09-11T06:49:23.873 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.873 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.873 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:23.873 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446479+0000", 2024-09-11T06:49:23.873 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0058734989999999999 2024-09-11T06:49:23.873 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.873 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.874 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:23.874 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446649+0000", 2024-09-11T06:49:23.874 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016948199999999999 2024-09-11T06:49:23.874 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.874 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.875 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:23.875 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446854+0000", 2024-09-11T06:49:23.875 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020491500000000001 2024-09-11T06:49:23.875 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.875 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.875 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:23.876 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451380+0000", 2024-09-11T06:49:23.876 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0045264110000000001 2024-09-11T06:49:23.876 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.876 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.876 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:23.876 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451479+0000", 2024-09-11T06:49:23.877 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.8975999999999996e-05 2024-09-11T06:49:23.877 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.877 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.877 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:23.877 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451562+0000", 2024-09-11T06:49:23.877 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3213000000000004e-05 2024-09-11T06:49:23.878 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.878 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.878 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:23.878 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451626+0000", 2024-09-11T06:49:23.878 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3331999999999996e-05 2024-09-11T06:49:23.878 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.878 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:23.878 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.879 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.879 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.879 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:23.879 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:49:23.879 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.389680622, 2024-09-11T06:49:23.879 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011252866, 2024-09-11T06:49:23.879 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:23.879 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:23.880 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:23.880 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:23.880 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:23.880 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:23.880 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 110 2024-09-11T06:49:23.880 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.880 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:23.880 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.881 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:23.881 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:49:23.881 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.881 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.881 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.881 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:23.881 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:49:23.882 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.882 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.882 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.882 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:23.882 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440640+0000", 2024-09-11T06:49:23.882 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5430000000000001e-06 2024-09-11T06:49:23.883 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.883 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.883 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:23.883 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440651+0000", 2024-09-11T06:49:23.883 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0672999999999999e-05 2024-09-11T06:49:23.883 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.884 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.884 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:23.884 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440670+0000", 2024-09-11T06:49:23.884 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9726999999999999e-05 2024-09-11T06:49:23.884 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.884 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.885 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:23.885 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440691+0000", 2024-09-11T06:49:23.885 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0526e-05 2024-09-11T06:49:23.885 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.885 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.885 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:23.885 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448191+0000", 2024-09-11T06:49:23.885 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0075005990000000002 2024-09-11T06:49:23.885 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.885 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.886 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:23.886 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448390+0000", 2024-09-11T06:49:23.886 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000198815 2024-09-11T06:49:23.886 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.886 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.886 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:23.886 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448735+0000", 2024-09-11T06:49:23.886 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034424999999999999 2024-09-11T06:49:23.886 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.886 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.886 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:23.886 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451742+0000", 2024-09-11T06:49:23.886 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030077440000000001 2024-09-11T06:49:23.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.887 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.887 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:23.887 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451761+0000", 2024-09-11T06:49:23.887 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9196999999999999e-05 2024-09-11T06:49:23.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.887 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.887 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:23.887 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451834+0000", 2024-09-11T06:49:23.887 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2071999999999997e-05 2024-09-11T06:49:23.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.887 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.887 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:23.888 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451891+0000", 2024-09-11T06:49:23.888 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7720000000000003e-05 2024-09-11T06:49:23.888 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.888 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:23.888 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.888 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.888 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.888 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:23.888 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:49:23.888 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.389572746999999, 2024-09-11T06:49:23.888 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014944941, 2024-09-11T06:49:23.888 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:23.888 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:23.889 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:23.889 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:23.889 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:23.889 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:23.889 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 111 2024-09-11T06:49:23.889 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.889 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:23.889 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.889 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:23.889 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:49:23.889 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.889 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.889 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.889 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:23.890 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:49:23.890 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.890 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.890 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.890 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:23.890 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440748+0000", 2024-09-11T06:49:23.890 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:49:23.890 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.890 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.890 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:23.890 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440760+0000", 2024-09-11T06:49:23.890 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1897e-05 2024-09-11T06:49:23.890 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.891 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.891 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:23.891 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440782+0000", 2024-09-11T06:49:23.891 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2552999999999999e-05 2024-09-11T06:49:23.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.891 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.891 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:23.891 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440793+0000", 2024-09-11T06:49:23.891 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0643e-05 2024-09-11T06:49:23.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.891 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.891 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:23.891 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449755+0000", 2024-09-11T06:49:23.891 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089622620000000004 2024-09-11T06:49:23.892 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.892 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.892 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:23.892 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449918+0000", 2024-09-11T06:49:23.892 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163086 2024-09-11T06:49:23.892 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.892 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.892 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:23.892 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450150+0000", 2024-09-11T06:49:23.892 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023204500000000001 2024-09-11T06:49:23.892 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.892 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.892 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:23.893 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454533+0000", 2024-09-11T06:49:23.893 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0043831349999999998 2024-09-11T06:49:23.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.893 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.893 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:23.893 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455511+0000", 2024-09-11T06:49:23.893 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0009776050000000001 2024-09-11T06:49:23.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.893 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.893 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:23.893 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455597+0000", 2024-09-11T06:49:23.893 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6446000000000005e-05 2024-09-11T06:49:23.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.894 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.894 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:23.894 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455691+0000", 2024-09-11T06:49:23.894 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3832000000000006e-05 2024-09-11T06:49:23.894 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.894 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:23.894 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.894 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.894 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:23.895 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:49:23.895 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.389502984, 2024-09-11T06:49:23.895 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018885411000000001, 2024-09-11T06:49:23.895 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:23.895 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:23.895 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:23.896 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:23.896 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:23.896 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:23.896 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 112 2024-09-11T06:49:23.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.896 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:23.896 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.896 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:23.896 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:49:23.896 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.896 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.897 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:23.897 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:49:23.897 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.897 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.897 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.897 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:23.897 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440817+0000", 2024-09-11T06:49:23.897 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:49:23.897 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.897 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.897 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:23.897 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441003+0000", 2024-09-11T06:49:23.897 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018661200000000001 2024-09-11T06:49:23.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.898 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.898 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:23.898 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441027+0000", 2024-09-11T06:49:23.898 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3920000000000001e-05 2024-09-11T06:49:23.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.898 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.898 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:23.898 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441062+0000", 2024-09-11T06:49:23.898 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4663000000000003e-05 2024-09-11T06:49:23.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.898 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.898 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:23.899 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451981+0000", 2024-09-11T06:49:23.899 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010918926000000001 2024-09-11T06:49:23.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.899 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:23.899 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452111+0000", 2024-09-11T06:49:23.899 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129529 2024-09-11T06:49:23.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.899 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:23.899 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452311+0000", 2024-09-11T06:49:23.899 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020013799999999999 2024-09-11T06:49:23.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.900 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:23.900 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455809+0000", 2024-09-11T06:49:23.900 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003498443 2024-09-11T06:49:23.900 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.900 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.900 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:23.900 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459566+0000", 2024-09-11T06:49:23.900 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003757271 2024-09-11T06:49:23.900 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.900 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.900 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:23.901 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459640+0000", 2024-09-11T06:49:23.901 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3949000000000006e-05 2024-09-11T06:49:23.901 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.901 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.901 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:23.901 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459701+0000", 2024-09-11T06:49:23.901 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1117000000000005e-05 2024-09-11T06:49:23.901 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.901 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:23.901 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.901 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.901 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.902 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:23.902 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:49:23.902 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.389224132999999, 2024-09-11T06:49:23.902 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018760915, 2024-09-11T06:49:23.902 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:23.902 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:23.902 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:23.902 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:23.902 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:23.903 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:23.903 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 113 2024-09-11T06:49:23.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.903 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:23.903 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.903 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:23.903 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:49:23.903 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.903 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.903 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:23.903 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:49:23.903 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.904 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.904 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:23.904 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441096+0000", 2024-09-11T06:49:23.904 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4130000000000001e-06 2024-09-11T06:49:23.904 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.904 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.904 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:23.904 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441110+0000", 2024-09-11T06:49:23.904 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3900000000000001e-05 2024-09-11T06:49:23.904 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.904 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.904 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:23.904 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441131+0000", 2024-09-11T06:49:23.904 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0636999999999999e-05 2024-09-11T06:49:23.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.905 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.905 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:23.905 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441147+0000", 2024-09-11T06:49:23.905 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6512999999999999e-05 2024-09-11T06:49:23.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.905 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.905 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:23.905 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453274+0000", 2024-09-11T06:49:23.905 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012126764999999999 2024-09-11T06:49:23.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.905 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.905 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:23.906 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453498+0000", 2024-09-11T06:49:23.906 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000223568 2024-09-11T06:49:23.906 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.906 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.906 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:23.906 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453872+0000", 2024-09-11T06:49:23.906 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037467099999999998 2024-09-11T06:49:23.907 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.907 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.907 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:23.907 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458709+0000", 2024-09-11T06:49:23.907 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0048365680000000003 2024-09-11T06:49:23.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.908 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.908 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:23.908 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459716+0000", 2024-09-11T06:49:23.908 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010075240000000001 2024-09-11T06:49:23.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.908 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.908 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:23.908 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459779+0000", 2024-09-11T06:49:23.908 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2336999999999996e-05 2024-09-11T06:49:23.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.908 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.909 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:23.909 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459856+0000", 2024-09-11T06:49:23.909 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7019000000000002e-05 2024-09-11T06:49:23.909 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.909 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:23.909 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.909 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.909 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.909 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:23.910 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:23.910 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.389147393999998, 2024-09-11T06:49:23.910 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018963114, 2024-09-11T06:49:23.910 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:23.910 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:23.910 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:23.910 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:23.910 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:23.910 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:23.910 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 114 2024-09-11T06:49:23.910 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.911 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:23.911 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.911 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:23.911 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:23.911 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.911 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.911 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.911 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:23.911 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:23.911 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.911 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.911 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.911 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:23.912 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:23.912 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9299999999999996e-07 2024-09-11T06:49:23.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.912 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.912 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:23.912 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441180+0000", 2024-09-11T06:49:23.912 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7200000000000006e-06 2024-09-11T06:49:23.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.912 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.912 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:23.912 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441265+0000", 2024-09-11T06:49:23.912 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4753000000000007e-05 2024-09-11T06:49:23.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.913 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.913 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:23.913 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441288+0000", 2024-09-11T06:49:23.913 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2860000000000001e-05 2024-09-11T06:49:23.913 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.913 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.913 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:23.913 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454585+0000", 2024-09-11T06:49:23.913 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013297286 2024-09-11T06:49:23.913 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.913 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.913 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:23.913 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454714+0000", 2024-09-11T06:49:23.913 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000128465 2024-09-11T06:49:23.914 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.914 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.914 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:23.914 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454909+0000", 2024-09-11T06:49:23.914 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019522499999999999 2024-09-11T06:49:23.914 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.914 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.914 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:23.914 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459868+0000", 2024-09-11T06:49:23.914 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0049591239999999997 2024-09-11T06:49:23.914 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.914 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.914 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:23.914 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459988+0000", 2024-09-11T06:49:23.915 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000120532 2024-09-11T06:49:23.915 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.915 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.915 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:23.915 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460071+0000", 2024-09-11T06:49:23.915 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2256000000000007e-05 2024-09-11T06:49:23.915 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.915 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.915 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:23.915 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460135+0000", 2024-09-11T06:49:23.915 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3999999999999997e-05 2024-09-11T06:49:23.915 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.915 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:23.915 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.916 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.916 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.916 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:23.916 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:49:23.916 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.388989765000002, 2024-09-11T06:49:23.916 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.019036849000000002, 2024-09-11T06:49:23.916 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:23.916 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:23.916 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:23.916 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:23.917 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:23.917 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:23.917 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 115 2024-09-11T06:49:23.917 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.917 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:23.917 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.917 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:23.917 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:49:23.917 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.917 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.917 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.917 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:23.918 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:49:23.918 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.918 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.918 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.918 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:23.918 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441330+0000", 2024-09-11T06:49:23.919 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1930000000000001e-06 2024-09-11T06:49:23.919 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.919 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.919 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:23.919 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441544+0000", 2024-09-11T06:49:23.919 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021405799999999999 2024-09-11T06:49:23.920 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.920 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.920 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:23.920 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441572+0000", 2024-09-11T06:49:23.920 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7317e-05 2024-09-11T06:49:23.920 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.921 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.921 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:23.921 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441593+0000", 2024-09-11T06:49:23.921 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1702999999999999e-05 2024-09-11T06:49:23.921 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.921 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.922 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:23.922 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455891+0000", 2024-09-11T06:49:23.922 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01429708 2024-09-11T06:49:23.922 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.922 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.922 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:23.922 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456046+0000", 2024-09-11T06:49:23.923 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000155682 2024-09-11T06:49:23.923 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.923 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.923 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:23.923 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456242+0000", 2024-09-11T06:49:23.923 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000195692 2024-09-11T06:49:23.924 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.924 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.924 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:23.924 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459876+0000", 2024-09-11T06:49:23.924 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036336480000000002 2024-09-11T06:49:23.924 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.924 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.925 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:23.925 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460245+0000", 2024-09-11T06:49:23.925 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036901700000000002 2024-09-11T06:49:23.925 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.925 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.925 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:23.926 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460316+0000", 2024-09-11T06:49:23.926 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1846000000000003e-05 2024-09-11T06:49:23.926 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.926 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.926 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:23.926 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460366+0000", 2024-09-11T06:49:23.927 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9613000000000001e-05 2024-09-11T06:49:23.927 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.927 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:23.927 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.927 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.927 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.927 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:23.927 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:49:23.928 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.388667685000001, 2024-09-11T06:49:23.928 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01904432, 2024-09-11T06:49:23.928 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:23.928 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:23.928 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:23.928 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:23.928 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:23.929 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:23.929 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 116 2024-09-11T06:49:23.929 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.929 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:23.929 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.929 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:23.929 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:49:23.930 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.930 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.930 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:23.930 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:49:23.930 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.930 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.931 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:23.931 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441653+0000", 2024-09-11T06:49:23.931 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:49:23.931 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.931 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.931 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:23.932 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441665+0000", 2024-09-11T06:49:23.932 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1653e-05 2024-09-11T06:49:23.932 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.932 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.932 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:23.932 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441684+0000", 2024-09-11T06:49:23.933 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9490000000000001e-05 2024-09-11T06:49:23.933 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.933 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.933 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:23.933 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441699+0000", 2024-09-11T06:49:23.934 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5044e-05 2024-09-11T06:49:23.934 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.934 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.934 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:23.934 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457297+0000", 2024-09-11T06:49:23.934 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015598241000000001 2024-09-11T06:49:23.935 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.935 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.935 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:23.935 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457577+0000", 2024-09-11T06:49:23.935 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027930799999999998 2024-09-11T06:49:23.935 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.935 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.936 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:23.936 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457957+0000", 2024-09-11T06:49:23.936 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00038088 2024-09-11T06:49:23.936 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.936 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.936 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:23.936 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460432+0000", 2024-09-11T06:49:23.936 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024743349999999998 2024-09-11T06:49:23.936 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.937 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.937 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:23.937 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460584+0000", 2024-09-11T06:49:23.937 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015232900000000001 2024-09-11T06:49:23.937 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.937 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.937 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:23.937 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460646+0000", 2024-09-11T06:49:23.938 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1883000000000004e-05 2024-09-11T06:49:23.938 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.938 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.938 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:23.938 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460696+0000", 2024-09-11T06:49:23.938 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9589999999999998e-05 2024-09-11T06:49:23.939 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.939 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:23.939 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.939 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.939 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.939 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:23.940 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:49:23.940 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.388595619, 2024-09-11T06:49:23.940 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.021834473, 2024-09-11T06:49:23.940 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:23.940 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:23.940 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:23.941 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:23.941 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:23.941 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:23.941 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 117 2024-09-11T06:49:23.941 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.941 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:23.942 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.942 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:23.942 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:49:23.942 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.942 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.942 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.942 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:23.942 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:49:23.942 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.942 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.942 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.942 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:23.942 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441725+0000", 2024-09-11T06:49:23.942 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5069999999999999e-06 2024-09-11T06:49:23.943 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.943 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.943 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:23.943 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441738+0000", 2024-09-11T06:49:23.943 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2813000000000001e-05 2024-09-11T06:49:23.943 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.943 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.944 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:23.944 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441759+0000", 2024-09-11T06:49:23.944 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1246999999999998e-05 2024-09-11T06:49:23.944 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.944 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.944 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:23.944 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441779+0000", 2024-09-11T06:49:23.944 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0392999999999999e-05 2024-09-11T06:49:23.944 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.944 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.944 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:23.944 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458804+0000", 2024-09-11T06:49:23.944 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.017024182999999998 2024-09-11T06:49:23.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.945 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.945 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:23.945 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458977+0000", 2024-09-11T06:49:23.945 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000173926 2024-09-11T06:49:23.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.945 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.945 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:23.945 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459170+0000", 2024-09-11T06:49:23.945 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000192184 2024-09-11T06:49:23.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.945 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.945 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:23.946 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.462300+0000", 2024-09-11T06:49:23.946 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031304969999999999 2024-09-11T06:49:23.946 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.946 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.946 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:23.946 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463408+0000", 2024-09-11T06:49:23.946 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011077839999999999 2024-09-11T06:49:23.946 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.946 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.946 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:23.946 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463497+0000", 2024-09-11T06:49:23.946 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8828999999999999e-05 2024-09-11T06:49:23.946 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.946 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.947 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:23.947 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463558+0000", 2024-09-11T06:49:23.947 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1110000000000003e-05 2024-09-11T06:49:23.947 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.947 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:23.947 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.947 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.947 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.947 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:23.947 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:49:23.947 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.012087931, 2024-09-11T06:49:23.947 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012216898, 2024-09-11T06:49:23.947 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:23.948 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:23.948 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:23.948 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:23.948 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:23.948 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:23.948 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 137 2024-09-11T06:49:23.948 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.948 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:23.948 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.948 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:23.948 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:49:23.948 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.948 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.949 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.949 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:23.949 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:49:23.949 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.949 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.949 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.949 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:23.949 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818232+0000", 2024-09-11T06:49:23.949 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2839999999999999e-06 2024-09-11T06:49:23.949 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.949 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.949 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:23.949 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818326+0000", 2024-09-11T06:49:23.949 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3531999999999998e-05 2024-09-11T06:49:23.950 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.950 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.950 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:23.950 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818337+0000", 2024-09-11T06:49:23.950 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0655999999999999e-05 2024-09-11T06:49:23.950 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.950 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.950 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:23.950 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818342+0000", 2024-09-11T06:49:23.950 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4600000000000002e-06 2024-09-11T06:49:23.950 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.950 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.950 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:23.950 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823890+0000", 2024-09-11T06:49:23.950 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0055480700000000004 2024-09-11T06:49:23.951 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.951 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.951 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:23.951 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823953+0000", 2024-09-11T06:49:23.951 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2848999999999994e-05 2024-09-11T06:49:23.951 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.951 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.951 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:23.951 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824074+0000", 2024-09-11T06:49:23.951 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121309 2024-09-11T06:49:23.951 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.951 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.951 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:23.952 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828043+0000", 2024-09-11T06:49:23.952 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0039685240000000002 2024-09-11T06:49:23.952 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.952 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.952 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:23.952 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830293+0000", 2024-09-11T06:49:23.952 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0022503179999999998 2024-09-11T06:49:23.953 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.953 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.953 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:23.953 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830366+0000", 2024-09-11T06:49:23.953 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2539999999999993e-05 2024-09-11T06:49:23.953 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.953 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.953 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:23.953 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830448+0000", 2024-09-11T06:49:23.953 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2355999999999996e-05 2024-09-11T06:49:23.953 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.953 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:23.954 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.954 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.954 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.954 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:23.954 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:23.954 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.011954116999998, 2024-09-11T06:49:23.954 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013136297, 2024-09-11T06:49:23.954 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:23.954 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:23.954 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:23.954 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:23.954 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:23.954 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:23.954 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 138 2024-09-11T06:49:23.955 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.955 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:23.955 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.955 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:23.955 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:23.955 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.955 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.955 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.955 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:23.955 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:23.955 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.955 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.955 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.956 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:23.956 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:23.956 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.89e-07 2024-09-11T06:49:23.956 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.956 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.956 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:23.956 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818369+0000", 2024-09-11T06:49:23.956 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4170000000000001e-06 2024-09-11T06:49:23.956 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.956 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.956 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:23.956 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818376+0000", 2024-09-11T06:49:23.956 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4270000000000002e-06 2024-09-11T06:49:23.957 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.957 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.957 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:23.957 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818383+0000", 2024-09-11T06:49:23.957 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3060000000000003e-06 2024-09-11T06:49:23.957 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.957 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.957 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:23.957 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824783+0000", 2024-09-11T06:49:23.957 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064003020000000001 2024-09-11T06:49:23.957 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.957 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.957 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:23.958 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825013+0000", 2024-09-11T06:49:23.958 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023039499999999999 2024-09-11T06:49:23.958 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.958 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.958 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:23.958 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825484+0000", 2024-09-11T06:49:23.958 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000470562 2024-09-11T06:49:23.958 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.958 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.958 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:23.959 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828633+0000", 2024-09-11T06:49:23.959 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031496689999999999 2024-09-11T06:49:23.959 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.959 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.959 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:23.959 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831413+0000", 2024-09-11T06:49:23.959 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027793219999999999 2024-09-11T06:49:23.959 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.959 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.959 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:23.959 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831450+0000", 2024-09-11T06:49:23.959 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7370999999999998e-05 2024-09-11T06:49:23.959 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.959 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.960 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:23.960 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831501+0000", 2024-09-11T06:49:23.960 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1036999999999999e-05 2024-09-11T06:49:23.960 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.960 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:23.960 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.960 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.960 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.960 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:23.960 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:49:23.960 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.011929637000001, 2024-09-11T06:49:23.960 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013223011, 2024-09-11T06:49:23.961 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:23.961 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:23.961 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:23.961 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:23.961 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:23.961 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:23.961 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 139 2024-09-11T06:49:23.961 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.961 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:23.961 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.961 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:23.961 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:49:23.961 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.962 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.962 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.962 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:23.962 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:49:23.962 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.962 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.962 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.962 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:23.962 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818390+0000", 2024-09-11T06:49:23.962 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5600000000000001e-07 2024-09-11T06:49:23.962 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.963 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.963 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:23.963 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818394+0000", 2024-09-11T06:49:23.963 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5929999999999999e-06 2024-09-11T06:49:23.963 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.963 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.964 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:23.964 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818399+0000", 2024-09-11T06:49:23.964 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7740000000000004e-06 2024-09-11T06:49:23.964 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.964 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.964 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:23.964 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818412+0000", 2024-09-11T06:49:23.964 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2625e-05 2024-09-11T06:49:23.964 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.965 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.965 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:23.965 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826205+0000", 2024-09-11T06:49:23.965 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0077930730000000002 2024-09-11T06:49:23.965 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.965 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.965 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:23.965 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826281+0000", 2024-09-11T06:49:23.966 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5876000000000003e-05 2024-09-11T06:49:23.966 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.966 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.966 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:23.966 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826384+0000", 2024-09-11T06:49:23.966 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103838 2024-09-11T06:49:23.966 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.966 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.966 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:23.966 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830642+0000", 2024-09-11T06:49:23.966 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004257083 2024-09-11T06:49:23.966 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.966 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.967 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:23.967 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831509+0000", 2024-09-11T06:49:23.967 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086730900000000003 2024-09-11T06:49:23.967 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.967 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.967 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:23.967 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831547+0000", 2024-09-11T06:49:23.967 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7793000000000002e-05 2024-09-11T06:49:23.967 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.967 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.967 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:23.968 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831612+0000", 2024-09-11T06:49:23.968 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5691000000000005e-05 2024-09-11T06:49:23.968 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.968 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:23.968 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.968 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.968 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.968 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:23.968 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:23.968 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.011900943000001, 2024-09-11T06:49:23.968 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016098024999999998, 2024-09-11T06:49:23.968 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:23.968 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:23.969 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:23.969 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:23.969 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:23.969 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:23.969 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 140 2024-09-11T06:49:23.969 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.969 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:23.969 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.969 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:23.969 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:23.969 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.969 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.969 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.970 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:23.970 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:23.970 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.970 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.970 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.970 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:23.970 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:23.970 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5199999999999998e-07 2024-09-11T06:49:23.970 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.970 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.970 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:23.970 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818479+0000", 2024-09-11T06:49:23.970 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0368000000000002e-05 2024-09-11T06:49:23.971 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.971 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.971 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:23.971 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818487+0000", 2024-09-11T06:49:23.971 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2339999999999994e-06 2024-09-11T06:49:23.971 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.971 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.971 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:23.971 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818491+0000", 2024-09-11T06:49:23.971 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4329999999999997e-06 2024-09-11T06:49:23.971 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.971 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.971 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:23.971 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828080+0000", 2024-09-11T06:49:23.971 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0095890449999999992 2024-09-11T06:49:23.972 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.972 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.972 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:23.972 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828130+0000", 2024-09-11T06:49:23.972 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9239000000000001e-05 2024-09-11T06:49:23.972 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.972 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.972 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:23.972 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828208+0000", 2024-09-11T06:49:23.972 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.852e-05 2024-09-11T06:49:23.972 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.972 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.972 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:23.973 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831625+0000", 2024-09-11T06:49:23.973 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034169410000000002 2024-09-11T06:49:23.973 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.973 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.973 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:23.973 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834411+0000", 2024-09-11T06:49:23.973 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002786111 2024-09-11T06:49:23.973 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.973 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.973 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:23.973 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834457+0000", 2024-09-11T06:49:23.973 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6041000000000002e-05 2024-09-11T06:49:23.973 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.974 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.974 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:23.974 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834516+0000", 2024-09-11T06:49:23.974 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8740999999999999e-05 2024-09-11T06:49:23.974 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.974 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:23.974 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.974 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.974 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.974 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:23.974 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:49:23.975 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.011819876000001, 2024-09-11T06:49:23.975 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016588772000000002, 2024-09-11T06:49:23.975 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:23.975 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:23.975 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:23.975 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:23.976 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:23.976 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:23.976 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 141 2024-09-11T06:49:23.976 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.976 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:23.976 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.977 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:23.977 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:49:23.977 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.977 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.977 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.977 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:23.977 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:49:23.978 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.978 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.978 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.978 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:23.978 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818500+0000", 2024-09-11T06:49:23.978 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0800000000000005e-07 2024-09-11T06:49:23.978 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.979 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.979 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:23.979 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818505+0000", 2024-09-11T06:49:23.979 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2429999999999996e-06 2024-09-11T06:49:23.979 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.979 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.979 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:23.980 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818510+0000", 2024-09-11T06:49:23.980 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9339999999999997e-06 2024-09-11T06:49:23.980 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.980 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.980 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:23.980 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818514+0000", 2024-09-11T06:49:23.981 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7270000000000001e-06 2024-09-11T06:49:23.981 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.981 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.981 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:23.981 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828711+0000", 2024-09-11T06:49:23.981 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010197031 2024-09-11T06:49:23.982 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.982 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.982 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:23.982 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828853+0000", 2024-09-11T06:49:23.982 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014201200000000001 2024-09-11T06:49:23.982 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.983 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.983 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:23.983 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.829115+0000", 2024-09-11T06:49:23.983 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026257500000000003 2024-09-11T06:49:23.983 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.983 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.983 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:23.984 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831629+0000", 2024-09-11T06:49:23.984 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025138169999999998 2024-09-11T06:49:23.984 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.984 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.984 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:23.984 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834991+0000", 2024-09-11T06:49:23.984 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033624459999999998 2024-09-11T06:49:23.984 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.985 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.985 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:23.985 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835032+0000", 2024-09-11T06:49:23.985 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1001000000000003e-05 2024-09-11T06:49:23.985 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.985 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.985 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:23.986 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835088+0000", 2024-09-11T06:49:23.986 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5478e-05 2024-09-11T06:49:23.986 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.986 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:23.986 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.986 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.986 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.987 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:49:23.987 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:49:23.987 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 16.388184973000001, 2024-09-11T06:49:23.987 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011161912, 2024-09-11T06:49:23.987 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:23.987 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:23.987 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:23.988 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:23.988 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:23.988 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:23.988 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 158 2024-09-11T06:49:23.988 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.989 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:23.989 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.989 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:23.989 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:49:23.989 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.989 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.990 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.990 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:23.990 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:49:23.990 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:23.990 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.991 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.991 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:23.991 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442135+0000", 2024-09-11T06:49:23.991 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3e-07 2024-09-11T06:49:23.991 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.991 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.991 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:23.992 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442259+0000", 2024-09-11T06:49:23.992 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000123903 2024-09-11T06:49:23.992 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.992 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.992 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:23.992 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442289+0000", 2024-09-11T06:49:23.992 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0833000000000001e-05 2024-09-11T06:49:23.992 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.993 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.993 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:23.993 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442301+0000", 2024-09-11T06:49:23.993 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1433e-05 2024-09-11T06:49:23.993 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.993 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.993 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:23.993 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449796+0000", 2024-09-11T06:49:23.994 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0074954959999999999 2024-09-11T06:49:23.994 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.994 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.994 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:23.994 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449917+0000", 2024-09-11T06:49:23.994 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121033 2024-09-11T06:49:23.994 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.994 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.995 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:23.995 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.450166+0000", 2024-09-11T06:49:23.995 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00024832400000000002 2024-09-11T06:49:23.995 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.995 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.995 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:23.996 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.452630+0000", 2024-09-11T06:49:23.996 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002464188 2024-09-11T06:49:23.996 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.996 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.996 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:23.996 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453149+0000", 2024-09-11T06:49:23.997 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00051860999999999997 2024-09-11T06:49:23.997 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.997 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.997 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:23.997 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453234+0000", 2024-09-11T06:49:23.997 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.5865000000000005e-05 2024-09-11T06:49:23.998 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.998 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.998 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:23.998 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453296+0000", 2024-09-11T06:49:23.998 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1496999999999998e-05 2024-09-11T06:49:23.998 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.999 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:23.999 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:23.999 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:23.999 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:23.999 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:49:23.999 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:49:23.999 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 16.387998871000001, 2024-09-11T06:49:23.999 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011561900999999999, 2024-09-11T06:49:23.999 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:23.999 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:23.999 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:23.999 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:23.999 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:24.000 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:24.000 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 159 2024-09-11T06:49:24.000 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:24.000 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:24.000 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:24.000 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:24.000 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:49:24.000 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:24.000 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:24.000 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:24.001 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:24.001 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:49:24.001 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:24.001 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:24.001 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:24.001 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:24.001 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442321+0000", 2024-09-11T06:49:24.001 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8400000000000003e-07 2024-09-11T06:49:24.001 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:24.001 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:24.001 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:24.001 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442328+0000", 2024-09-11T06:49:24.001 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5130000000000002e-06 2024-09-11T06:49:24.002 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:24.002 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:24.002 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:24.002 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442341+0000", 2024-09-11T06:49:24.002 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.363e-05 2024-09-11T06:49:24.002 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:24.002 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:24.002 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:24.002 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442349+0000", 2024-09-11T06:49:24.002 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4760000000000001e-06 2024-09-11T06:49:24.002 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:24.003 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:24.003 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:24.003 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451257+0000", 2024-09-11T06:49:24.003 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089088569999999992 2024-09-11T06:49:24.003 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:24.003 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:24.003 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:24.003 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451389+0000", 2024-09-11T06:49:24.003 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013110899999999999 2024-09-11T06:49:24.003 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:24.003 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:24.003 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:24.003 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451608+0000", 2024-09-11T06:49:24.004 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000219588 2024-09-11T06:49:24.004 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:24.004 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:24.004 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:24.004 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453517+0000", 2024-09-11T06:49:24.004 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0019088779999999999 2024-09-11T06:49:24.004 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:24.004 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:24.004 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:24.004 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453727+0000", 2024-09-11T06:49:24.004 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020955100000000001 2024-09-11T06:49:24.004 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:24.004 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:24.005 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:24.005 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453815+0000", 2024-09-11T06:49:24.005 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8858999999999997e-05 2024-09-11T06:49:24.005 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:24.005 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:24.005 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:24.005 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453882+0000", 2024-09-11T06:49:24.005 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.6556000000000004e-05 2024-09-11T06:49:24.005 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:24.005 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:24.005 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:24.005 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:24.005 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:24.006 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:24.006 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:49:24.007 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:49:24.007 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:49:24.008 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:49:24.008 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:49:24.009 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:49:24.009 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:49:24.010 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:49:24.010 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:49:24.011 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:49:24.011 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:49:24.011 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:49:24.012 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:49:24.012 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:24.006+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:24.108 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:24.106+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:24.181 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:24.181 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:24.195 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:49:24.209 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:24.206+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:24.309 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:24.306+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:24.364 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:24.364 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:24.378 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:49:24.410 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:24.406+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:24.511 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:24.510+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:24.545 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:24.545 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:24.559 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:49:24.612 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:24.610+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:24.677 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:24.678 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:24.691 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:49:24.713 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:24.710+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:24.814 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:24.810+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:24.861 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:24.861 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:24.875 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:49:24.915 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:24.914+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:24.993 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:24.993 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:25.007 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:49:25.016 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:25.014+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:25.116 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:25.118+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:25.137 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:25.137 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:25.151 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:49:25.217 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:25.214+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:25.270 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:25.271 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:25.284 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:49:25.294 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T06:49:25.294 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T06:49:25.294 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-11T06:49:25.294 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 0.120411 2024-09-11T06:49:25.294 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 0.12041139652256294 2024-09-11T06:49:25.316 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T06:49:25.317 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:25.314+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:25.418 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:25.414+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:25.450 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:25.450 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:25.471 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:49:25.520 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:25.518+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:25.621 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:25.622+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:25.629 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:25.629 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:25.642 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:49:25.722 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:25.722+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:25.756 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:25.756 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:25.770 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:49:25.823 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:25.822+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:25.884 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:25.884 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:25.896 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:49:25.939 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:25.938+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:26.013 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:49:26.057 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:26.057 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:26.058 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:26.054+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:26.071 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:49:26.152 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:49:26.152 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:49:26.152 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:26.150+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:26.170 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:49:26.170 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:49:26.186 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:49:26.187 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:49:26.207 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:49:26.207 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:49:26.227 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:26.226+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:26.228 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:49:26.228 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:49:26.233 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:26.233 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:26.234 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:49:26.234 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:49:26.234 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:26.247 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:49:26.248 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:49:26.249 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:49:26.295 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is unset 2024-09-11T06:49:26.297 INFO:teuthology.orchestra.run.smithi071.stderr:set osd.0 primary-affinity to 0.120411 (878912) 2024-09-11T06:49:26.328 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:26.326+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:26.370 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:26.370 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:49:26.370 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:49:26.370 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:49:26.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:26.370 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:1 2.3 e16/15 2:c4c92e5a:::rbd_trash:head v 16'1, mlcod=0'0)", 2024-09-11T06:49:26.370 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:49:26.370 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.13480159, 2024-09-11T06:49:26.370 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001234761, 2024-09-11T06:49:26.370 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:26.370 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:26.371 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:26.371 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:26.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:26.371 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:26.371 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:49:26.371 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:26.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:26.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:26.371 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:26.371 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:49:26.371 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:26.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:26.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:26.371 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:26.372 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:49:26.372 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6000000000000003e-07 2024-09-11T06:49:26.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:26.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:26.372 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:26.372 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236296+0000", 2024-09-11T06:49:26.372 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6393999999999999e-05 2024-09-11T06:49:26.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:26.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:26.372 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:26.372 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236317+0000", 2024-09-11T06:49:26.372 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1243e-05 2024-09-11T06:49:26.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:26.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:26.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:26.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236333+0000", 2024-09-11T06:49:26.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.558e-05 2024-09-11T06:49:26.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:26.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:26.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:26.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236435+0000", 2024-09-11T06:49:26.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000102803 2024-09-11T06:49:26.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:26.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:26.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:26.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236474+0000", 2024-09-11T06:49:26.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8383999999999997e-05 2024-09-11T06:49:26.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:26.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:26.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:26.374 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.237432+0000", 2024-09-11T06:49:26.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00095849300000000004 2024-09-11T06:49:26.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:26.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:26.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:26.374 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.237513+0000", 2024-09-11T06:49:26.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1204000000000004e-05 2024-09-11T06:49:26.374 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:26.374 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:26.375 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:26.375 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:26.375 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:26.375 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:26.383 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:49:26.431 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:26.430+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:26.514 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:26.514 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:26.514 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:49:26.514 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:26.528 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:49:26.570 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:26.570+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:26.630 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:26.626+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:26.691 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:26.691 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:26.691 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:49:26.691 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:49:26.691 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:26.706 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:49:26.751 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:26.750+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:26.831 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:26.830+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:26.880 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:26.880 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:49:26.880 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:49:26.880 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:49:26.880 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.880 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:26.880 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:49:26.880 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 20.707348262, 2024-09-11T06:49:26.880 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011420669, 2024-09-11T06:49:26.880 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:26.880 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:26.880 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:26.881 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:26.881 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:26.881 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:26.881 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 37 2024-09-11T06:49:26.881 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.881 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:26.881 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.881 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:26.881 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:49:26.881 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.881 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.881 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.881 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:26.882 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:49:26.882 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.882 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.882 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.882 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:26.882 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170509+0000", 2024-09-11T06:49:26.882 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.13e-06 2024-09-11T06:49:26.882 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.882 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.882 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:26.882 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170521+0000", 2024-09-11T06:49:26.882 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1776999999999999e-05 2024-09-11T06:49:26.882 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.882 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.883 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:26.883 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170538+0000", 2024-09-11T06:49:26.883 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7413000000000001e-05 2024-09-11T06:49:26.883 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.883 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.883 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:26.883 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170570+0000", 2024-09-11T06:49:26.883 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2313e-05 2024-09-11T06:49:26.883 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.883 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.883 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:26.883 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179768+0000", 2024-09-11T06:49:26.883 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0091978010000000002 2024-09-11T06:49:26.883 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.883 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.883 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:26.884 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179951+0000", 2024-09-11T06:49:26.884 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018227900000000001 2024-09-11T06:49:26.884 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.884 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.884 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:26.884 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.180175+0000", 2024-09-11T06:49:26.884 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022432800000000001 2024-09-11T06:49:26.884 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.884 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.884 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:26.884 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181112+0000", 2024-09-11T06:49:26.884 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00093712500000000002 2024-09-11T06:49:26.884 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.885 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.885 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:26.885 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181789+0000", 2024-09-11T06:49:26.885 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00067705100000000002 2024-09-11T06:49:26.885 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.885 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.885 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:26.885 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181871+0000", 2024-09-11T06:49:26.885 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1816000000000004e-05 2024-09-11T06:49:26.885 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.885 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.885 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:26.885 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181929+0000", 2024-09-11T06:49:26.885 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7636000000000002e-05 2024-09-11T06:49:26.885 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.886 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:26.886 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.886 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.886 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.886 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:26.886 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:49:26.886 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 20.606089301000001, 2024-09-11T06:49:26.886 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283336, 2024-09-11T06:49:26.886 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:26.886 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:26.886 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:26.886 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:26.886 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:26.887 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:26.887 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 87 2024-09-11T06:49:26.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.887 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:26.887 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.887 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:26.887 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:49:26.887 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.887 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.887 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:26.887 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:49:26.887 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.887 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.888 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:26.888 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271768+0000", 2024-09-11T06:49:26.888 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7599999999999996e-07 2024-09-11T06:49:26.888 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.888 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.888 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:26.888 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271860+0000", 2024-09-11T06:49:26.888 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2639999999999994e-05 2024-09-11T06:49:26.888 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.888 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.888 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:26.888 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271873+0000", 2024-09-11T06:49:26.888 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3139000000000001e-05 2024-09-11T06:49:26.888 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.888 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.889 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:26.889 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271882+0000", 2024-09-11T06:49:26.889 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2369999999999992e-06 2024-09-11T06:49:26.889 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.889 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.889 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:26.889 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277150+0000", 2024-09-11T06:49:26.889 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052684419999999999 2024-09-11T06:49:26.889 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.889 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.889 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:26.889 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277273+0000", 2024-09-11T06:49:26.889 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012278899999999999 2024-09-11T06:49:26.889 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.890 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.890 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:26.890 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277466+0000", 2024-09-11T06:49:26.890 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193578 2024-09-11T06:49:26.890 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.890 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.890 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:26.890 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.280808+0000", 2024-09-11T06:49:26.890 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003341971 2024-09-11T06:49:26.890 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.890 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.890 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:26.890 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284419+0000", 2024-09-11T06:49:26.890 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003610982 2024-09-11T06:49:26.890 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.891 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.891 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:26.891 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284531+0000", 2024-09-11T06:49:26.891 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011121299999999999 2024-09-11T06:49:26.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.891 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.891 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:26.891 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284600+0000", 2024-09-11T06:49:26.891 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9592999999999996e-05 2024-09-11T06:49:26.891 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.891 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:26.891 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.891 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.891 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:26.892 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:49:26.892 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 20.605959048999999, 2024-09-11T06:49:26.892 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283314, 2024-09-11T06:49:26.892 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:26.892 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:26.892 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:26.892 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:26.892 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:26.892 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:26.892 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 88 2024-09-11T06:49:26.892 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.892 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:26.892 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.892 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:26.892 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:49:26.893 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.893 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.893 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:26.893 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:49:26.893 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.893 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.893 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:26.893 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271898+0000", 2024-09-11T06:49:26.893 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2999999999999999e-07 2024-09-11T06:49:26.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.893 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.893 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:26.894 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271904+0000", 2024-09-11T06:49:26.894 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.767e-06 2024-09-11T06:49:26.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.894 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.894 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:26.894 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271913+0000", 2024-09-11T06:49:26.894 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7200000000000001e-06 2024-09-11T06:49:26.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.894 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.894 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:26.894 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271920+0000", 2024-09-11T06:49:26.894 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7100000000000001e-06 2024-09-11T06:49:26.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.894 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.894 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:26.895 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278684+0000", 2024-09-11T06:49:26.895 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0067642650000000002 2024-09-11T06:49:26.895 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.895 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.895 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:26.895 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278838+0000", 2024-09-11T06:49:26.895 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015371499999999999 2024-09-11T06:49:26.895 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.895 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.895 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:26.895 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.279040+0000", 2024-09-11T06:49:26.895 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020207499999999999 2024-09-11T06:49:26.895 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.895 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.895 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:26.896 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.281448+0000", 2024-09-11T06:49:26.896 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002407686 2024-09-11T06:49:26.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.896 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.896 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:26.896 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284615+0000", 2024-09-11T06:49:26.896 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031675499999999999 2024-09-11T06:49:26.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.896 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.896 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:26.896 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284682+0000", 2024-09-11T06:49:26.896 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7032000000000005e-05 2024-09-11T06:49:26.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.897 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.897 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:26.897 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284730+0000", 2024-09-11T06:49:26.897 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7790000000000002e-05 2024-09-11T06:49:26.897 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.897 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:26.897 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.897 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.897 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.897 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:26.897 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:49:26.897 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 20.438057827000002, 2024-09-11T06:49:26.897 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011114877, 2024-09-11T06:49:26.897 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:26.898 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:26.898 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:26.898 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:26.898 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:26.898 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:26.898 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 107 2024-09-11T06:49:26.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.898 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:26.898 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.898 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:26.898 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:49:26.898 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.899 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:26.899 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:49:26.899 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.899 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:26.899 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439799+0000", 2024-09-11T06:49:26.899 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.16e-07 2024-09-11T06:49:26.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.899 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:26.899 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439806+0000", 2024-09-11T06:49:26.899 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3640000000000004e-06 2024-09-11T06:49:26.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.900 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.900 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:26.900 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439816+0000", 2024-09-11T06:49:26.900 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0485999999999999e-05 2024-09-11T06:49:26.900 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.900 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.900 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:26.900 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439824+0000", 2024-09-11T06:49:26.900 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.6699999999999994e-06 2024-09-11T06:49:26.900 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.900 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.900 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:26.900 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443328+0000", 2024-09-11T06:49:26.900 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035044329999999999 2024-09-11T06:49:26.901 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.901 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.901 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:26.901 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443461+0000", 2024-09-11T06:49:26.901 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013272199999999999 2024-09-11T06:49:26.901 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.901 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.901 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:26.901 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443678+0000", 2024-09-11T06:49:26.901 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021674500000000001 2024-09-11T06:49:26.901 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.901 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.901 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:26.901 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449635+0000", 2024-09-11T06:49:26.902 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0059572119999999999 2024-09-11T06:49:26.902 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.902 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.902 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:26.902 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450750+0000", 2024-09-11T06:49:26.902 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001115654 2024-09-11T06:49:26.902 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.902 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.902 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:26.902 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450832+0000", 2024-09-11T06:49:26.902 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1188999999999998e-05 2024-09-11T06:49:26.902 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.902 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.902 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:26.902 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450913+0000", 2024-09-11T06:49:26.903 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1485999999999999e-05 2024-09-11T06:49:26.903 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.903 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:26.903 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.903 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.903 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:26.903 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:49:26.903 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 20.437466852, 2024-09-11T06:49:26.903 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011236556, 2024-09-11T06:49:26.903 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:26.903 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:26.903 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:26.903 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:26.903 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:26.903 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:26.904 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 109 2024-09-11T06:49:26.904 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.904 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:26.904 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.904 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:26.904 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:49:26.904 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.904 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.904 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.904 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:26.904 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:49:26.904 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.904 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.904 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.905 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:26.905 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440391+0000", 2024-09-11T06:49:26.905 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8360000000000001e-06 2024-09-11T06:49:26.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.905 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.905 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:26.905 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440560+0000", 2024-09-11T06:49:26.905 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000169085 2024-09-11T06:49:26.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.905 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.905 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:26.905 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440585+0000", 2024-09-11T06:49:26.905 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4530000000000001e-05 2024-09-11T06:49:26.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.905 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.905 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:26.906 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440606+0000", 2024-09-11T06:49:26.906 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1277e-05 2024-09-11T06:49:26.906 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.906 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.906 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:26.906 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446479+0000", 2024-09-11T06:49:26.906 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0058734989999999999 2024-09-11T06:49:26.906 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.906 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.906 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:26.906 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446649+0000", 2024-09-11T06:49:26.906 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016948199999999999 2024-09-11T06:49:26.906 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.906 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.906 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:26.907 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446854+0000", 2024-09-11T06:49:26.907 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020491500000000001 2024-09-11T06:49:26.907 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.907 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.907 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:26.907 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451380+0000", 2024-09-11T06:49:26.907 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0045264110000000001 2024-09-11T06:49:26.907 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.907 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.907 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:26.907 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451479+0000", 2024-09-11T06:49:26.907 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.8975999999999996e-05 2024-09-11T06:49:26.907 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.907 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.907 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:26.908 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451562+0000", 2024-09-11T06:49:26.908 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3213000000000004e-05 2024-09-11T06:49:26.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.908 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.908 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:26.908 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451626+0000", 2024-09-11T06:49:26.908 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3331999999999996e-05 2024-09-11T06:49:26.908 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.908 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:26.908 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.908 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.908 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:26.909 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:49:26.909 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 20.437217724, 2024-09-11T06:49:26.909 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011252866, 2024-09-11T06:49:26.909 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:26.909 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:26.909 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:26.909 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:26.909 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:26.909 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:26.909 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 110 2024-09-11T06:49:26.909 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.909 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:26.909 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.909 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:26.910 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:49:26.910 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.910 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.910 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.910 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:26.910 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:49:26.910 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.910 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.910 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.910 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:26.910 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440640+0000", 2024-09-11T06:49:26.910 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5430000000000001e-06 2024-09-11T06:49:26.910 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.910 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.910 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:26.911 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440651+0000", 2024-09-11T06:49:26.911 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0672999999999999e-05 2024-09-11T06:49:26.911 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.911 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.911 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:26.911 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440670+0000", 2024-09-11T06:49:26.911 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9726999999999999e-05 2024-09-11T06:49:26.911 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.911 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.911 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:26.911 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440691+0000", 2024-09-11T06:49:26.911 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0526e-05 2024-09-11T06:49:26.911 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.911 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.911 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:26.911 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448191+0000", 2024-09-11T06:49:26.912 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0075005990000000002 2024-09-11T06:49:26.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.912 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.912 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:26.912 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448390+0000", 2024-09-11T06:49:26.912 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000198815 2024-09-11T06:49:26.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.912 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.912 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:26.912 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448735+0000", 2024-09-11T06:49:26.912 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034424999999999999 2024-09-11T06:49:26.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.912 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.912 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:26.912 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451742+0000", 2024-09-11T06:49:26.913 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030077440000000001 2024-09-11T06:49:26.913 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.913 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.913 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:26.913 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451761+0000", 2024-09-11T06:49:26.913 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9196999999999999e-05 2024-09-11T06:49:26.913 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.913 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.913 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:26.913 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451834+0000", 2024-09-11T06:49:26.913 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2071999999999997e-05 2024-09-11T06:49:26.913 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.914 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.914 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:26.914 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451891+0000", 2024-09-11T06:49:26.914 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7720000000000003e-05 2024-09-11T06:49:26.914 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.914 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:26.914 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.914 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.914 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.914 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:26.914 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:49:26.914 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 20.437109848999999, 2024-09-11T06:49:26.914 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014944941, 2024-09-11T06:49:26.914 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:26.914 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:26.915 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:26.915 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:26.915 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:26.915 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:26.915 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 111 2024-09-11T06:49:26.915 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.915 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:26.915 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.915 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:26.915 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:49:26.915 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.915 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.915 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.915 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:26.916 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:49:26.916 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.916 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.916 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.916 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:26.916 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440748+0000", 2024-09-11T06:49:26.916 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:49:26.916 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.916 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.916 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:26.916 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440760+0000", 2024-09-11T06:49:26.916 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1897e-05 2024-09-11T06:49:26.916 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.916 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.916 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:26.917 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440782+0000", 2024-09-11T06:49:26.917 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2552999999999999e-05 2024-09-11T06:49:26.917 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.917 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.917 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:26.917 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440793+0000", 2024-09-11T06:49:26.917 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0643e-05 2024-09-11T06:49:26.917 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.917 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.917 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:26.917 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449755+0000", 2024-09-11T06:49:26.917 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089622620000000004 2024-09-11T06:49:26.917 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.918 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.918 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:26.918 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449918+0000", 2024-09-11T06:49:26.918 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163086 2024-09-11T06:49:26.918 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.918 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.918 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:26.918 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450150+0000", 2024-09-11T06:49:26.918 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023204500000000001 2024-09-11T06:49:26.918 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.918 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.918 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:26.918 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454533+0000", 2024-09-11T06:49:26.919 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0043831349999999998 2024-09-11T06:49:26.919 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.919 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.919 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:26.919 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455511+0000", 2024-09-11T06:49:26.919 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0009776050000000001 2024-09-11T06:49:26.919 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.919 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.919 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:26.919 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455597+0000", 2024-09-11T06:49:26.919 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6446000000000005e-05 2024-09-11T06:49:26.919 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.919 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.920 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:26.920 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455691+0000", 2024-09-11T06:49:26.920 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3832000000000006e-05 2024-09-11T06:49:26.920 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.920 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:26.920 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.920 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.920 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.920 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:26.920 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:49:26.920 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 20.437040086, 2024-09-11T06:49:26.920 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018885411000000001, 2024-09-11T06:49:26.920 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:26.920 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:26.921 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:26.921 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:26.921 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:26.921 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:26.921 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 112 2024-09-11T06:49:26.921 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.921 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:26.921 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.921 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:26.921 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:49:26.921 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.921 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.921 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.921 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:26.922 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:49:26.922 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.922 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.922 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.922 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:26.922 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440817+0000", 2024-09-11T06:49:26.922 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:49:26.922 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.922 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.922 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:26.922 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441003+0000", 2024-09-11T06:49:26.922 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018661200000000001 2024-09-11T06:49:26.922 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.922 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.922 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:26.923 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441027+0000", 2024-09-11T06:49:26.923 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3920000000000001e-05 2024-09-11T06:49:26.923 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.923 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.923 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:26.923 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441062+0000", 2024-09-11T06:49:26.923 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4663000000000003e-05 2024-09-11T06:49:26.923 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.923 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.923 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:26.923 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451981+0000", 2024-09-11T06:49:26.923 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010918926000000001 2024-09-11T06:49:26.923 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.923 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.923 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:26.923 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452111+0000", 2024-09-11T06:49:26.924 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129529 2024-09-11T06:49:26.924 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.924 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.924 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:26.924 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452311+0000", 2024-09-11T06:49:26.924 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020013799999999999 2024-09-11T06:49:26.924 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.924 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.924 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:26.924 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455809+0000", 2024-09-11T06:49:26.924 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003498443 2024-09-11T06:49:26.924 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.924 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.925 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:26.925 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459566+0000", 2024-09-11T06:49:26.925 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003757271 2024-09-11T06:49:26.925 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.925 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.925 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:26.925 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459640+0000", 2024-09-11T06:49:26.925 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3949000000000006e-05 2024-09-11T06:49:26.925 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.925 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.925 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:26.925 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459701+0000", 2024-09-11T06:49:26.925 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1117000000000005e-05 2024-09-11T06:49:26.925 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.925 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:26.926 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.926 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.926 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.926 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:26.926 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:49:26.926 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 20.436761234999999, 2024-09-11T06:49:26.926 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018760915, 2024-09-11T06:49:26.926 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:26.926 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:26.926 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:26.926 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:26.926 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:26.926 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:26.926 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 113 2024-09-11T06:49:26.926 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.927 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:26.927 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.927 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:26.927 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:49:26.927 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.927 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.927 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.927 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:26.927 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:49:26.927 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.927 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.927 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.927 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:26.927 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441096+0000", 2024-09-11T06:49:26.928 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4130000000000001e-06 2024-09-11T06:49:26.928 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.928 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.928 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:26.928 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441110+0000", 2024-09-11T06:49:26.928 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3900000000000001e-05 2024-09-11T06:49:26.928 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.928 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.928 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:26.928 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441131+0000", 2024-09-11T06:49:26.928 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0636999999999999e-05 2024-09-11T06:49:26.928 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.928 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.928 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:26.929 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441147+0000", 2024-09-11T06:49:26.929 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6512999999999999e-05 2024-09-11T06:49:26.929 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.929 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.929 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:26.929 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453274+0000", 2024-09-11T06:49:26.929 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012126764999999999 2024-09-11T06:49:26.929 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.929 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.929 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:26.929 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453498+0000", 2024-09-11T06:49:26.929 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000223568 2024-09-11T06:49:26.929 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.929 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.930 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:26.930 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453872+0000", 2024-09-11T06:49:26.930 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037467099999999998 2024-09-11T06:49:26.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.930 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.930 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:26.930 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458709+0000", 2024-09-11T06:49:26.930 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0048365680000000003 2024-09-11T06:49:26.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.930 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.930 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:26.930 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459716+0000", 2024-09-11T06:49:26.930 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010075240000000001 2024-09-11T06:49:26.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.930 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.930 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:26.931 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459779+0000", 2024-09-11T06:49:26.931 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2336999999999996e-05 2024-09-11T06:49:26.931 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.931 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.931 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:26.931 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459856+0000", 2024-09-11T06:49:26.931 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7019000000000002e-05 2024-09-11T06:49:26.931 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.931 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:26.931 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.931 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.931 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.931 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:26.931 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:26.931 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 20.436684496000002, 2024-09-11T06:49:26.932 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018963114, 2024-09-11T06:49:26.932 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:26.932 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:26.932 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:26.932 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:26.932 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:26.932 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:26.932 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 114 2024-09-11T06:49:26.932 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.932 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:26.932 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.932 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:26.932 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:26.932 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.933 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.933 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.933 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:26.933 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:26.933 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.933 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.933 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.933 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:26.933 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:26.933 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9299999999999996e-07 2024-09-11T06:49:26.933 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.933 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.934 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:26.934 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441180+0000", 2024-09-11T06:49:26.934 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7200000000000006e-06 2024-09-11T06:49:26.934 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.934 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.934 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:26.934 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441265+0000", 2024-09-11T06:49:26.934 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4753000000000007e-05 2024-09-11T06:49:26.934 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.934 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.934 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:26.934 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441288+0000", 2024-09-11T06:49:26.934 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2860000000000001e-05 2024-09-11T06:49:26.935 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.935 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.935 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:26.935 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454585+0000", 2024-09-11T06:49:26.935 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013297286 2024-09-11T06:49:26.935 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.935 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.935 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:26.935 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454714+0000", 2024-09-11T06:49:26.935 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000128465 2024-09-11T06:49:26.935 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.935 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.935 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:26.935 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454909+0000", 2024-09-11T06:49:26.936 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019522499999999999 2024-09-11T06:49:26.936 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.936 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.936 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:26.936 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459868+0000", 2024-09-11T06:49:26.936 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0049591239999999997 2024-09-11T06:49:26.936 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.936 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.936 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:26.936 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459988+0000", 2024-09-11T06:49:26.936 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000120532 2024-09-11T06:49:26.936 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.936 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.936 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:26.936 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460071+0000", 2024-09-11T06:49:26.937 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2256000000000007e-05 2024-09-11T06:49:26.937 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.937 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.937 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:26.937 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460135+0000", 2024-09-11T06:49:26.937 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3999999999999997e-05 2024-09-11T06:49:26.937 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.937 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:26.937 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.937 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.937 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.937 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:26.937 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:49:26.937 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 20.436526867000001, 2024-09-11T06:49:26.937 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.019036849000000002, 2024-09-11T06:49:26.938 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:26.938 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:26.938 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:26.938 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:26.938 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:26.938 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:26.938 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 115 2024-09-11T06:49:26.938 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.938 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:26.938 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.938 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:26.938 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:49:26.938 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.938 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.938 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.939 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:26.939 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:49:26.939 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.939 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.939 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.939 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:26.939 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441330+0000", 2024-09-11T06:49:26.939 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1930000000000001e-06 2024-09-11T06:49:26.939 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.939 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.939 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:26.939 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441544+0000", 2024-09-11T06:49:26.939 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021405799999999999 2024-09-11T06:49:26.939 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.940 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.940 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:26.940 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441572+0000", 2024-09-11T06:49:26.940 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7317e-05 2024-09-11T06:49:26.940 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.940 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.940 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:26.940 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441593+0000", 2024-09-11T06:49:26.940 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1702999999999999e-05 2024-09-11T06:49:26.940 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.940 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.940 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:26.940 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455891+0000", 2024-09-11T06:49:26.940 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01429708 2024-09-11T06:49:26.940 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.941 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.941 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:26.941 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456046+0000", 2024-09-11T06:49:26.941 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000155682 2024-09-11T06:49:26.941 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.941 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.941 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:26.941 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456242+0000", 2024-09-11T06:49:26.941 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000195692 2024-09-11T06:49:26.941 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.941 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.941 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:26.941 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459876+0000", 2024-09-11T06:49:26.941 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036336480000000002 2024-09-11T06:49:26.942 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.942 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.942 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:26.942 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460245+0000", 2024-09-11T06:49:26.942 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036901700000000002 2024-09-11T06:49:26.942 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.942 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.942 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:26.942 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460316+0000", 2024-09-11T06:49:26.942 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1846000000000003e-05 2024-09-11T06:49:26.942 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.942 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.942 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:26.942 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460366+0000", 2024-09-11T06:49:26.942 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9613000000000001e-05 2024-09-11T06:49:26.942 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.943 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:26.943 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.943 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.943 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.943 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:26.943 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:49:26.943 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 20.436204787000001, 2024-09-11T06:49:26.943 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01904432, 2024-09-11T06:49:26.943 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:26.943 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:26.943 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:26.943 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:26.943 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:26.943 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:26.943 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 116 2024-09-11T06:49:26.944 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.944 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:26.944 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.944 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:26.944 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:49:26.944 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.944 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.944 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.944 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:26.944 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:49:26.944 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.944 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.944 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.944 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:26.945 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441653+0000", 2024-09-11T06:49:26.945 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:49:26.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.945 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.945 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:26.945 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441665+0000", 2024-09-11T06:49:26.945 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1653e-05 2024-09-11T06:49:26.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.945 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.945 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:26.945 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441684+0000", 2024-09-11T06:49:26.945 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9490000000000001e-05 2024-09-11T06:49:26.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.945 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.945 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:26.946 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441699+0000", 2024-09-11T06:49:26.946 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5044e-05 2024-09-11T06:49:26.946 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.946 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.946 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:26.946 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457297+0000", 2024-09-11T06:49:26.946 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015598241000000001 2024-09-11T06:49:26.946 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.946 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.946 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:26.946 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457577+0000", 2024-09-11T06:49:26.946 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027930799999999998 2024-09-11T06:49:26.946 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.946 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.946 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:26.947 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457957+0000", 2024-09-11T06:49:26.947 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00038088 2024-09-11T06:49:26.947 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.947 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.947 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:26.947 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460432+0000", 2024-09-11T06:49:26.947 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024743349999999998 2024-09-11T06:49:26.947 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.947 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.947 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:26.947 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460584+0000", 2024-09-11T06:49:26.947 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015232900000000001 2024-09-11T06:49:26.947 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.947 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.947 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:26.948 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460646+0000", 2024-09-11T06:49:26.948 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1883000000000004e-05 2024-09-11T06:49:26.948 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.948 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.948 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:26.948 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460696+0000", 2024-09-11T06:49:26.948 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9589999999999998e-05 2024-09-11T06:49:26.948 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.948 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:26.948 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.948 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.948 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.948 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:26.948 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:49:26.948 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 20.436132721, 2024-09-11T06:49:26.949 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.021834473, 2024-09-11T06:49:26.949 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:26.949 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:26.949 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:26.949 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:26.949 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:26.949 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:26.949 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 117 2024-09-11T06:49:26.949 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.949 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:26.949 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.949 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:26.949 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:49:26.949 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.950 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.950 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.950 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:26.950 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:49:26.950 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.950 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.950 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.950 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:26.950 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441725+0000", 2024-09-11T06:49:26.950 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5069999999999999e-06 2024-09-11T06:49:26.950 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.950 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.950 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:26.950 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441738+0000", 2024-09-11T06:49:26.950 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2813000000000001e-05 2024-09-11T06:49:26.951 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.951 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.951 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:26.951 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441759+0000", 2024-09-11T06:49:26.951 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1246999999999998e-05 2024-09-11T06:49:26.951 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.951 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.951 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:26.951 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441779+0000", 2024-09-11T06:49:26.951 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0392999999999999e-05 2024-09-11T06:49:26.951 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.951 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.951 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:26.951 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458804+0000", 2024-09-11T06:49:26.951 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.017024182999999998 2024-09-11T06:49:26.952 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.952 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.952 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:26.952 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458977+0000", 2024-09-11T06:49:26.952 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000173926 2024-09-11T06:49:26.952 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.952 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.952 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:26.952 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459170+0000", 2024-09-11T06:49:26.952 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000192184 2024-09-11T06:49:26.952 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.952 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.952 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:26.952 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.462300+0000", 2024-09-11T06:49:26.952 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031304969999999999 2024-09-11T06:49:26.953 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.953 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.953 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:26.953 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463408+0000", 2024-09-11T06:49:26.953 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011077839999999999 2024-09-11T06:49:26.953 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.953 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.953 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:26.953 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463497+0000", 2024-09-11T06:49:26.953 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8828999999999999e-05 2024-09-11T06:49:26.953 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.953 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.953 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:26.953 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463558+0000", 2024-09-11T06:49:26.954 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1110000000000003e-05 2024-09-11T06:49:26.954 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.954 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:26.954 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.954 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.954 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.954 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:26.954 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:49:26.954 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 20.059625033, 2024-09-11T06:49:26.954 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012216898, 2024-09-11T06:49:26.954 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:26.954 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:26.954 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:26.954 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:26.954 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:26.955 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:26.955 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 137 2024-09-11T06:49:26.955 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.955 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:26.955 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.955 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:26.955 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:49:26.955 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.955 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.955 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.955 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:26.955 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:49:26.955 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.955 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.955 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.956 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:26.956 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818232+0000", 2024-09-11T06:49:26.956 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2839999999999999e-06 2024-09-11T06:49:26.956 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.956 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.956 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:26.956 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818326+0000", 2024-09-11T06:49:26.956 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3531999999999998e-05 2024-09-11T06:49:26.956 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.956 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.956 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:26.956 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818337+0000", 2024-09-11T06:49:26.956 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0655999999999999e-05 2024-09-11T06:49:26.956 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.956 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.957 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:26.957 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818342+0000", 2024-09-11T06:49:26.957 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4600000000000002e-06 2024-09-11T06:49:26.957 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.957 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.957 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:26.957 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823890+0000", 2024-09-11T06:49:26.957 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0055480700000000004 2024-09-11T06:49:26.957 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.957 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.957 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:26.957 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823953+0000", 2024-09-11T06:49:26.957 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2848999999999994e-05 2024-09-11T06:49:26.957 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.957 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.958 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:26.958 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824074+0000", 2024-09-11T06:49:26.958 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121309 2024-09-11T06:49:26.958 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.958 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.958 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:26.958 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828043+0000", 2024-09-11T06:49:26.958 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0039685240000000002 2024-09-11T06:49:26.958 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.958 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.958 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:26.958 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830293+0000", 2024-09-11T06:49:26.958 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0022503179999999998 2024-09-11T06:49:26.958 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.958 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.959 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:26.959 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830366+0000", 2024-09-11T06:49:26.959 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2539999999999993e-05 2024-09-11T06:49:26.959 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.959 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.959 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:26.959 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830448+0000", 2024-09-11T06:49:26.959 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2355999999999996e-05 2024-09-11T06:49:26.959 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.959 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:26.959 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.959 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.959 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.959 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:26.959 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:26.960 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 20.059491219000002, 2024-09-11T06:49:26.960 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013136297, 2024-09-11T06:49:26.960 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:26.960 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:26.960 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:26.960 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:26.960 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:26.960 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:26.960 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 138 2024-09-11T06:49:26.960 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.960 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:26.960 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.960 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:26.960 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:26.960 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.961 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.961 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.961 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:26.961 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:26.961 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.961 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.961 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.961 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:26.961 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:26.961 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.89e-07 2024-09-11T06:49:26.961 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.961 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.961 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:26.962 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818369+0000", 2024-09-11T06:49:26.962 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4170000000000001e-06 2024-09-11T06:49:26.962 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.962 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.962 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:26.962 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818376+0000", 2024-09-11T06:49:26.962 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4270000000000002e-06 2024-09-11T06:49:26.962 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.962 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.962 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:26.962 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818383+0000", 2024-09-11T06:49:26.962 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3060000000000003e-06 2024-09-11T06:49:26.962 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.962 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.962 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:26.962 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824783+0000", 2024-09-11T06:49:26.963 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064003020000000001 2024-09-11T06:49:26.963 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.963 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.963 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:26.963 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825013+0000", 2024-09-11T06:49:26.963 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023039499999999999 2024-09-11T06:49:26.963 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.963 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.963 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:26.963 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825484+0000", 2024-09-11T06:49:26.963 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000470562 2024-09-11T06:49:26.963 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.963 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.963 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:26.963 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828633+0000", 2024-09-11T06:49:26.964 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031496689999999999 2024-09-11T06:49:26.964 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.964 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.964 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:26.964 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831413+0000", 2024-09-11T06:49:26.964 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027793219999999999 2024-09-11T06:49:26.964 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.964 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.964 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:26.964 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831450+0000", 2024-09-11T06:49:26.964 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7370999999999998e-05 2024-09-11T06:49:26.964 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.964 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.964 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:26.964 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831501+0000", 2024-09-11T06:49:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1036999999999999e-05 2024-09-11T06:49:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:49:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 20.059466739000001, 2024-09-11T06:49:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013223011, 2024-09-11T06:49:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 139 2024-09-11T06:49:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:49:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:49:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.967 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:26.967 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818390+0000", 2024-09-11T06:49:26.967 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5600000000000001e-07 2024-09-11T06:49:26.967 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.967 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.967 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:26.967 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818394+0000", 2024-09-11T06:49:26.967 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5929999999999999e-06 2024-09-11T06:49:26.967 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.967 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.967 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:26.967 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818399+0000", 2024-09-11T06:49:26.967 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7740000000000004e-06 2024-09-11T06:49:26.967 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.967 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.968 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:26.968 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818412+0000", 2024-09-11T06:49:26.968 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2625e-05 2024-09-11T06:49:26.968 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.968 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.968 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:26.968 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826205+0000", 2024-09-11T06:49:26.968 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0077930730000000002 2024-09-11T06:49:26.968 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.968 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.968 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:26.968 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826281+0000", 2024-09-11T06:49:26.968 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5876000000000003e-05 2024-09-11T06:49:26.968 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.969 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.969 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:26.969 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826384+0000", 2024-09-11T06:49:26.969 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103838 2024-09-11T06:49:26.969 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.969 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.969 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:26.969 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830642+0000", 2024-09-11T06:49:26.969 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004257083 2024-09-11T06:49:26.969 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.970 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.970 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:26.970 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831509+0000", 2024-09-11T06:49:26.970 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086730900000000003 2024-09-11T06:49:26.970 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.970 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.970 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:26.970 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831547+0000", 2024-09-11T06:49:26.970 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7793000000000002e-05 2024-09-11T06:49:26.970 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.970 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.970 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:26.970 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831612+0000", 2024-09-11T06:49:26.971 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5691000000000005e-05 2024-09-11T06:49:26.971 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.971 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:26.971 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.971 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.971 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.971 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:26.971 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:26.971 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 20.059438045, 2024-09-11T06:49:26.971 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016098024999999998, 2024-09-11T06:49:26.971 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:26.971 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:26.971 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:26.972 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:26.972 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:26.972 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:26.972 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 140 2024-09-11T06:49:26.972 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.972 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:26.972 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.972 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:26.972 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:26.972 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.972 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.972 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.972 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:26.972 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:26.973 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.973 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.973 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.973 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:26.973 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:26.973 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5199999999999998e-07 2024-09-11T06:49:26.973 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.973 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.973 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:26.973 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818479+0000", 2024-09-11T06:49:26.973 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0368000000000002e-05 2024-09-11T06:49:26.973 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.973 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.974 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:26.974 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818487+0000", 2024-09-11T06:49:26.974 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2339999999999994e-06 2024-09-11T06:49:26.974 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.974 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.974 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:26.974 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818491+0000", 2024-09-11T06:49:26.974 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4329999999999997e-06 2024-09-11T06:49:26.974 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.974 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.974 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:26.974 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828080+0000", 2024-09-11T06:49:26.974 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0095890449999999992 2024-09-11T06:49:26.974 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.975 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.975 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:26.975 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828130+0000", 2024-09-11T06:49:26.975 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9239000000000001e-05 2024-09-11T06:49:26.975 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.975 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.975 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:26.975 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828208+0000", 2024-09-11T06:49:26.975 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.852e-05 2024-09-11T06:49:26.975 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.975 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.975 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:26.975 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831625+0000", 2024-09-11T06:49:26.975 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034169410000000002 2024-09-11T06:49:26.976 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.976 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.976 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:26.976 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834411+0000", 2024-09-11T06:49:26.976 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002786111 2024-09-11T06:49:26.976 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.976 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.976 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:26.976 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834457+0000", 2024-09-11T06:49:26.977 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6041000000000002e-05 2024-09-11T06:49:26.977 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.977 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.977 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:26.977 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834516+0000", 2024-09-11T06:49:26.977 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8740999999999999e-05 2024-09-11T06:49:26.977 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.977 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:26.977 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.977 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.977 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.977 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:26.977 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:49:26.978 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 20.059356978, 2024-09-11T06:49:26.978 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016588772000000002, 2024-09-11T06:49:26.978 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:26.978 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:26.978 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:26.978 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:26.978 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:26.978 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:26.978 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 141 2024-09-11T06:49:26.978 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.978 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:26.978 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.978 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:26.978 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:49:26.978 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.979 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.979 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.979 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:26.979 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:49:26.979 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.979 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.979 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.979 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:26.979 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818500+0000", 2024-09-11T06:49:26.979 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0800000000000005e-07 2024-09-11T06:49:26.979 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.979 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.979 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:26.979 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818505+0000", 2024-09-11T06:49:26.980 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2429999999999996e-06 2024-09-11T06:49:26.980 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.980 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.980 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:26.980 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818510+0000", 2024-09-11T06:49:26.980 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9339999999999997e-06 2024-09-11T06:49:26.980 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.980 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.980 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:26.980 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818514+0000", 2024-09-11T06:49:26.980 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7270000000000001e-06 2024-09-11T06:49:26.980 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.980 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.980 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:26.981 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828711+0000", 2024-09-11T06:49:26.981 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010197031 2024-09-11T06:49:26.981 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.981 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.981 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:26.981 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828853+0000", 2024-09-11T06:49:26.981 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014201200000000001 2024-09-11T06:49:26.981 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.981 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.981 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:26.981 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.829115+0000", 2024-09-11T06:49:26.981 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026257500000000003 2024-09-11T06:49:26.981 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.981 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.982 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:26.982 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831629+0000", 2024-09-11T06:49:26.982 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025138169999999998 2024-09-11T06:49:26.982 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.982 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.982 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:26.982 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834991+0000", 2024-09-11T06:49:26.982 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033624459999999998 2024-09-11T06:49:26.982 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.982 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.982 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:26.982 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835032+0000", 2024-09-11T06:49:26.982 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1001000000000003e-05 2024-09-11T06:49:26.982 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.983 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.983 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:26.983 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835088+0000", 2024-09-11T06:49:26.983 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5478e-05 2024-09-11T06:49:26.983 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.983 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:26.983 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.983 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.983 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.983 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:49:26.983 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:49:26.983 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 19.435722075000001, 2024-09-11T06:49:26.983 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011161912, 2024-09-11T06:49:26.983 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:26.983 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:26.984 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:26.984 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:26.984 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:26.984 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:26.984 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 158 2024-09-11T06:49:26.984 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.984 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:26.984 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.984 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:26.984 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:49:26.984 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.984 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.984 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.984 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:26.984 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:49:26.985 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.985 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.985 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.985 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:26.985 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442135+0000", 2024-09-11T06:49:26.985 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3e-07 2024-09-11T06:49:26.985 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.985 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.985 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:26.985 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442259+0000", 2024-09-11T06:49:26.985 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000123903 2024-09-11T06:49:26.985 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.985 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.986 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:26.986 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442289+0000", 2024-09-11T06:49:26.986 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0833000000000001e-05 2024-09-11T06:49:26.986 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.986 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.986 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:26.986 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442301+0000", 2024-09-11T06:49:26.986 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1433e-05 2024-09-11T06:49:26.986 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.986 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.986 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:26.986 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449796+0000", 2024-09-11T06:49:26.986 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0074954959999999999 2024-09-11T06:49:26.986 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.986 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.987 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:26.987 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449917+0000", 2024-09-11T06:49:26.987 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121033 2024-09-11T06:49:26.987 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.987 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.987 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:26.987 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.450166+0000", 2024-09-11T06:49:26.987 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00024832400000000002 2024-09-11T06:49:26.987 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.987 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.987 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:26.987 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.452630+0000", 2024-09-11T06:49:26.987 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002464188 2024-09-11T06:49:26.987 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.987 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.988 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:26.988 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453149+0000", 2024-09-11T06:49:26.988 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00051860999999999997 2024-09-11T06:49:26.988 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.988 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.988 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:26.988 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453234+0000", 2024-09-11T06:49:26.988 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.5865000000000005e-05 2024-09-11T06:49:26.988 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.988 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.988 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:26.988 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453296+0000", 2024-09-11T06:49:26.988 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1496999999999998e-05 2024-09-11T06:49:26.988 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.988 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:26.989 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.989 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.989 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.989 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:49:26.989 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:49:26.989 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 19.435535973, 2024-09-11T06:49:26.989 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011561900999999999, 2024-09-11T06:49:26.989 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:26.989 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:26.989 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:26.989 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:26.989 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:26.989 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:26.989 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 159 2024-09-11T06:49:26.989 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.990 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:26.990 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.990 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:26.990 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:49:26.990 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.990 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.990 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.990 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:26.990 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:49:26.990 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:26.990 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.991 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.991 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:26.991 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442321+0000", 2024-09-11T06:49:26.991 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8400000000000003e-07 2024-09-11T06:49:26.991 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.991 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.991 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:26.991 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442328+0000", 2024-09-11T06:49:26.991 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5130000000000002e-06 2024-09-11T06:49:26.991 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.991 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.991 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:26.991 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442341+0000", 2024-09-11T06:49:26.991 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.363e-05 2024-09-11T06:49:26.991 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.992 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.992 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:26.992 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442349+0000", 2024-09-11T06:49:26.992 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4760000000000001e-06 2024-09-11T06:49:26.992 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.992 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.992 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:26.992 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451257+0000", 2024-09-11T06:49:26.992 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089088569999999992 2024-09-11T06:49:26.992 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.992 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.992 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:26.992 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451389+0000", 2024-09-11T06:49:26.992 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013110899999999999 2024-09-11T06:49:26.992 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.993 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.993 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:26.993 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451608+0000", 2024-09-11T06:49:26.993 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000219588 2024-09-11T06:49:26.993 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.993 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.993 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:26.993 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453517+0000", 2024-09-11T06:49:26.993 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0019088779999999999 2024-09-11T06:49:26.993 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.993 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.993 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:26.993 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453727+0000", 2024-09-11T06:49:26.993 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020955100000000001 2024-09-11T06:49:26.993 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.994 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.994 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:26.994 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453815+0000", 2024-09-11T06:49:26.994 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8858999999999997e-05 2024-09-11T06:49:26.994 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:26.994 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:26.994 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:26.994 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453882+0000", 2024-09-11T06:49:26.994 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.6556000000000004e-05 2024-09-11T06:49:26.994 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.994 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:26.994 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.994 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:26.994 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:26.994 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:26.996 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:49:26.996 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:26.994+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:27.096 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:27.094+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:27.165 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:27.165 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:27.166 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:49:27.166 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:27.179 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:49:27.197 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:27.194+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:27.298 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:27.294+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:27.347 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:27.347 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:27.348 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:49:27.348 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:49:27.348 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:27.362 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:49:27.400 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:27.398+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:27.483 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:27.483 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:49:27.483 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:49:27.483 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [] 2024-09-11T06:49:27.483 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:27.497 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:49:27.500 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:27.498+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:27.601 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:27.598+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:27.620 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:27.620 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:27.620 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:49:27.620 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:27.634 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:49:27.702 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:27.702+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:27.751 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:27.751 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:27.751 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:49:27.752 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:49:27.752 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:27.765 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:49:27.815 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:27.814+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:27.889 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:27.889 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:49:27.889 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:49:27.890 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:49:27.890 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.890 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:49:27.890 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:49:27.890 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 16.661059389999998, 2024-09-11T06:49:27.890 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010553425999999999, 2024-09-11T06:49:27.890 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:27.890 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:27.890 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:27.890 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:27.890 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:49:27.890 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:49:27.890 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:49:27.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.891 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:27.891 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.891 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:27.891 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:49:27.891 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:27.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.891 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.891 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:27.891 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:49:27.891 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:27.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.891 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.891 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:27.891 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230274+0000", 2024-09-11T06:49:27.892 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.533e-06 2024-09-11T06:49:27.892 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.892 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.892 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:27.892 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230285+0000", 2024-09-11T06:49:27.892 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0540000000000001e-05 2024-09-11T06:49:27.892 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.892 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.892 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:27.892 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230309+0000", 2024-09-11T06:49:27.892 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4223000000000002e-05 2024-09-11T06:49:27.892 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.892 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.893 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:27.893 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230327+0000", 2024-09-11T06:49:27.893 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:49:27.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.893 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.893 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:27.893 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230478+0000", 2024-09-11T06:49:27.893 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000151459 2024-09-11T06:49:27.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.893 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.893 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:49:27.893 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230490+0000", 2024-09-11T06:49:27.893 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.147e-05 2024-09-11T06:49:27.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.893 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.894 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:27.894 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231803+0000", 2024-09-11T06:49:27.894 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001313158 2024-09-11T06:49:27.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.894 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.894 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:49:27.894 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231817+0000", 2024-09-11T06:49:27.894 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4197e-05 2024-09-11T06:49:27.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.894 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.894 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:27.894 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.235945+0000", 2024-09-11T06:49:27.894 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0041282480000000002 2024-09-11T06:49:27.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.895 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.895 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:27.895 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236312+0000", 2024-09-11T06:49:27.895 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036652700000000001 2024-09-11T06:49:27.895 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.895 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.895 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:49:27.895 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236689+0000", 2024-09-11T06:49:27.895 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000377117 2024-09-11T06:49:27.895 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.895 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.895 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:27.895 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238460+0000", 2024-09-11T06:49:27.896 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001770918 2024-09-11T06:49:27.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.896 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.896 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:27.896 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240627+0000", 2024-09-11T06:49:27.896 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021670779999999998 2024-09-11T06:49:27.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.896 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.896 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:27.896 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240751+0000", 2024-09-11T06:49:27.896 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012384900000000001 2024-09-11T06:49:27.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.896 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.896 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:27.897 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240821+0000", 2024-09-11T06:49:27.897 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0516e-05 2024-09-11T06:49:27.897 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:27.897 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:27.897 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:27.897 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.897 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.897 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop_reply(client.4230.0:1 2.3 e16/15 ondisk, result = 0)", 2024-09-11T06:49:27.897 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.238282+0000", 2024-09-11T06:49:27.897 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 16.653045065000001, 2024-09-11T06:49:27.897 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018970799999999999, 2024-09-11T06:49:27.897 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:27.897 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:27.897 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:49:27.898 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:27.898 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.898 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:27.898 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238282+0000", 2024-09-11T06:49:27.898 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:27.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.898 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.898 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:27.898 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238282+0000", 2024-09-11T06:49:27.898 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:27.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.898 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.898 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:27.898 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238284+0000", 2024-09-11T06:49:27.899 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:49:27.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.899 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:27.899 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238292+0000", 2024-09-11T06:49:27.899 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2390000000000002e-06 2024-09-11T06:49:27.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.899 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:27.899 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238314+0000", 2024-09-11T06:49:27.899 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1776999999999998e-05 2024-09-11T06:49:27.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.900 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:27.900 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238339+0000", 2024-09-11T06:49:27.900 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5119000000000001e-05 2024-09-11T06:49:27.900 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.900 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.900 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:27.900 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238438+0000", 2024-09-11T06:49:27.900 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9553e-05 2024-09-11T06:49:27.900 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.900 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.900 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:27.900 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238451+0000", 2024-09-11T06:49:27.900 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2286000000000001e-05 2024-09-11T06:49:27.900 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.900 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.901 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:27.901 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238472+0000", 2024-09-11T06:49:27.901 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1166999999999999e-05 2024-09-11T06:49:27.901 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:27.901 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:27.901 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:27.901 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.901 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.901 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:49:27.901 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.244627+0000", 2024-09-11T06:49:27.901 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 16.646700367000001, 2024-09-11T06:49:27.901 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00046141999999999998, 2024-09-11T06:49:27.901 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:27.902 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:27.902 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:49:27.902 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:27.902 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:49:27.902 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:49:27.902 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 2 2024-09-11T06:49:27.902 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.902 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:27.902 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.902 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:27.902 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244627+0000", 2024-09-11T06:49:27.902 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:27.902 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.902 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.902 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:27.903 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244627+0000", 2024-09-11T06:49:27.903 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:27.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.903 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.903 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:27.903 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244629+0000", 2024-09-11T06:49:27.903 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.95e-06 2024-09-11T06:49:27.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.903 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.903 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:27.903 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244640+0000", 2024-09-11T06:49:27.903 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1494e-05 2024-09-11T06:49:27.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.903 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.904 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:27.904 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244664+0000", 2024-09-11T06:49:27.904 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4006e-05 2024-09-11T06:49:27.904 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.904 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.904 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:27.904 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244684+0000", 2024-09-11T06:49:27.904 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.961e-05 2024-09-11T06:49:27.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.905 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.905 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:27.905 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244862+0000", 2024-09-11T06:49:27.905 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017863500000000001 2024-09-11T06:49:27.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.905 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.905 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:27.905 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245088+0000", 2024-09-11T06:49:27.905 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000225725 2024-09-11T06:49:27.905 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:27.905 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:27.905 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:27.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.906 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.906 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:49:27.906 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.245492+0000", 2024-09-11T06:49:27.906 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 16.645835424000001, 2024-09-11T06:49:27.906 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00042092600000000001, 2024-09-11T06:49:27.906 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:27.906 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:27.906 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:49:27.906 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:27.906 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:49:27.906 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:49:27.906 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 3 2024-09-11T06:49:27.906 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.907 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:27.907 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.907 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:27.907 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245492+0000", 2024-09-11T06:49:27.907 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:27.907 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.907 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.907 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:27.907 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245492+0000", 2024-09-11T06:49:27.907 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:27.907 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.907 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.907 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:27.907 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245493+0000", 2024-09-11T06:49:27.907 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.657e-06 2024-09-11T06:49:27.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.908 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.908 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:27.908 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245501+0000", 2024-09-11T06:49:27.908 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2490000000000004e-06 2024-09-11T06:49:27.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.908 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.908 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:27.908 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245523+0000", 2024-09-11T06:49:27.908 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2016999999999999e-05 2024-09-11T06:49:27.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.908 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.908 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:27.909 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245548+0000", 2024-09-11T06:49:27.909 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5060000000000001e-05 2024-09-11T06:49:27.909 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.909 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.909 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:27.909 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245720+0000", 2024-09-11T06:49:27.909 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017190200000000001 2024-09-11T06:49:27.909 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.909 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.909 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:27.909 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245913+0000", 2024-09-11T06:49:27.909 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193041 2024-09-11T06:49:27.909 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:27.909 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:27.910 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:27.910 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.910 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.910 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-11T06:49:27.910 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:49:27.910 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 15.658888547, 2024-09-11T06:49:27.910 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069904770000000001, 2024-09-11T06:49:27.910 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:27.910 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:27.910 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:27.910 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:27.910 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:49:27.910 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:49:27.910 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 5 2024-09-11T06:49:27.911 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.911 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:27.911 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.911 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:27.911 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:49:27.911 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:27.911 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.911 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.911 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:27.911 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:49:27.911 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:27.911 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.911 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.911 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:27.911 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232442+0000", 2024-09-11T06:49:27.912 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.044e-06 2024-09-11T06:49:27.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.912 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.912 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:27.912 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232460+0000", 2024-09-11T06:49:27.912 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7863000000000002e-05 2024-09-11T06:49:27.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.912 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.912 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:27.912 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232492+0000", 2024-09-11T06:49:27.912 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2812999999999999e-05 2024-09-11T06:49:27.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.913 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.913 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:27.913 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232510+0000", 2024-09-11T06:49:27.913 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7552999999999999e-05 2024-09-11T06:49:27.913 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.913 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.913 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:27.913 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232640+0000", 2024-09-11T06:49:27.913 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013029900000000001 2024-09-11T06:49:27.913 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.913 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.913 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:49:27.913 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232650+0000", 2024-09-11T06:49:27.914 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0023999999999999e-05 2024-09-11T06:49:27.914 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.914 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.914 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:27.914 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235641+0000", 2024-09-11T06:49:27.914 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00299089 2024-09-11T06:49:27.914 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.914 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.914 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:27.914 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235980+0000", 2024-09-11T06:49:27.914 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000338907 2024-09-11T06:49:27.914 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.914 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.914 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:49:27.914 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.236351+0000", 2024-09-11T06:49:27.915 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037087699999999998 2024-09-11T06:49:27.915 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.915 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.915 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:27.915 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238074+0000", 2024-09-11T06:49:27.915 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001722859 2024-09-11T06:49:27.915 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.915 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.915 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:27.915 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239160+0000", 2024-09-11T06:49:27.915 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001086074 2024-09-11T06:49:27.915 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.915 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.915 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:27.916 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239299+0000", 2024-09-11T06:49:27.916 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013933500000000001 2024-09-11T06:49:27.916 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.916 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.916 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:27.916 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239429+0000", 2024-09-11T06:49:27.916 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129939 2024-09-11T06:49:27.916 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:27.916 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:27.916 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:27.916 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.916 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.916 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop_reply(client.4230.0:5 2.2 e16/15 ondisk, result = 0)", 2024-09-11T06:49:27.916 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:12.238911+0000", 2024-09-11T06:49:27.917 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 15.652416259000001, 2024-09-11T06:49:27.917 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00053624900000000004, 2024-09-11T06:49:27.917 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:27.917 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:27.917 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:49:27.917 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:27.917 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.917 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:27.917 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238911+0000", 2024-09-11T06:49:27.917 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:27.917 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.917 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.917 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:27.917 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238911+0000", 2024-09-11T06:49:27.918 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:27.918 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.918 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.918 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:27.918 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238913+0000", 2024-09-11T06:49:27.918 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2069999999999998e-06 2024-09-11T06:49:27.918 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.918 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.918 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:27.918 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238923+0000", 2024-09-11T06:49:27.918 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0360000000000001e-05 2024-09-11T06:49:27.918 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.918 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.918 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:27.919 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238953+0000", 2024-09-11T06:49:27.919 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9258999999999999e-05 2024-09-11T06:49:27.919 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.919 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.919 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:27.919 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238967+0000", 2024-09-11T06:49:27.919 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3797e-05 2024-09-11T06:49:27.919 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.919 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.919 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:27.919 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239117+0000", 2024-09-11T06:49:27.919 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000150765 2024-09-11T06:49:27.919 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.919 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.919 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:27.920 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239143+0000", 2024-09-11T06:49:27.920 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5514e-05 2024-09-11T06:49:27.920 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.920 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.920 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:27.920 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239447+0000", 2024-09-11T06:49:27.920 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00030434700000000001 2024-09-11T06:49:27.920 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:27.920 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:27.920 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:27.920 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.920 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.920 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-11T06:49:27.921 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:49:27.921 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 14.644800595, 2024-09-11T06:49:27.921 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069253250000000004, 2024-09-11T06:49:27.921 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:27.921 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:27.921 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:27.921 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:27.921 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:49:27.921 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:49:27.921 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 7 2024-09-11T06:49:27.922 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.922 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:27.922 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.922 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:27.922 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:49:27.922 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:27.922 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.922 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.922 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:27.922 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:49:27.922 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:27.922 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.922 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.922 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:27.923 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246530+0000", 2024-09-11T06:49:27.923 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9469999999999999e-06 2024-09-11T06:49:27.923 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.923 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.923 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:27.923 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246545+0000", 2024-09-11T06:49:27.923 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5973000000000001e-05 2024-09-11T06:49:27.923 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.923 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.923 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:27.923 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246578+0000", 2024-09-11T06:49:27.923 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2759999999999998e-05 2024-09-11T06:49:27.923 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.923 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.924 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:27.924 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246595+0000", 2024-09-11T06:49:27.924 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6939000000000002e-05 2024-09-11T06:49:27.924 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.924 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.924 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:27.924 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246770+0000", 2024-09-11T06:49:27.924 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000174723 2024-09-11T06:49:27.924 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.924 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.924 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:49:27.924 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246789+0000", 2024-09-11T06:49:27.924 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9159000000000001e-05 2024-09-11T06:49:27.924 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.925 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.925 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:27.925 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250337+0000", 2024-09-11T06:49:27.925 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035480759999999998 2024-09-11T06:49:27.925 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.925 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.925 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:27.925 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250549+0000", 2024-09-11T06:49:27.925 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021192199999999999 2024-09-11T06:49:27.925 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.925 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.925 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:49:27.925 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250750+0000", 2024-09-11T06:49:27.925 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020104199999999999 2024-09-11T06:49:27.926 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.926 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.926 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:27.926 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.252455+0000", 2024-09-11T06:49:27.926 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001705249 2024-09-11T06:49:27.926 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.926 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.926 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:27.926 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253305+0000", 2024-09-11T06:49:27.926 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00084941200000000002 2024-09-11T06:49:27.926 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.926 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.926 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:27.926 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253388+0000", 2024-09-11T06:49:27.927 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3642999999999999e-05 2024-09-11T06:49:27.927 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.927 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.927 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:27.927 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253452+0000", 2024-09-11T06:49:27.927 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3479999999999995e-05 2024-09-11T06:49:27.927 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:27.927 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:27.927 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:27.927 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.927 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.927 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop_reply(client.4230.0:7 2.2 e18/15 ondisk, result = 0)", 2024-09-11T06:49:27.927 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:13.253128+0000", 2024-09-11T06:49:27.927 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 14.638199021, 2024-09-11T06:49:27.928 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034023700000000002, 2024-09-11T06:49:27.928 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:27.928 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:27.928 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:49:27.928 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:27.928 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.928 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:27.928 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253128+0000", 2024-09-11T06:49:27.928 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:27.928 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.928 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.928 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:27.928 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253128+0000", 2024-09-11T06:49:27.928 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:27.929 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.929 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.929 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:27.929 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253135+0000", 2024-09-11T06:49:27.929 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.657e-06 2024-09-11T06:49:27.929 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.929 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.929 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:27.929 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253144+0000", 2024-09-11T06:49:27.929 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0869999999999996e-06 2024-09-11T06:49:27.929 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.929 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.929 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:27.929 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253167+0000", 2024-09-11T06:49:27.930 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3252e-05 2024-09-11T06:49:27.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.930 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.930 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:27.930 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253186+0000", 2024-09-11T06:49:27.930 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9239999999999999e-05 2024-09-11T06:49:27.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.930 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.930 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:27.930 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253282+0000", 2024-09-11T06:49:27.930 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.6093000000000003e-05 2024-09-11T06:49:27.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.930 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.930 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:27.930 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253294+0000", 2024-09-11T06:49:27.931 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1986e-05 2024-09-11T06:49:27.931 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:27.931 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:27.931 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:27.931 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253468+0000", 2024-09-11T06:49:27.931 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017392199999999999 2024-09-11T06:49:27.931 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:27.931 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:27.931 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:27.931 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:27.931 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:27.931 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:27.933 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:49:27.933 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:27.930+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:28.034 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:28.030+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:28.065 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:28.065 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:28.065 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:49:28.065 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:28.079 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:49:28.135 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:28.134+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:28.195 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:28.195 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:28.195 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:49:28.195 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:49:28.195 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:28.209 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:49:28.255 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:28.254+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:28.265 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:49:28.321 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T06:49:28.335 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:28.335 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:49:28.335 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:49:28.335 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:49:28.335 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.335 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:54 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'32, mlcod=13'29)", 2024-09-11T06:49:28.335 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:49:28.335 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.124591915, 2024-09-11T06:49:28.335 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034271670000000001, 2024-09-11T06:49:28.335 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:28.336 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:28.336 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:28.336 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:28.336 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.336 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:28.336 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:49:28.336 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.336 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.336 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:28.336 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:49:28.336 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.337 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.337 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:28.337 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210361+0000", 2024-09-11T06:49:28.337 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1260000000000001e-06 2024-09-11T06:49:28.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.337 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.337 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:28.337 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210370+0000", 2024-09-11T06:49:28.337 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8200000000000003e-06 2024-09-11T06:49:28.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.337 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.337 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:28.337 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210390+0000", 2024-09-11T06:49:28.337 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0126e-05 2024-09-11T06:49:28.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.338 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:28.338 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210404+0000", 2024-09-11T06:49:28.338 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4589999999999999e-05 2024-09-11T06:49:28.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.338 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:28.338 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210538+0000", 2024-09-11T06:49:28.338 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000133719 2024-09-11T06:49:28.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.338 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:28.338 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210593+0000", 2024-09-11T06:49:28.339 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5016999999999999e-05 2024-09-11T06:49:28.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.339 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:28.339 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213695+0000", 2024-09-11T06:49:28.339 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003101866 2024-09-11T06:49:28.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.339 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:28.339 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213787+0000", 2024-09-11T06:49:28.339 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1903000000000004e-05 2024-09-11T06:49:28.339 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.339 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:28.339 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.340 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.340 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:55 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'33, mlcod=13'29)", 2024-09-11T06:49:28.340 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:49:28.340 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.123572676999999, 2024-09-11T06:49:28.340 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.005086091, 2024-09-11T06:49:28.340 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:28.340 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:28.340 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:28.340 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:28.340 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.340 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:28.340 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:49:28.340 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.341 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.341 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:28.341 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:49:28.341 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.341 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.341 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:28.341 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211380+0000", 2024-09-11T06:49:28.341 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.097e-06 2024-09-11T06:49:28.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.341 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.341 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:28.341 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211537+0000", 2024-09-11T06:49:28.342 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000157369 2024-09-11T06:49:28.342 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.342 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.342 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:28.342 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211559+0000", 2024-09-11T06:49:28.342 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000001e-05 2024-09-11T06:49:28.342 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.342 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.342 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:28.342 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211573+0000", 2024-09-11T06:49:28.342 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4946000000000001e-05 2024-09-11T06:49:28.342 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.342 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.342 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:28.343 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211676+0000", 2024-09-11T06:49:28.343 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0001026 2024-09-11T06:49:28.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.343 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.343 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:28.343 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211709+0000", 2024-09-11T06:49:28.343 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3071999999999998e-05 2024-09-11T06:49:28.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.343 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.343 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:28.343 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216379+0000", 2024-09-11T06:49:28.343 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0046693780000000001 2024-09-11T06:49:28.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.343 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.344 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:28.344 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216465+0000", 2024-09-11T06:49:28.344 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6469000000000002e-05 2024-09-11T06:49:28.344 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.344 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:28.344 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.344 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.344 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:56 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'34, mlcod=13'29)", 2024-09-11T06:49:28.344 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:49:28.344 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.122138818, 2024-09-11T06:49:28.344 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0060310629999999997, 2024-09-11T06:49:28.344 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:28.344 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:28.344 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:28.345 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:28.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.345 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:28.345 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:49:28.345 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.345 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.345 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:28.345 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:49:28.345 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.345 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.345 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:28.345 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212814+0000", 2024-09-11T06:49:28.346 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0869999999999999e-06 2024-09-11T06:49:28.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.346 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.346 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:28.346 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212822+0000", 2024-09-11T06:49:28.346 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0870000000000007e-06 2024-09-11T06:49:28.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.346 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.346 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:28.346 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212840+0000", 2024-09-11T06:49:28.346 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:49:28.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.346 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.346 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:28.346 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212854+0000", 2024-09-11T06:49:28.347 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4003e-05 2024-09-11T06:49:28.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.347 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.347 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:28.347 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213026+0000", 2024-09-11T06:49:28.347 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017207499999999999 2024-09-11T06:49:28.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.347 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.347 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:28.347 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213063+0000", 2024-09-11T06:49:28.347 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7187000000000002e-05 2024-09-11T06:49:28.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.347 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.347 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:28.348 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218762+0000", 2024-09-11T06:49:28.348 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0056993290000000004 2024-09-11T06:49:28.348 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.348 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.348 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:28.348 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218844+0000", 2024-09-11T06:49:28.348 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1701999999999998e-05 2024-09-11T06:49:28.348 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.348 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:28.348 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.348 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.348 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.348 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:60 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'38, mlcod=13'36)", 2024-09-11T06:49:28.348 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:49:28.349 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.108197469, 2024-09-11T06:49:28.349 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039254270000000004, 2024-09-11T06:49:28.349 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:28.349 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:28.349 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:28.349 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:28.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.349 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:28.349 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:49:28.349 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.349 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.349 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:28.350 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:49:28.350 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.350 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:28.350 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226756+0000", 2024-09-11T06:49:28.350 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-11T06:49:28.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.350 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:28.350 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226768+0000", 2024-09-11T06:49:28.350 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1919999999999999e-05 2024-09-11T06:49:28.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.350 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:28.351 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226797+0000", 2024-09-11T06:49:28.351 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9573e-05 2024-09-11T06:49:28.351 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.351 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.351 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:28.351 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226818+0000", 2024-09-11T06:49:28.351 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0432999999999999e-05 2024-09-11T06:49:28.351 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.351 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.351 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:28.351 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226909+0000", 2024-09-11T06:49:28.351 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0920000000000004e-05 2024-09-11T06:49:28.351 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.351 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.352 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:28.352 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226936+0000", 2024-09-11T06:49:28.352 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7472999999999999e-05 2024-09-11T06:49:28.352 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.352 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.352 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:28.352 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230596+0000", 2024-09-11T06:49:28.352 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0036602520000000001 2024-09-11T06:49:28.352 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.352 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.352 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:28.352 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230680+0000", 2024-09-11T06:49:28.352 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3106e-05 2024-09-11T06:49:28.352 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.352 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:28.353 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.353 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.353 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:107 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'69, mlcod=13'64)", 2024-09-11T06:49:28.353 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:49:28.353 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.891750011999999, 2024-09-11T06:49:28.353 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040728830000000002, 2024-09-11T06:49:28.353 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:28.353 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:28.353 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:28.353 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:28.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.353 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:28.353 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:49:28.354 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.354 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.354 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:28.354 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:49:28.354 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.354 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.354 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:28.354 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443203+0000", 2024-09-11T06:49:28.354 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1480000000000001e-06 2024-09-11T06:49:28.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.354 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.354 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:28.355 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443222+0000", 2024-09-11T06:49:28.355 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9040000000000001e-05 2024-09-11T06:49:28.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.355 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:28.355 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443246+0000", 2024-09-11T06:49:28.355 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4606000000000001e-05 2024-09-11T06:49:28.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.355 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:28.355 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443277+0000", 2024-09-11T06:49:28.355 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0207999999999999e-05 2024-09-11T06:49:28.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.355 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:28.356 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443340+0000", 2024-09-11T06:49:28.356 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3602000000000005e-05 2024-09-11T06:49:28.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.356 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:28.356 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443363+0000", 2024-09-11T06:49:28.356 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2331e-05 2024-09-11T06:49:28.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.356 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:28.356 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447216+0000", 2024-09-11T06:49:28.356 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0038539690000000001 2024-09-11T06:49:28.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.357 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:28.357 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447274+0000", 2024-09-11T06:49:28.357 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7979000000000002e-05 2024-09-11T06:49:28.357 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.357 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:28.357 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.357 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:111 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'73, mlcod=13'67)", 2024-09-11T06:49:28.357 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:49:28.357 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.885238382000001, 2024-09-11T06:49:28.357 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034372360000000002, 2024-09-11T06:49:28.357 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:28.357 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:28.358 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:28.358 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:28.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.358 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:28.358 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:49:28.358 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.358 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:28.358 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:49:28.358 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.358 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:28.358 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449714+0000", 2024-09-11T06:49:28.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5000000000000001e-07 2024-09-11T06:49:28.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.359 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:28.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449721+0000", 2024-09-11T06:49:28.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8530000000000003e-06 2024-09-11T06:49:28.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.359 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:28.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449736+0000", 2024-09-11T06:49:28.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5114e-05 2024-09-11T06:49:28.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.359 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:28.360 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449743+0000", 2024-09-11T06:49:28.360 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0210000000000002e-06 2024-09-11T06:49:28.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.360 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:28.360 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449804+0000", 2024-09-11T06:49:28.360 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0479999999999997e-05 2024-09-11T06:49:28.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.360 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:28.360 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449826+0000", 2024-09-11T06:49:28.360 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1922000000000001e-05 2024-09-11T06:49:28.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.360 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:28.361 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453104+0000", 2024-09-11T06:49:28.361 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003278652 2024-09-11T06:49:28.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.361 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:28.361 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453150+0000", 2024-09-11T06:49:28.361 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6244e-05 2024-09-11T06:49:28.361 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.361 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:28.361 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.361 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:114 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'76, mlcod=13'71)", 2024-09-11T06:49:28.361 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:49:28.362 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.880560311, 2024-09-11T06:49:28.362 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0042576660000000002, 2024-09-11T06:49:28.362 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:28.362 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:28.362 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:28.362 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:28.362 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.362 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:28.362 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:49:28.362 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.362 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.362 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.362 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:28.362 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:49:28.363 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.363 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.363 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:28.363 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454392+0000", 2024-09-11T06:49:28.363 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5999999999999991e-07 2024-09-11T06:49:28.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.363 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.363 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:28.363 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454399+0000", 2024-09-11T06:49:28.363 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.878e-06 2024-09-11T06:49:28.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.363 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.363 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:28.364 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454415+0000", 2024-09-11T06:49:28.364 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5659999999999999e-05 2024-09-11T06:49:28.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.364 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:28.364 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454422+0000", 2024-09-11T06:49:28.364 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:49:28.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.364 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:28.364 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454474+0000", 2024-09-11T06:49:28.364 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1860999999999997e-05 2024-09-11T06:49:28.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454497+0000", 2024-09-11T06:49:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3686e-05 2024-09-11T06:49:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458552+0000", 2024-09-11T06:49:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040544550000000002 2024-09-11T06:49:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458649+0000", 2024-09-11T06:49:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7005999999999999e-05 2024-09-11T06:49:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:115 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'77, mlcod=13'72)", 2024-09-11T06:49:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.879219699, 2024-09-11T06:49:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003355584, 2024-09-11T06:49:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:28.367 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:28.367 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.367 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.367 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:28.367 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:28.367 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.367 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.367 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:28.367 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:28.367 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5999999999999999e-07 2024-09-11T06:49:28.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.367 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.367 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:28.368 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455803+0000", 2024-09-11T06:49:28.368 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0310999999999998e-05 2024-09-11T06:49:28.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.368 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.368 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:28.368 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455815+0000", 2024-09-11T06:49:28.368 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2787e-05 2024-09-11T06:49:28.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.368 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.368 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:28.368 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455823+0000", 2024-09-11T06:49:28.368 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5959999999999998e-06 2024-09-11T06:49:28.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.368 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.369 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:28.369 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455875+0000", 2024-09-11T06:49:28.369 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2054e-05 2024-09-11T06:49:28.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.369 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:28.369 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455896+0000", 2024-09-11T06:49:28.369 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0554999999999999e-05 2024-09-11T06:49:28.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.369 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:28.369 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459000+0000", 2024-09-11T06:49:28.369 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003104842 2024-09-11T06:49:28.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.370 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:28.370 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459087+0000", 2024-09-11T06:49:28.370 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6978999999999995e-05 2024-09-11T06:49:28.370 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.370 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:28.370 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.370 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.370 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:120 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'82, mlcod=13'78)", 2024-09-11T06:49:28.370 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:49:28.370 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.868542414, 2024-09-11T06:49:28.370 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0064612549999999999, 2024-09-11T06:49:28.370 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:28.370 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:28.371 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:28.371 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:28.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.371 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:28.371 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:49:28.371 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.371 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:28.371 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:49:28.371 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.371 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466410+0000", 2024-09-11T06:49:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0389999999999999e-06 2024-09-11T06:49:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466496+0000", 2024-09-11T06:49:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5519000000000006e-05 2024-09-11T06:49:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466510+0000", 2024-09-11T06:49:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3842999999999999e-05 2024-09-11T06:49:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466517+0000", 2024-09-11T06:49:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5229999999999998e-06 2024-09-11T06:49:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466595+0000", 2024-09-11T06:49:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8019e-05 2024-09-11T06:49:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466612+0000", 2024-09-11T06:49:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7013000000000001e-05 2024-09-11T06:49:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472787+0000", 2024-09-11T06:49:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0061750269999999996 2024-09-11T06:49:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472870+0000", 2024-09-11T06:49:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3271999999999998e-05 2024-09-11T06:49:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:121 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'83, mlcod=13'78)", 2024-09-11T06:49:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:49:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.867317618000001, 2024-09-11T06:49:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055613069999999997, 2024-09-11T06:49:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:49:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:49:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467635+0000", 2024-09-11T06:49:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6899999999999996e-07 2024-09-11T06:49:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467727+0000", 2024-09-11T06:49:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1800999999999997e-05 2024-09-11T06:49:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467740+0000", 2024-09-11T06:49:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3747999999999999e-05 2024-09-11T06:49:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467753+0000", 2024-09-11T06:49:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2807e-05 2024-09-11T06:49:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467812+0000", 2024-09-11T06:49:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8381999999999998e-05 2024-09-11T06:49:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467828+0000", 2024-09-11T06:49:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6728e-05 2024-09-11T06:49:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473112+0000", 2024-09-11T06:49:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052833619999999998 2024-09-11T06:49:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473195+0000", 2024-09-11T06:49:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3609999999999994e-05 2024-09-11T06:49:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:131 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'89, mlcod=13'85)", 2024-09-11T06:49:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:49:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.515911902999999, 2024-09-11T06:49:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052492260000000001, 2024-09-11T06:49:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:49:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:49:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819041+0000", 2024-09-11T06:49:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4360000000000001e-06 2024-09-11T06:49:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819100+0000", 2024-09-11T06:49:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9225999999999998e-05 2024-09-11T06:49:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819128+0000", 2024-09-11T06:49:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.779e-05 2024-09-11T06:49:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819142+0000", 2024-09-11T06:49:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3573e-05 2024-09-11T06:49:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819263+0000", 2024-09-11T06:49:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000121086 2024-09-11T06:49:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819292+0000", 2024-09-11T06:49:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9363e-05 2024-09-11T06:49:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824185+0000", 2024-09-11T06:49:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0048924620000000002 2024-09-11T06:49:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824289+0000", 2024-09-11T06:49:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010429000000000001 2024-09-11T06:49:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:132 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'90, mlcod=13'85)", 2024-09-11T06:49:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:49:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.515137212999999, 2024-09-11T06:49:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.004532342, 2024-09-11T06:49:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:49:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:49:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819816+0000", 2024-09-11T06:49:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.61e-06 2024-09-11T06:49:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819827+0000", 2024-09-11T06:49:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1e-05 2024-09-11T06:49:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819851+0000", 2024-09-11T06:49:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4267e-05 2024-09-11T06:49:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819864+0000", 2024-09-11T06:49:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2767e-05 2024-09-11T06:49:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820071+0000", 2024-09-11T06:49:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000207061 2024-09-11T06:49:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820103+0000", 2024-09-11T06:49:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2326999999999997e-05 2024-09-11T06:49:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824302+0000", 2024-09-11T06:49:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0041986000000000002 2024-09-11T06:49:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824347+0000", 2024-09-11T06:49:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4709999999999997e-05 2024-09-11T06:49:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:133 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'91, mlcod=13'86)", 2024-09-11T06:49:28.389 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:49:28.389 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.514505553999999, 2024-09-11T06:49:28.389 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039421459999999997, 2024-09-11T06:49:28.389 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:28.389 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:28.389 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:28.389 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:28.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.389 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:28.389 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:49:28.389 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.389 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:28.390 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:49:28.390 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.390 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:28.390 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820448+0000", 2024-09-11T06:49:28.390 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.84e-06 2024-09-11T06:49:28.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.390 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:28.390 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820463+0000", 2024-09-11T06:49:28.390 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5356e-05 2024-09-11T06:49:28.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.390 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:28.391 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820499+0000", 2024-09-11T06:49:28.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.625e-05 2024-09-11T06:49:28.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.391 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:28.391 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820546+0000", 2024-09-11T06:49:28.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7017000000000001e-05 2024-09-11T06:49:28.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.391 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:28.391 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820882+0000", 2024-09-11T06:49:28.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033512 2024-09-11T06:49:28.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.392 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:28.392 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820941+0000", 2024-09-11T06:49:28.392 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8922999999999998e-05 2024-09-11T06:49:28.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.392 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:28.392 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824351+0000", 2024-09-11T06:49:28.392 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034101940000000001 2024-09-11T06:49:28.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.392 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:28.392 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824388+0000", 2024-09-11T06:49:28.392 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7446e-05 2024-09-11T06:49:28.392 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.393 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:28.393 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.393 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:134 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'92, mlcod=13'86)", 2024-09-11T06:49:28.393 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:49:28.393 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.513846436000001, 2024-09-11T06:49:28.393 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033248179999999998, 2024-09-11T06:49:28.393 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:28.393 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:28.393 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:28.393 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:28.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.394 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:28.394 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:49:28.394 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.394 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:28.394 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:49:28.394 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.394 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:28.394 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821107+0000", 2024-09-11T06:49:28.394 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7430000000000001e-06 2024-09-11T06:49:28.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:28.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821205+0000", 2024-09-11T06:49:28.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7666999999999998e-05 2024-09-11T06:49:28.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:28.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821234+0000", 2024-09-11T06:49:28.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9756000000000001e-05 2024-09-11T06:49:28.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:28.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821254+0000", 2024-09-11T06:49:28.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9446e-05 2024-09-11T06:49:28.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.396 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:28.396 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821846+0000", 2024-09-11T06:49:28.396 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00059187599999999997 2024-09-11T06:49:28.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.396 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:28.396 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821894+0000", 2024-09-11T06:49:28.396 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8766e-05 2024-09-11T06:49:28.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.396 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:28.396 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824392+0000", 2024-09-11T06:49:28.397 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0024977509999999999 2024-09-11T06:49:28.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.397 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.397 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:28.397 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824430+0000", 2024-09-11T06:49:28.397 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7812999999999999e-05 2024-09-11T06:49:28.397 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.397 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:28.397 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.397 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.397 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:137 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'95, mlcod=13'86)", 2024-09-11T06:49:28.397 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:49:28.398 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.511384294999999, 2024-09-11T06:49:28.398 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003084299, 2024-09-11T06:49:28.398 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:28.398 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:28.398 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:28.398 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:28.398 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.398 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:28.398 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:49:28.398 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.398 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.398 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.398 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:28.398 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:49:28.398 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.399 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:28.399 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823568+0000", 2024-09-11T06:49:28.399 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1030000000000001e-06 2024-09-11T06:49:28.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.399 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:28.399 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823662+0000", 2024-09-11T06:49:28.399 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3529999999999994e-05 2024-09-11T06:49:28.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.399 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:28.399 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823686+0000", 2024-09-11T06:49:28.400 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4358999999999998e-05 2024-09-11T06:49:28.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.400 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.400 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:28.400 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823701+0000", 2024-09-11T06:49:28.400 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4847e-05 2024-09-11T06:49:28.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.400 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.400 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:28.400 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824518+0000", 2024-09-11T06:49:28.400 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00081716700000000005 2024-09-11T06:49:28.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.400 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.401 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:28.401 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824546+0000", 2024-09-11T06:49:28.401 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7860000000000001e-05 2024-09-11T06:49:28.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.401 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:28.401 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826558+0000", 2024-09-11T06:49:28.401 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0020120009999999998 2024-09-11T06:49:28.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.401 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:28.401 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826652+0000", 2024-09-11T06:49:28.401 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3431999999999996e-05 2024-09-11T06:49:28.401 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.402 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:28.402 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.402 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.402 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:139 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'97, mlcod=13'86)", 2024-09-11T06:49:28.402 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:49:28.402 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.509095576, 2024-09-11T06:49:28.402 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003732221, 2024-09-11T06:49:28.402 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:28.402 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:28.402 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:28.402 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:28.402 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:49:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:49:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825858+0000", 2024-09-11T06:49:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5829999999999999e-06 2024-09-11T06:49:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.404 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.404 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:28.404 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825874+0000", 2024-09-11T06:49:28.404 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5967000000000001e-05 2024-09-11T06:49:28.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.404 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.404 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:28.404 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825909+0000", 2024-09-11T06:49:28.404 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5692999999999999e-05 2024-09-11T06:49:28.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.404 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.404 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:28.404 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825923+0000", 2024-09-11T06:49:28.404 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4080000000000001e-05 2024-09-11T06:49:28.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.405 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:28.405 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826010+0000", 2024-09-11T06:49:28.405 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6248999999999994e-05 2024-09-11T06:49:28.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.405 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:28.405 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826038+0000", 2024-09-11T06:49:28.405 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8010000000000001e-05 2024-09-11T06:49:28.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.405 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:28.405 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829499+0000", 2024-09-11T06:49:28.406 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003461783 2024-09-11T06:49:28.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.406 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:28.406 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829588+0000", 2024-09-11T06:49:28.406 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8856000000000004e-05 2024-09-11T06:49:28.406 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.406 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:28.406 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.406 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:140 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'98, mlcod=13'93)", 2024-09-11T06:49:28.406 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:49:28.406 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.507244846999999, 2024-09-11T06:49:28.407 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0057995929999999996, 2024-09-11T06:49:28.407 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:28.407 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:28.407 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:28.407 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:28.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.407 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:28.407 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:49:28.407 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.407 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:28.407 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:49:28.407 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.408 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.408 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:28.408 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827708+0000", 2024-09-11T06:49:28.408 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.547e-06 2024-09-11T06:49:28.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.408 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.408 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:28.408 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827765+0000", 2024-09-11T06:49:28.408 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6266e-05 2024-09-11T06:49:28.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.408 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.408 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:28.408 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827791+0000", 2024-09-11T06:49:28.409 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6098999999999999e-05 2024-09-11T06:49:28.409 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.409 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.409 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:28.409 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827806+0000", 2024-09-11T06:49:28.409 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5537e-05 2024-09-11T06:49:28.409 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.409 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.409 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:28.409 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827928+0000", 2024-09-11T06:49:28.409 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000122012 2024-09-11T06:49:28.409 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.409 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.409 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:28.409 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827961+0000", 2024-09-11T06:49:28.410 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2663999999999998e-05 2024-09-11T06:49:28.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.410 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.410 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:28.410 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833409+0000", 2024-09-11T06:49:28.410 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054479350000000001 2024-09-11T06:49:28.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.410 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.410 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:28.410 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833506+0000", 2024-09-11T06:49:28.410 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7533000000000003e-05 2024-09-11T06:49:28.410 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.410 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:28.410 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.411 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:141 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'99, mlcod=13'93)", 2024-09-11T06:49:28.411 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:49:28.411 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.506351464000002, 2024-09-11T06:49:28.411 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055394709999999998, 2024-09-11T06:49:28.411 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:28.411 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:28.411 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:28.411 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:28.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.411 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:28.411 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:49:28.411 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.412 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:28.412 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:49:28.412 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.412 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:28.412 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828602+0000", 2024-09-11T06:49:28.412 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.46e-06 2024-09-11T06:49:28.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.412 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:28.412 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828618+0000", 2024-09-11T06:49:28.413 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6592999999999999e-05 2024-09-11T06:49:28.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.413 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:28.413 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828646+0000", 2024-09-11T06:49:28.413 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7552999999999999e-05 2024-09-11T06:49:28.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.413 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:28.413 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828663+0000", 2024-09-11T06:49:28.413 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.719e-05 2024-09-11T06:49:28.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.413 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:28.413 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829023+0000", 2024-09-11T06:49:28.414 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036054799999999998 2024-09-11T06:49:28.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.414 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:28.414 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829062+0000", 2024-09-11T06:49:28.414 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8865999999999997e-05 2024-09-11T06:49:28.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.414 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:28.414 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834078+0000", 2024-09-11T06:49:28.414 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050157480000000004 2024-09-11T06:49:28.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.414 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:28.415 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834140+0000", 2024-09-11T06:49:28.415 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1513000000000005e-05 2024-09-11T06:49:28.415 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.415 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:28.415 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.415 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.415 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:144 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'102, mlcod=13'98)", 2024-09-11T06:49:28.415 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:49:28.415 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.497190850999999, 2024-09-11T06:49:28.415 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003210348, 2024-09-11T06:49:28.415 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:28.415 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:28.415 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:28.416 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:28.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.416 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:28.416 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:49:28.416 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.416 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:28.416 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:49:28.416 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.416 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:28.416 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837768+0000", 2024-09-11T06:49:28.417 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9330000000000004e-06 2024-09-11T06:49:28.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.417 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:28.417 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837858+0000", 2024-09-11T06:49:28.417 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0185999999999993e-05 2024-09-11T06:49:28.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.417 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:28.417 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837890+0000", 2024-09-11T06:49:28.417 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2437000000000001e-05 2024-09-11T06:49:28.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.417 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:28.418 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837905+0000", 2024-09-11T06:49:28.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:49:28.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.420 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:28.420 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838019+0000", 2024-09-11T06:49:28.420 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000114336 2024-09-11T06:49:28.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.420 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:28.420 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838052+0000", 2024-09-11T06:49:28.420 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3139000000000002e-05 2024-09-11T06:49:28.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.420 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:28.420 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840914+0000", 2024-09-11T06:49:28.420 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028612780000000001 2024-09-11T06:49:28.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.421 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.421 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:28.421 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840971+0000", 2024-09-11T06:49:28.421 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7402999999999999e-05 2024-09-11T06:49:28.421 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.421 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:28.421 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.421 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.421 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:145 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'103, mlcod=13'98)", 2024-09-11T06:49:28.421 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:49:28.421 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.496398967000001, 2024-09-11T06:49:28.421 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054455409999999999, 2024-09-11T06:49:28.421 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:28.422 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:28.422 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:28.422 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:28.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.422 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:28.422 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:49:28.422 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.422 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:28.422 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:49:28.422 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:28.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.422 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:28.423 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838554+0000", 2024-09-11T06:49:28.423 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3430000000000001e-06 2024-09-11T06:49:28.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.423 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:28.423 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838640+0000", 2024-09-11T06:49:28.423 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5569999999999996e-05 2024-09-11T06:49:28.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.423 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:28.423 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838660+0000", 2024-09-11T06:49:28.423 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0576000000000001e-05 2024-09-11T06:49:28.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.424 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:28.424 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838674+0000", 2024-09-11T06:49:28.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3529999999999999e-05 2024-09-11T06:49:28.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.424 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:28.424 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838838+0000", 2024-09-11T06:49:28.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016468799999999999 2024-09-11T06:49:28.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.424 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:28.424 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838885+0000", 2024-09-11T06:49:28.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6884e-05 2024-09-11T06:49:28.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.425 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:28.425 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843901+0000", 2024-09-11T06:49:28.425 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050162310000000003 2024-09-11T06:49:28.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:28.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:28.425 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:28.425 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843998+0000", 2024-09-11T06:49:28.425 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.6718999999999993e-05 2024-09-11T06:49:28.425 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.425 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:28.425 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.425 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:28.425 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:28.426 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:28.427 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:49:28.427 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:49:28.428 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:49:28.428 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:49:28.428 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:49:28.429 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:28.426+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:28.450 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:49:28.450 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:49:28.472 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:49:28.472 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:49:28.493 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:49:28.493 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:49:28.514 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:49:28.514 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:49:28.527 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:28.526+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:28.545 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:28.545 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:28.559 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:49:28.647 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:28.646+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:28.716 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:28.716 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:28.729 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:28.726+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:28.731 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:49:28.830 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:28.830+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:28.899 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:28.899 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:28.912 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:49:28.959 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:28.958+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:29.029 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:29.029 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:29.031 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:29.030+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:29.042 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:49:29.132 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:29.134+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:29.204 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:29.204 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:29.218 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:49:29.234 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:29.230+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:29.315 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is unset 2024-09-11T06:49:29.334 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:29.330+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:29.345 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:29.345 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:29.358 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:49:29.435 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:29.430+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:29.489 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:29.489 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:29.503 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:49:29.547 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:29.546+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:29.623 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:29.623 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:29.637 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:49:29.637 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:29.634+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:29.738 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:29.738+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:29.802 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:29.802 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:29.817 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:49:29.838 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:29.834+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:29.934 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:29.934 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:29.939 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:29.934+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:29.948 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:49:30.040 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:30.038+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:30.066 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:30.066 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:30.079 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:49:30.167 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:30.166+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:30.241 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:30.238+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:30.245 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:30.245 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:30.259 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:49:30.350 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:30.350+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:30.426 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:30.426 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:30.439 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:49:30.483 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:30.482+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:30.532 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:49:30.580 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:30.578+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:30.602 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:30.602 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:30.617 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:49:30.666 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:30.666+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:30.685 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:49:30.686 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:49:30.702 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:49:30.702 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:49:30.718 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:49:30.718 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:49:30.737 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:30.737 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:30.738 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:49:30.738 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:49:30.745 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:30.742+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:30.753 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:49:30.756 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:49:30.756 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:49:30.776 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:49:30.776 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:49:30.847 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:30.846+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:30.880 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:30.880 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:30.880 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:49:30.880 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:30.894 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:49:30.947 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:30.946+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:31.048 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:31.050+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:31.057 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:31.057 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:31.057 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:49:31.057 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:49:31.057 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:31.071 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:49:31.149 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:31.146+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:31.196 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:31.197 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:49:31.197 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:49:31.197 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:49:31.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.197 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:54 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'32, mlcod=13'29)", 2024-09-11T06:49:31.197 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:49:31.197 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.984756695000002, 2024-09-11T06:49:31.197 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034271670000000001, 2024-09-11T06:49:31.197 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:31.197 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:31.197 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:31.198 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:31.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:31.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:49:31.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:31.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:49:31.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:31.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210361+0000", 2024-09-11T06:49:31.199 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1260000000000001e-06 2024-09-11T06:49:31.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.199 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:31.199 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210370+0000", 2024-09-11T06:49:31.199 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8200000000000003e-06 2024-09-11T06:49:31.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.199 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:31.199 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210390+0000", 2024-09-11T06:49:31.199 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0126e-05 2024-09-11T06:49:31.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.199 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:31.199 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210404+0000", 2024-09-11T06:49:31.200 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4589999999999999e-05 2024-09-11T06:49:31.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.200 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.200 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:31.200 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210538+0000", 2024-09-11T06:49:31.200 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000133719 2024-09-11T06:49:31.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.200 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.200 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:31.200 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210593+0000", 2024-09-11T06:49:31.200 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5016999999999999e-05 2024-09-11T06:49:31.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.200 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.200 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:31.201 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213695+0000", 2024-09-11T06:49:31.201 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003101866 2024-09-11T06:49:31.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.201 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.201 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:31.201 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213787+0000", 2024-09-11T06:49:31.201 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1903000000000004e-05 2024-09-11T06:49:31.201 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.201 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:31.201 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.201 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.201 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:55 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'33, mlcod=13'29)", 2024-09-11T06:49:31.201 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:49:31.202 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.983737457, 2024-09-11T06:49:31.202 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.005086091, 2024-09-11T06:49:31.202 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:31.202 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:31.202 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:31.202 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:31.202 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.202 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:31.202 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:49:31.202 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.202 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.202 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:31.203 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:49:31.203 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.203 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:31.203 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211380+0000", 2024-09-11T06:49:31.203 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.097e-06 2024-09-11T06:49:31.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.203 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:31.203 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211537+0000", 2024-09-11T06:49:31.203 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000157369 2024-09-11T06:49:31.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.204 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.204 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:31.204 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211559+0000", 2024-09-11T06:49:31.204 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000001e-05 2024-09-11T06:49:31.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.204 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.204 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:31.204 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211573+0000", 2024-09-11T06:49:31.204 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4946000000000001e-05 2024-09-11T06:49:31.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.204 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.204 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:31.204 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211676+0000", 2024-09-11T06:49:31.204 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0001026 2024-09-11T06:49:31.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.205 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:31.205 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211709+0000", 2024-09-11T06:49:31.205 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3071999999999998e-05 2024-09-11T06:49:31.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.205 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:31.205 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216379+0000", 2024-09-11T06:49:31.205 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0046693780000000001 2024-09-11T06:49:31.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.205 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:31.206 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216465+0000", 2024-09-11T06:49:31.206 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6469000000000002e-05 2024-09-11T06:49:31.206 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.206 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:31.206 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.206 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.206 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:56 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'34, mlcod=13'29)", 2024-09-11T06:49:31.206 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:49:31.206 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.982303598000001, 2024-09-11T06:49:31.206 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0060310629999999997, 2024-09-11T06:49:31.206 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:31.206 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:31.206 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:31.207 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:31.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.207 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:31.207 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:49:31.207 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.207 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:31.207 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:49:31.207 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.207 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:31.207 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212814+0000", 2024-09-11T06:49:31.208 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0869999999999999e-06 2024-09-11T06:49:31.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.208 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.208 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:31.208 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212822+0000", 2024-09-11T06:49:31.208 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0870000000000007e-06 2024-09-11T06:49:31.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.208 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.208 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:31.208 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212840+0000", 2024-09-11T06:49:31.208 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:49:31.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.208 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.208 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:31.209 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212854+0000", 2024-09-11T06:49:31.209 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4003e-05 2024-09-11T06:49:31.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:31.209 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213026+0000", 2024-09-11T06:49:31.209 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017207499999999999 2024-09-11T06:49:31.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:31.209 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213063+0000", 2024-09-11T06:49:31.209 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7187000000000002e-05 2024-09-11T06:49:31.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:31.210 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218762+0000", 2024-09-11T06:49:31.210 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0056993290000000004 2024-09-11T06:49:31.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.210 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.210 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:31.210 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218844+0000", 2024-09-11T06:49:31.210 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1701999999999998e-05 2024-09-11T06:49:31.210 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.210 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:31.210 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.210 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.210 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:60 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'38, mlcod=13'36)", 2024-09-11T06:49:31.211 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:49:31.211 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.968362248999998, 2024-09-11T06:49:31.211 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039254270000000004, 2024-09-11T06:49:31.211 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:31.211 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:31.211 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:31.211 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:31.211 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.211 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:31.211 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:49:31.211 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.211 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.211 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:31.212 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:49:31.212 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.212 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:31.212 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226756+0000", 2024-09-11T06:49:31.212 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-11T06:49:31.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.212 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:31.212 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226768+0000", 2024-09-11T06:49:31.212 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1919999999999999e-05 2024-09-11T06:49:31.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.213 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:31.213 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226797+0000", 2024-09-11T06:49:31.213 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9573e-05 2024-09-11T06:49:31.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.213 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:31.213 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226818+0000", 2024-09-11T06:49:31.213 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0432999999999999e-05 2024-09-11T06:49:31.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.213 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:31.213 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226909+0000", 2024-09-11T06:49:31.213 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0920000000000004e-05 2024-09-11T06:49:31.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.214 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.214 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:31.214 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226936+0000", 2024-09-11T06:49:31.214 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7472999999999999e-05 2024-09-11T06:49:31.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.214 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.214 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:31.214 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230596+0000", 2024-09-11T06:49:31.214 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0036602520000000001 2024-09-11T06:49:31.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.214 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.214 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:31.215 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230680+0000", 2024-09-11T06:49:31.215 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3106e-05 2024-09-11T06:49:31.215 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.215 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:31.215 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.215 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.215 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.215 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:107 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'69, mlcod=13'64)", 2024-09-11T06:49:31.215 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:49:31.215 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.751914792000001, 2024-09-11T06:49:31.215 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040728830000000002, 2024-09-11T06:49:31.215 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:31.215 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:31.215 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:31.215 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:31.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.216 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:31.216 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:49:31.216 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.216 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:31.216 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:49:31.216 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.216 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:31.217 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443203+0000", 2024-09-11T06:49:31.217 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1480000000000001e-06 2024-09-11T06:49:31.217 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.217 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.217 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:31.217 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443222+0000", 2024-09-11T06:49:31.217 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9040000000000001e-05 2024-09-11T06:49:31.217 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.217 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.217 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:31.217 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443246+0000", 2024-09-11T06:49:31.217 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4606000000000001e-05 2024-09-11T06:49:31.217 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.218 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:31.218 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443277+0000", 2024-09-11T06:49:31.218 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0207999999999999e-05 2024-09-11T06:49:31.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.218 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:31.218 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443340+0000", 2024-09-11T06:49:31.218 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3602000000000005e-05 2024-09-11T06:49:31.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.218 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:31.218 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443363+0000", 2024-09-11T06:49:31.219 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2331e-05 2024-09-11T06:49:31.219 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.219 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.219 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:31.219 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447216+0000", 2024-09-11T06:49:31.219 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0038539690000000001 2024-09-11T06:49:31.219 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.219 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.219 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:31.219 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447274+0000", 2024-09-11T06:49:31.219 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7979000000000002e-05 2024-09-11T06:49:31.219 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.220 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:31.220 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.220 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.220 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:111 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'73, mlcod=13'67)", 2024-09-11T06:49:31.220 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:49:31.220 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.745403161999999, 2024-09-11T06:49:31.220 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034372360000000002, 2024-09-11T06:49:31.220 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:31.220 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:31.220 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:31.220 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:31.220 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.221 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:31.221 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:49:31.221 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.221 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.221 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.221 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:31.221 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:49:31.221 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.221 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.221 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.221 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:31.221 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449714+0000", 2024-09-11T06:49:31.221 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5000000000000001e-07 2024-09-11T06:49:31.221 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.222 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:31.222 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449721+0000", 2024-09-11T06:49:31.222 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8530000000000003e-06 2024-09-11T06:49:31.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.222 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:31.222 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449736+0000", 2024-09-11T06:49:31.222 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5114e-05 2024-09-11T06:49:31.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.222 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:31.222 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449743+0000", 2024-09-11T06:49:31.222 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0210000000000002e-06 2024-09-11T06:49:31.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.223 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.223 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:31.223 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449804+0000", 2024-09-11T06:49:31.223 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0479999999999997e-05 2024-09-11T06:49:31.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.223 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.223 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:31.223 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449826+0000", 2024-09-11T06:49:31.223 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1922000000000001e-05 2024-09-11T06:49:31.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.223 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.223 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:31.223 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453104+0000", 2024-09-11T06:49:31.224 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003278652 2024-09-11T06:49:31.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.224 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.224 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:31.224 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453150+0000", 2024-09-11T06:49:31.224 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6244e-05 2024-09-11T06:49:31.224 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.224 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:31.224 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.224 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.224 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:114 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'76, mlcod=13'71)", 2024-09-11T06:49:31.224 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:49:31.224 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.740725091000002, 2024-09-11T06:49:31.225 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0042576660000000002, 2024-09-11T06:49:31.225 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:31.225 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:31.225 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:31.225 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:31.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.225 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:31.225 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:49:31.225 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.225 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:31.225 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:49:31.225 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.226 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:31.226 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454392+0000", 2024-09-11T06:49:31.226 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5999999999999991e-07 2024-09-11T06:49:31.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.226 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:31.226 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454399+0000", 2024-09-11T06:49:31.226 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.878e-06 2024-09-11T06:49:31.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.226 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:31.226 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454415+0000", 2024-09-11T06:49:31.227 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5659999999999999e-05 2024-09-11T06:49:31.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.227 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.227 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:31.227 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454422+0000", 2024-09-11T06:49:31.227 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:49:31.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.227 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.227 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:31.227 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454474+0000", 2024-09-11T06:49:31.227 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1860999999999997e-05 2024-09-11T06:49:31.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.227 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.227 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:31.228 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454497+0000", 2024-09-11T06:49:31.228 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3686e-05 2024-09-11T06:49:31.228 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.228 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.228 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:31.228 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458552+0000", 2024-09-11T06:49:31.228 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040544550000000002 2024-09-11T06:49:31.228 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.228 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.228 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:31.228 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458649+0000", 2024-09-11T06:49:31.228 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7005999999999999e-05 2024-09-11T06:49:31.228 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.228 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:31.229 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.229 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.229 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.229 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:115 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'77, mlcod=13'72)", 2024-09-11T06:49:31.229 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:31.229 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.739384479000002, 2024-09-11T06:49:31.229 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003355584, 2024-09-11T06:49:31.229 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:31.229 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:31.229 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:31.229 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:31.229 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.229 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:31.230 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:31.230 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.230 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:31.230 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:31.230 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.230 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:31.230 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:31.230 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5999999999999999e-07 2024-09-11T06:49:31.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.231 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:31.231 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455803+0000", 2024-09-11T06:49:31.231 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0310999999999998e-05 2024-09-11T06:49:31.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.231 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.231 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:31.231 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455815+0000", 2024-09-11T06:49:31.231 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2787e-05 2024-09-11T06:49:31.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.231 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.231 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:31.231 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455823+0000", 2024-09-11T06:49:31.231 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5959999999999998e-06 2024-09-11T06:49:31.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.232 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.232 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:31.232 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455875+0000", 2024-09-11T06:49:31.232 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2054e-05 2024-09-11T06:49:31.232 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.232 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.232 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:31.232 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455896+0000", 2024-09-11T06:49:31.232 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0554999999999999e-05 2024-09-11T06:49:31.232 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.232 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.232 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:31.232 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459000+0000", 2024-09-11T06:49:31.233 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003104842 2024-09-11T06:49:31.233 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.233 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.233 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:31.233 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459087+0000", 2024-09-11T06:49:31.233 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6978999999999995e-05 2024-09-11T06:49:31.233 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.233 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:31.233 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.233 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.233 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.233 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:120 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'82, mlcod=13'78)", 2024-09-11T06:49:31.233 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:49:31.233 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.728707193999998, 2024-09-11T06:49:31.234 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0064612549999999999, 2024-09-11T06:49:31.234 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:31.234 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:31.234 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:31.234 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:31.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.234 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:31.234 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:49:31.234 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.234 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:31.234 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:49:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466410+0000", 2024-09-11T06:49:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0389999999999999e-06 2024-09-11T06:49:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466496+0000", 2024-09-11T06:49:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5519000000000006e-05 2024-09-11T06:49:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:31.236 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466510+0000", 2024-09-11T06:49:31.236 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3842999999999999e-05 2024-09-11T06:49:31.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.236 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.236 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:31.236 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466517+0000", 2024-09-11T06:49:31.236 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5229999999999998e-06 2024-09-11T06:49:31.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.236 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.236 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:31.236 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466595+0000", 2024-09-11T06:49:31.236 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8019e-05 2024-09-11T06:49:31.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.236 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.237 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:31.237 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466612+0000", 2024-09-11T06:49:31.237 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7013000000000001e-05 2024-09-11T06:49:31.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.237 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.237 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:31.237 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472787+0000", 2024-09-11T06:49:31.237 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0061750269999999996 2024-09-11T06:49:31.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.237 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.237 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:31.237 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472870+0000", 2024-09-11T06:49:31.237 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3271999999999998e-05 2024-09-11T06:49:31.237 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.238 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:31.238 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.238 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.238 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:121 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'83, mlcod=13'78)", 2024-09-11T06:49:31.238 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:49:31.238 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.727482397999999, 2024-09-11T06:49:31.238 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055613069999999997, 2024-09-11T06:49:31.238 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:31.238 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:31.238 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:31.238 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:31.238 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.238 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:31.239 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:49:31.239 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.239 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.239 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:31.239 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:49:31.239 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.239 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.239 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:31.239 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467635+0000", 2024-09-11T06:49:31.239 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6899999999999996e-07 2024-09-11T06:49:31.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.239 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.240 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:31.240 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467727+0000", 2024-09-11T06:49:31.240 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1800999999999997e-05 2024-09-11T06:49:31.240 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.240 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.240 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:31.240 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467740+0000", 2024-09-11T06:49:31.240 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3747999999999999e-05 2024-09-11T06:49:31.240 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.240 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.240 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:31.240 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467753+0000", 2024-09-11T06:49:31.240 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2807e-05 2024-09-11T06:49:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467812+0000", 2024-09-11T06:49:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8381999999999998e-05 2024-09-11T06:49:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467828+0000", 2024-09-11T06:49:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6728e-05 2024-09-11T06:49:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473112+0000", 2024-09-11T06:49:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052833619999999998 2024-09-11T06:49:31.242 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.242 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.242 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:31.242 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473195+0000", 2024-09-11T06:49:31.242 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3609999999999994e-05 2024-09-11T06:49:31.242 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.242 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:31.242 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.242 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.242 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.242 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:131 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'89, mlcod=13'85)", 2024-09-11T06:49:31.242 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:49:31.242 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.376076683000001, 2024-09-11T06:49:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052492260000000001, 2024-09-11T06:49:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:49:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:49:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.244 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.244 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.244 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:31.244 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819041+0000", 2024-09-11T06:49:31.244 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4360000000000001e-06 2024-09-11T06:49:31.244 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.244 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.244 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:31.244 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819100+0000", 2024-09-11T06:49:31.244 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9225999999999998e-05 2024-09-11T06:49:31.244 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.244 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.244 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:31.245 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819128+0000", 2024-09-11T06:49:31.245 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.779e-05 2024-09-11T06:49:31.245 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.245 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.245 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:31.245 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819142+0000", 2024-09-11T06:49:31.245 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3573e-05 2024-09-11T06:49:31.245 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.245 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.245 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:31.245 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819263+0000", 2024-09-11T06:49:31.245 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000121086 2024-09-11T06:49:31.245 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.246 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.246 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:31.246 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819292+0000", 2024-09-11T06:49:31.246 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9363e-05 2024-09-11T06:49:31.246 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.246 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.246 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:31.246 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824185+0000", 2024-09-11T06:49:31.246 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0048924620000000002 2024-09-11T06:49:31.246 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.246 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.246 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:31.246 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824289+0000", 2024-09-11T06:49:31.246 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010429000000000001 2024-09-11T06:49:31.247 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.247 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:31.247 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.247 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.247 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.247 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:132 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'90, mlcod=13'85)", 2024-09-11T06:49:31.247 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:49:31.247 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.375301993000001, 2024-09-11T06:49:31.247 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.004532342, 2024-09-11T06:49:31.247 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:31.247 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:31.247 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:31.247 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:31.247 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.248 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:31.248 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:49:31.248 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.248 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.248 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.248 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:31.248 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:49:31.248 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.248 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.248 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.248 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:31.248 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819816+0000", 2024-09-11T06:49:31.248 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.61e-06 2024-09-11T06:49:31.249 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.249 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.249 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:31.249 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819827+0000", 2024-09-11T06:49:31.249 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1e-05 2024-09-11T06:49:31.249 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.249 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.249 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:31.249 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819851+0000", 2024-09-11T06:49:31.249 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4267e-05 2024-09-11T06:49:31.249 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.249 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.249 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:31.249 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819864+0000", 2024-09-11T06:49:31.250 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2767e-05 2024-09-11T06:49:31.250 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.250 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.250 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:31.250 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820071+0000", 2024-09-11T06:49:31.250 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000207061 2024-09-11T06:49:31.250 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.250 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.250 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:31.250 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820103+0000", 2024-09-11T06:49:31.250 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2326999999999997e-05 2024-09-11T06:49:31.250 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.250 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.250 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:31.251 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824302+0000", 2024-09-11T06:49:31.251 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0041986000000000002 2024-09-11T06:49:31.251 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.251 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.251 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:31.251 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824347+0000", 2024-09-11T06:49:31.251 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4709999999999997e-05 2024-09-11T06:49:31.251 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.251 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:31.251 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.251 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.251 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.251 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:133 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'91, mlcod=13'86)", 2024-09-11T06:49:31.251 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:49:31.252 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.374670334000001, 2024-09-11T06:49:31.252 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039421459999999997, 2024-09-11T06:49:31.252 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:31.252 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:31.252 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:31.252 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:31.252 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.252 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:31.252 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:49:31.252 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.252 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.252 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.252 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:31.252 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:49:31.253 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.253 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:31.253 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820448+0000", 2024-09-11T06:49:31.253 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.84e-06 2024-09-11T06:49:31.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.253 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:31.253 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820463+0000", 2024-09-11T06:49:31.253 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5356e-05 2024-09-11T06:49:31.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.254 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:31.254 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820499+0000", 2024-09-11T06:49:31.254 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.625e-05 2024-09-11T06:49:31.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.254 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.254 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:31.254 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820546+0000", 2024-09-11T06:49:31.254 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7017000000000001e-05 2024-09-11T06:49:31.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.254 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.254 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:31.254 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820882+0000", 2024-09-11T06:49:31.254 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033512 2024-09-11T06:49:31.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.255 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.255 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:31.255 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820941+0000", 2024-09-11T06:49:31.255 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8922999999999998e-05 2024-09-11T06:49:31.255 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.255 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.255 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:31.255 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824351+0000", 2024-09-11T06:49:31.255 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034101940000000001 2024-09-11T06:49:31.255 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.255 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.255 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:31.255 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824388+0000", 2024-09-11T06:49:31.256 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7446e-05 2024-09-11T06:49:31.256 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.256 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:31.256 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.256 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.256 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:134 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'92, mlcod=13'86)", 2024-09-11T06:49:31.256 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:49:31.256 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.374011216, 2024-09-11T06:49:31.256 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033248179999999998, 2024-09-11T06:49:31.256 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:31.256 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:31.256 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:31.256 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:31.257 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.257 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:31.257 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:49:31.257 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.257 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.257 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:31.257 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:49:31.257 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.257 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.257 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:31.257 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821107+0000", 2024-09-11T06:49:31.257 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7430000000000001e-06 2024-09-11T06:49:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821205+0000", 2024-09-11T06:49:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7666999999999998e-05 2024-09-11T06:49:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821234+0000", 2024-09-11T06:49:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9756000000000001e-05 2024-09-11T06:49:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821254+0000", 2024-09-11T06:49:31.259 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9446e-05 2024-09-11T06:49:31.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.259 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.259 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:31.259 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821846+0000", 2024-09-11T06:49:31.259 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00059187599999999997 2024-09-11T06:49:31.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.259 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.259 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:31.259 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821894+0000", 2024-09-11T06:49:31.259 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8766e-05 2024-09-11T06:49:31.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.259 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.259 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:31.260 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824392+0000", 2024-09-11T06:49:31.260 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0024977509999999999 2024-09-11T06:49:31.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.260 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.260 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:31.260 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824430+0000", 2024-09-11T06:49:31.260 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7812999999999999e-05 2024-09-11T06:49:31.260 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.260 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:31.260 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.260 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.260 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:137 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'95, mlcod=13'86)", 2024-09-11T06:49:31.260 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:49:31.261 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.371549075000001, 2024-09-11T06:49:31.261 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003084299, 2024-09-11T06:49:31.261 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:31.261 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:31.261 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:31.261 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:31.261 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.261 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:31.261 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:49:31.261 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.261 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.261 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.261 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:31.261 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:49:31.262 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.262 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.262 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:31.262 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823568+0000", 2024-09-11T06:49:31.262 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1030000000000001e-06 2024-09-11T06:49:31.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.262 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.262 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:31.262 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823662+0000", 2024-09-11T06:49:31.262 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3529999999999994e-05 2024-09-11T06:49:31.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.262 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.262 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:31.263 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823686+0000", 2024-09-11T06:49:31.263 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4358999999999998e-05 2024-09-11T06:49:31.263 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.263 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.263 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:31.263 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823701+0000", 2024-09-11T06:49:31.263 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4847e-05 2024-09-11T06:49:31.263 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.263 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.263 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:31.263 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824518+0000", 2024-09-11T06:49:31.263 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00081716700000000005 2024-09-11T06:49:31.263 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.264 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:31.264 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824546+0000", 2024-09-11T06:49:31.264 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7860000000000001e-05 2024-09-11T06:49:31.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.264 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:31.264 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826558+0000", 2024-09-11T06:49:31.264 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0020120009999999998 2024-09-11T06:49:31.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.264 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:31.264 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826652+0000", 2024-09-11T06:49:31.264 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3431999999999996e-05 2024-09-11T06:49:31.265 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.265 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:31.265 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.265 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:139 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'97, mlcod=13'86)", 2024-09-11T06:49:31.265 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:49:31.265 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.369260356000002, 2024-09-11T06:49:31.265 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003732221, 2024-09-11T06:49:31.265 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:31.265 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:31.265 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:31.265 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:31.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.265 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:31.266 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:49:31.266 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.266 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.266 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:31.266 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:49:31.266 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.266 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.266 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:31.266 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825858+0000", 2024-09-11T06:49:31.266 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5829999999999999e-06 2024-09-11T06:49:31.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.267 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.267 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:31.267 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825874+0000", 2024-09-11T06:49:31.267 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5967000000000001e-05 2024-09-11T06:49:31.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.267 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.267 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:31.267 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825909+0000", 2024-09-11T06:49:31.267 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5692999999999999e-05 2024-09-11T06:49:31.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.267 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.267 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:31.267 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825923+0000", 2024-09-11T06:49:31.267 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4080000000000001e-05 2024-09-11T06:49:31.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.268 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:31.268 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826010+0000", 2024-09-11T06:49:31.268 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6248999999999994e-05 2024-09-11T06:49:31.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.268 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:31.268 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826038+0000", 2024-09-11T06:49:31.268 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8010000000000001e-05 2024-09-11T06:49:31.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.268 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:31.268 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829499+0000", 2024-09-11T06:49:31.269 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003461783 2024-09-11T06:49:31.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.269 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.269 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:31.269 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829588+0000", 2024-09-11T06:49:31.269 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8856000000000004e-05 2024-09-11T06:49:31.269 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.269 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:31.269 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.269 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.269 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:140 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'98, mlcod=13'93)", 2024-09-11T06:49:31.269 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:49:31.269 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.367409627000001, 2024-09-11T06:49:31.270 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0057995929999999996, 2024-09-11T06:49:31.270 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:31.270 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:31.270 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:31.270 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:31.270 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.270 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:31.270 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:49:31.270 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.270 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.270 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.270 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:31.270 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:49:31.270 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.271 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:31.271 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827708+0000", 2024-09-11T06:49:31.271 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.547e-06 2024-09-11T06:49:31.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.271 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:31.271 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827765+0000", 2024-09-11T06:49:31.271 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6266e-05 2024-09-11T06:49:31.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.271 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:31.272 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827791+0000", 2024-09-11T06:49:31.272 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6098999999999999e-05 2024-09-11T06:49:31.272 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.272 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.272 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:31.272 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827806+0000", 2024-09-11T06:49:31.272 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5537e-05 2024-09-11T06:49:31.272 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.272 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.272 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:31.272 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827928+0000", 2024-09-11T06:49:31.272 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000122012 2024-09-11T06:49:31.272 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.272 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.273 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:31.273 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827961+0000", 2024-09-11T06:49:31.273 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2663999999999998e-05 2024-09-11T06:49:31.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.273 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:31.273 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833409+0000", 2024-09-11T06:49:31.273 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054479350000000001 2024-09-11T06:49:31.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.273 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:31.273 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833506+0000", 2024-09-11T06:49:31.273 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7533000000000003e-05 2024-09-11T06:49:31.274 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.274 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:31.274 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.274 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.274 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.274 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:141 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'99, mlcod=13'93)", 2024-09-11T06:49:31.274 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:49:31.274 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.366516244, 2024-09-11T06:49:31.274 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055394709999999998, 2024-09-11T06:49:31.274 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:31.274 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:31.274 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:31.274 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:31.274 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:49:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:49:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828602+0000", 2024-09-11T06:49:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.46e-06 2024-09-11T06:49:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.276 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.276 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:31.276 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828618+0000", 2024-09-11T06:49:31.276 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6592999999999999e-05 2024-09-11T06:49:31.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.276 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.276 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:31.276 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828646+0000", 2024-09-11T06:49:31.276 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7552999999999999e-05 2024-09-11T06:49:31.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.276 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.276 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:31.276 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828663+0000", 2024-09-11T06:49:31.277 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.719e-05 2024-09-11T06:49:31.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.277 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:31.277 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829023+0000", 2024-09-11T06:49:31.277 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036054799999999998 2024-09-11T06:49:31.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.277 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:31.277 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829062+0000", 2024-09-11T06:49:31.277 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8865999999999997e-05 2024-09-11T06:49:31.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.278 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:31.278 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834078+0000", 2024-09-11T06:49:31.278 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050157480000000004 2024-09-11T06:49:31.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.278 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.278 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:31.278 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834140+0000", 2024-09-11T06:49:31.278 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1513000000000005e-05 2024-09-11T06:49:31.278 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.278 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:31.278 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.278 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:144 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'102, mlcod=13'98)", 2024-09-11T06:49:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:49:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.357355631000001, 2024-09-11T06:49:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003210348, 2024-09-11T06:49:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:49:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:49:31.280 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.280 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.280 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:31.280 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837768+0000", 2024-09-11T06:49:31.280 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9330000000000004e-06 2024-09-11T06:49:31.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.280 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.280 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:31.280 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837858+0000", 2024-09-11T06:49:31.280 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0185999999999993e-05 2024-09-11T06:49:31.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.280 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.280 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837890+0000", 2024-09-11T06:49:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2437000000000001e-05 2024-09-11T06:49:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837905+0000", 2024-09-11T06:49:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:49:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838019+0000", 2024-09-11T06:49:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000114336 2024-09-11T06:49:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838052+0000", 2024-09-11T06:49:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3139000000000002e-05 2024-09-11T06:49:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840914+0000", 2024-09-11T06:49:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028612780000000001 2024-09-11T06:49:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840971+0000", 2024-09-11T06:49:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7402999999999999e-05 2024-09-11T06:49:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:145 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'103, mlcod=13'98)", 2024-09-11T06:49:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:49:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.356563746999999, 2024-09-11T06:49:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054455409999999999, 2024-09-11T06:49:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:49:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.284 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:31.284 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:49:31.284 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.284 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.284 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:31.284 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838554+0000", 2024-09-11T06:49:31.284 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3430000000000001e-06 2024-09-11T06:49:31.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.284 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.284 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:31.284 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838640+0000", 2024-09-11T06:49:31.284 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5569999999999996e-05 2024-09-11T06:49:31.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.285 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:31.285 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838660+0000", 2024-09-11T06:49:31.285 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0576000000000001e-05 2024-09-11T06:49:31.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.285 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:31.285 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838674+0000", 2024-09-11T06:49:31.285 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3529999999999999e-05 2024-09-11T06:49:31.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.285 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:31.285 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838838+0000", 2024-09-11T06:49:31.286 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016468799999999999 2024-09-11T06:49:31.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.286 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:31.286 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838885+0000", 2024-09-11T06:49:31.286 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6884e-05 2024-09-11T06:49:31.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.286 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:31.286 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843901+0000", 2024-09-11T06:49:31.286 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050162310000000003 2024-09-11T06:49:31.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.286 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:31.287 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843998+0000", 2024-09-11T06:49:31.287 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.6718999999999993e-05 2024-09-11T06:49:31.287 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.287 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:31.287 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.287 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.287 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:31.287 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:31.288 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:49:31.289 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:31.290+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:31.321 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T06:49:31.322 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-11T06:49:31.340 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T06:49:31.389 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:31.386+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:31.466 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:31.466 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:31.466 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:49:31.466 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:31.484 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:49:31.490 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:31.486+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:31.591 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:31.590+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:31.599 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:31.599 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:31.599 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:49:31.599 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:49:31.599 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:31.613 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:49:31.692 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:31.690+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:31.727 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:49:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:49:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:49:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:5 2.2 e17/15 2:5cea7035:::rbd_info:head v 17'1, mlcod=0'0)", 2024-09-11T06:49:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:49:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.492553765, 2024-09-11T06:49:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0019446719999999999, 2024-09-11T06:49:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:49:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:49:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236085+0000", 2024-09-11T06:49:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6000000000000003e-07 2024-09-11T06:49:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236108+0000", 2024-09-11T06:49:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3113e-05 2024-09-11T06:49:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236133+0000", 2024-09-11T06:49:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5723000000000001e-05 2024-09-11T06:49:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236154+0000", 2024-09-11T06:49:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0333999999999999e-05 2024-09-11T06:49:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236297+0000", 2024-09-11T06:49:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014278500000000001 2024-09-11T06:49:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236383+0000", 2024-09-11T06:49:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5927999999999995e-05 2024-09-11T06:49:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.237950+0000", 2024-09-11T06:49:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015673690000000001 2024-09-11T06:49:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.238029+0000", 2024-09-11T06:49:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8659999999999996e-05 2024-09-11T06:49:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:7 2.2 e18/15 2:5cea7035:::rbd_info:head v 18'2, mlcod=0'0)", 2024-09-11T06:49:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:49:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.478228558000001, 2024-09-11T06:49:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001907192, 2024-09-11T06:49:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:49:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:49:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250410+0000", 2024-09-11T06:49:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7700000000000004e-07 2024-09-11T06:49:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250445+0000", 2024-09-11T06:49:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4749000000000002e-05 2024-09-11T06:49:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250478+0000", 2024-09-11T06:49:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3269999999999998e-05 2024-09-11T06:49:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250501+0000", 2024-09-11T06:49:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3632999999999999e-05 2024-09-11T06:49:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250633+0000", 2024-09-11T06:49:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013143200000000001 2024-09-11T06:49:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250700+0000", 2024-09-11T06:49:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7002999999999995e-05 2024-09-11T06:49:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.252172+0000", 2024-09-11T06:49:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014719550000000001 2024-09-11T06:49:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.252316+0000", 2024-09-11T06:49:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144473 2024-09-11T06:49:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:31.736 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:31.740 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:49:31.793 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:31.790+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:31.858 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:31.859 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:31.859 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:49:31.859 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:31.872 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:49:31.915 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:31.914+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:31.995 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:31.994+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:32.035 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:32.035 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:32.035 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:49:32.035 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:49:32.035 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:32.047 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:49:32.096 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:32.098+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:32.155 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:32.155 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:49:32.155 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:49:32.156 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:49:32.156 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:32.156 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:1 2.3 e16/15 2:c4c92e5a:::rbd_trash:head v 16'1, mlcod=0'0)", 2024-09-11T06:49:32.156 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:49:32.156 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 20.920696878000001, 2024-09-11T06:49:32.156 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001234761, 2024-09-11T06:49:32.156 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:32.156 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:32.156 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:32.156 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:32.156 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:32.156 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:32.156 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:49:32.157 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:32.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:32.157 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:32.157 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:32.157 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:49:32.157 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:32.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:32.157 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:32.157 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:32.157 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:49:32.157 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6000000000000003e-07 2024-09-11T06:49:32.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:32.157 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:32.158 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:32.158 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236296+0000", 2024-09-11T06:49:32.158 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6393999999999999e-05 2024-09-11T06:49:32.158 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:32.158 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:32.158 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:32.158 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236317+0000", 2024-09-11T06:49:32.158 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1243e-05 2024-09-11T06:49:32.158 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:32.158 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:32.158 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:32.158 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236333+0000", 2024-09-11T06:49:32.158 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.558e-05 2024-09-11T06:49:32.158 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:32.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:32.159 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:32.159 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236435+0000", 2024-09-11T06:49:32.159 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000102803 2024-09-11T06:49:32.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:32.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:32.159 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:32.159 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236474+0000", 2024-09-11T06:49:32.159 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8383999999999997e-05 2024-09-11T06:49:32.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:32.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:32.159 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:32.159 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.237432+0000", 2024-09-11T06:49:32.159 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00095849300000000004 2024-09-11T06:49:32.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:32.160 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:32.160 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:32.160 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.237513+0000", 2024-09-11T06:49:32.160 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1204000000000004e-05 2024-09-11T06:49:32.160 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:32.160 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:32.160 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:32.160 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:32.160 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:32.160 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:32.168 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:49:32.215 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:32.214+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:32.300 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:32.298+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:32.300 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:32.300 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:32.300 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:49:32.300 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:32.313 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:49:32.338 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.0 to 1 (10000) 2024-09-11T06:49:32.342 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is set 2024-09-11T06:49:32.360 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-11T06:49:32.399 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:32.398+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:32.471 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:32.471 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:32.471 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:49:32.471 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:49:32.471 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:32.485 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:49:32.499 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:32.498+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:32.600 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:32.598+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:32.642 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:32.643 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:49:32.643 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:49:32.643 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:49:32.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.643 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:32.643 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:49:32.643 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 26.472787223000001, 2024-09-11T06:49:32.643 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011420669, 2024-09-11T06:49:32.643 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:32.643 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:32.643 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:32.644 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:32.644 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:32.644 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:32.644 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 37 2024-09-11T06:49:32.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.644 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:32.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.644 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:32.644 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:49:32.644 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.644 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:32.644 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:49:32.645 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:32.645 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170509+0000", 2024-09-11T06:49:32.645 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.13e-06 2024-09-11T06:49:32.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:32.645 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170521+0000", 2024-09-11T06:49:32.645 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1776999999999999e-05 2024-09-11T06:49:32.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.646 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:32.646 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170538+0000", 2024-09-11T06:49:32.646 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7413000000000001e-05 2024-09-11T06:49:32.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.646 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:32.646 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170570+0000", 2024-09-11T06:49:32.646 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2313e-05 2024-09-11T06:49:32.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.646 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:32.646 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179768+0000", 2024-09-11T06:49:32.646 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0091978010000000002 2024-09-11T06:49:32.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.647 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:32.647 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179951+0000", 2024-09-11T06:49:32.647 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018227900000000001 2024-09-11T06:49:32.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.647 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:32.647 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.180175+0000", 2024-09-11T06:49:32.647 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022432800000000001 2024-09-11T06:49:32.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.647 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:32.647 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181112+0000", 2024-09-11T06:49:32.647 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00093712500000000002 2024-09-11T06:49:32.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.648 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:32.648 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181789+0000", 2024-09-11T06:49:32.648 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00067705100000000002 2024-09-11T06:49:32.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.648 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:32.648 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181871+0000", 2024-09-11T06:49:32.648 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1816000000000004e-05 2024-09-11T06:49:32.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.648 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:32.648 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181929+0000", 2024-09-11T06:49:32.648 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7636000000000002e-05 2024-09-11T06:49:32.648 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.649 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:32.649 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.649 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:32.649 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:49:32.649 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 26.371528261999998, 2024-09-11T06:49:32.649 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283336, 2024-09-11T06:49:32.649 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:32.649 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:32.649 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:32.649 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:32.649 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:32.649 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:32.649 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 87 2024-09-11T06:49:32.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.650 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:32.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.650 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:32.650 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:49:32.650 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.650 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:32.650 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:49:32.650 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.650 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:32.650 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271768+0000", 2024-09-11T06:49:32.651 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7599999999999996e-07 2024-09-11T06:49:32.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.651 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:32.651 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271860+0000", 2024-09-11T06:49:32.651 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2639999999999994e-05 2024-09-11T06:49:32.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.651 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:32.651 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271873+0000", 2024-09-11T06:49:32.651 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3139000000000001e-05 2024-09-11T06:49:32.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.652 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:32.652 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271882+0000", 2024-09-11T06:49:32.652 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2369999999999992e-06 2024-09-11T06:49:32.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.652 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:32.652 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277150+0000", 2024-09-11T06:49:32.652 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052684419999999999 2024-09-11T06:49:32.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.652 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:32.652 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277273+0000", 2024-09-11T06:49:32.652 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012278899999999999 2024-09-11T06:49:32.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.653 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:32.653 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277466+0000", 2024-09-11T06:49:32.653 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193578 2024-09-11T06:49:32.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.653 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:32.653 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.280808+0000", 2024-09-11T06:49:32.653 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003341971 2024-09-11T06:49:32.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.653 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:32.653 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284419+0000", 2024-09-11T06:49:32.653 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003610982 2024-09-11T06:49:32.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.654 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:32.654 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284531+0000", 2024-09-11T06:49:32.654 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011121299999999999 2024-09-11T06:49:32.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.654 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:32.654 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284600+0000", 2024-09-11T06:49:32.654 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9592999999999996e-05 2024-09-11T06:49:32.654 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.654 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:32.654 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.655 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:32.655 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:49:32.655 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 26.37139801, 2024-09-11T06:49:32.655 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283314, 2024-09-11T06:49:32.655 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:32.655 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:32.655 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:32.655 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:32.655 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:32.655 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:32.655 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 88 2024-09-11T06:49:32.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.655 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:32.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.656 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:32.656 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:49:32.656 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.656 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:32.656 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:49:32.656 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.656 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:32.656 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271898+0000", 2024-09-11T06:49:32.656 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2999999999999999e-07 2024-09-11T06:49:32.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.657 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:32.657 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271904+0000", 2024-09-11T06:49:32.657 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.767e-06 2024-09-11T06:49:32.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.657 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:32.657 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271913+0000", 2024-09-11T06:49:32.657 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7200000000000001e-06 2024-09-11T06:49:32.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.657 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:32.657 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271920+0000", 2024-09-11T06:49:32.657 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7100000000000001e-06 2024-09-11T06:49:32.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.658 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:32.658 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278684+0000", 2024-09-11T06:49:32.658 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0067642650000000002 2024-09-11T06:49:32.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.658 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:32.658 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278838+0000", 2024-09-11T06:49:32.658 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015371499999999999 2024-09-11T06:49:32.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.658 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:32.658 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.279040+0000", 2024-09-11T06:49:32.659 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020207499999999999 2024-09-11T06:49:32.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.659 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:32.659 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.281448+0000", 2024-09-11T06:49:32.659 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002407686 2024-09-11T06:49:32.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.659 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:32.659 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284615+0000", 2024-09-11T06:49:32.659 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031675499999999999 2024-09-11T06:49:32.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.659 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:32.659 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284682+0000", 2024-09-11T06:49:32.660 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7032000000000005e-05 2024-09-11T06:49:32.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.660 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:32.660 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284730+0000", 2024-09-11T06:49:32.660 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7790000000000002e-05 2024-09-11T06:49:32.660 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.660 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:32.660 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.660 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:32.660 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:49:32.660 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 26.203496787999999, 2024-09-11T06:49:32.661 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011114877, 2024-09-11T06:49:32.661 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:32.661 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:32.661 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:32.661 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:32.661 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:32.661 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:32.661 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 107 2024-09-11T06:49:32.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.661 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:32.661 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.661 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:32.661 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:49:32.661 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.662 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.662 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:32.662 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:49:32.662 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.662 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.662 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:32.662 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439799+0000", 2024-09-11T06:49:32.662 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.16e-07 2024-09-11T06:49:32.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.662 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.662 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:32.662 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439806+0000", 2024-09-11T06:49:32.662 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3640000000000004e-06 2024-09-11T06:49:32.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.663 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:32.663 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439816+0000", 2024-09-11T06:49:32.663 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0485999999999999e-05 2024-09-11T06:49:32.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.663 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:32.663 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439824+0000", 2024-09-11T06:49:32.663 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.6699999999999994e-06 2024-09-11T06:49:32.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.663 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:32.664 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443328+0000", 2024-09-11T06:49:32.664 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035044329999999999 2024-09-11T06:49:32.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.664 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.664 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:32.664 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443461+0000", 2024-09-11T06:49:32.664 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013272199999999999 2024-09-11T06:49:32.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.664 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.664 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:32.664 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443678+0000", 2024-09-11T06:49:32.664 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021674500000000001 2024-09-11T06:49:32.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.664 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.665 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:32.665 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449635+0000", 2024-09-11T06:49:32.665 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0059572119999999999 2024-09-11T06:49:32.665 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.665 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.665 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:32.665 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450750+0000", 2024-09-11T06:49:32.665 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001115654 2024-09-11T06:49:32.665 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.665 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.665 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:32.665 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450832+0000", 2024-09-11T06:49:32.665 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1188999999999998e-05 2024-09-11T06:49:32.665 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.665 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.666 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:32.666 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450913+0000", 2024-09-11T06:49:32.666 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1485999999999999e-05 2024-09-11T06:49:32.666 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.666 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:32.666 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.666 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.666 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.666 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:32.666 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:49:32.666 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 26.202905813000001, 2024-09-11T06:49:32.666 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011236556, 2024-09-11T06:49:32.666 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:32.666 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:32.666 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:32.667 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:32.667 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:32.667 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:32.667 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 109 2024-09-11T06:49:32.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.667 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:32.667 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.667 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:32.667 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:49:32.667 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.667 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.667 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:32.667 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:49:32.668 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.668 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:32.668 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440391+0000", 2024-09-11T06:49:32.668 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8360000000000001e-06 2024-09-11T06:49:32.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.668 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:32.668 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440560+0000", 2024-09-11T06:49:32.668 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000169085 2024-09-11T06:49:32.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.668 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:32.669 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440585+0000", 2024-09-11T06:49:32.669 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4530000000000001e-05 2024-09-11T06:49:32.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.669 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.669 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:32.669 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440606+0000", 2024-09-11T06:49:32.669 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1277e-05 2024-09-11T06:49:32.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.669 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.669 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:32.669 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446479+0000", 2024-09-11T06:49:32.669 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0058734989999999999 2024-09-11T06:49:32.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.669 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.670 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:32.670 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446649+0000", 2024-09-11T06:49:32.670 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016948199999999999 2024-09-11T06:49:32.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.670 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:32.670 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446854+0000", 2024-09-11T06:49:32.670 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020491500000000001 2024-09-11T06:49:32.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.670 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:32.670 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451380+0000", 2024-09-11T06:49:32.670 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0045264110000000001 2024-09-11T06:49:32.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.671 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.671 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:32.671 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451479+0000", 2024-09-11T06:49:32.671 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.8975999999999996e-05 2024-09-11T06:49:32.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.671 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.671 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:32.671 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451562+0000", 2024-09-11T06:49:32.671 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3213000000000004e-05 2024-09-11T06:49:32.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.671 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.671 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:32.671 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451626+0000", 2024-09-11T06:49:32.671 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3331999999999996e-05 2024-09-11T06:49:32.671 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.672 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:32.672 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.672 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.672 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:32.672 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:49:32.672 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 26.202656685000001, 2024-09-11T06:49:32.672 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011252866, 2024-09-11T06:49:32.672 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:32.672 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:32.672 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:32.672 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:32.672 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:32.672 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:32.672 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 110 2024-09-11T06:49:32.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.673 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:32.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.673 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:32.673 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:49:32.673 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.673 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:32.673 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:49:32.673 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.673 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:32.673 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440640+0000", 2024-09-11T06:49:32.674 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5430000000000001e-06 2024-09-11T06:49:32.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.674 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.674 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:32.674 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440651+0000", 2024-09-11T06:49:32.674 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0672999999999999e-05 2024-09-11T06:49:32.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.674 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.674 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:32.674 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440670+0000", 2024-09-11T06:49:32.674 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9726999999999999e-05 2024-09-11T06:49:32.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.674 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.674 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:32.674 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440691+0000", 2024-09-11T06:49:32.675 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0526e-05 2024-09-11T06:49:32.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.675 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.675 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:32.675 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448191+0000", 2024-09-11T06:49:32.675 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0075005990000000002 2024-09-11T06:49:32.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.675 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.675 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:32.675 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448390+0000", 2024-09-11T06:49:32.675 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000198815 2024-09-11T06:49:32.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.675 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.675 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:32.675 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448735+0000", 2024-09-11T06:49:32.675 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034424999999999999 2024-09-11T06:49:32.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.676 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:32.676 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451742+0000", 2024-09-11T06:49:32.676 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030077440000000001 2024-09-11T06:49:32.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.676 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:32.676 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451761+0000", 2024-09-11T06:49:32.676 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9196999999999999e-05 2024-09-11T06:49:32.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.676 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:32.676 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451834+0000", 2024-09-11T06:49:32.677 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2071999999999997e-05 2024-09-11T06:49:32.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.677 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:32.677 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451891+0000", 2024-09-11T06:49:32.677 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7720000000000003e-05 2024-09-11T06:49:32.677 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.677 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:32.677 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.677 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:32.677 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:49:32.678 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 26.20254881, 2024-09-11T06:49:32.678 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014944941, 2024-09-11T06:49:32.678 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:32.678 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:32.678 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:32.678 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:32.678 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:32.678 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:32.678 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 111 2024-09-11T06:49:32.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.678 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:32.678 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.678 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:32.678 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:49:32.678 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.679 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:32.679 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:49:32.679 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.679 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:32.679 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440748+0000", 2024-09-11T06:49:32.679 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:49:32.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.679 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:32.679 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440760+0000", 2024-09-11T06:49:32.680 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1897e-05 2024-09-11T06:49:32.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.680 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:32.680 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440782+0000", 2024-09-11T06:49:32.680 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2552999999999999e-05 2024-09-11T06:49:32.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.680 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:32.680 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440793+0000", 2024-09-11T06:49:32.680 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0643e-05 2024-09-11T06:49:32.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.681 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:32.681 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449755+0000", 2024-09-11T06:49:32.681 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089622620000000004 2024-09-11T06:49:32.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.681 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:32.681 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449918+0000", 2024-09-11T06:49:32.681 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163086 2024-09-11T06:49:32.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.681 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:32.681 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450150+0000", 2024-09-11T06:49:32.681 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023204500000000001 2024-09-11T06:49:32.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.682 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:32.682 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454533+0000", 2024-09-11T06:49:32.682 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0043831349999999998 2024-09-11T06:49:32.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.682 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:32.682 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455511+0000", 2024-09-11T06:49:32.682 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0009776050000000001 2024-09-11T06:49:32.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.682 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:32.682 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455597+0000", 2024-09-11T06:49:32.683 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6446000000000005e-05 2024-09-11T06:49:32.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.683 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.683 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:32.683 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455691+0000", 2024-09-11T06:49:32.683 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3832000000000006e-05 2024-09-11T06:49:32.683 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.683 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:32.683 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.683 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.683 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:32.683 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:49:32.683 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 26.202479047000001, 2024-09-11T06:49:32.684 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018885411000000001, 2024-09-11T06:49:32.684 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:32.684 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:32.684 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:32.684 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:32.684 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:32.684 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:32.684 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 112 2024-09-11T06:49:32.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.684 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:32.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.684 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:32.684 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:49:32.684 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.685 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:32.685 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:49:32.685 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.685 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:32.685 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440817+0000", 2024-09-11T06:49:32.685 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:49:32.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.685 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:32.685 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441003+0000", 2024-09-11T06:49:32.686 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018661200000000001 2024-09-11T06:49:32.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.686 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:32.686 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441027+0000", 2024-09-11T06:49:32.686 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3920000000000001e-05 2024-09-11T06:49:32.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.686 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:32.686 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441062+0000", 2024-09-11T06:49:32.686 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4663000000000003e-05 2024-09-11T06:49:32.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.686 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:32.686 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451981+0000", 2024-09-11T06:49:32.686 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010918926000000001 2024-09-11T06:49:32.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.687 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:32.687 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452111+0000", 2024-09-11T06:49:32.687 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129529 2024-09-11T06:49:32.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.687 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:32.687 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452311+0000", 2024-09-11T06:49:32.687 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020013799999999999 2024-09-11T06:49:32.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.687 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:32.687 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455809+0000", 2024-09-11T06:49:32.687 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003498443 2024-09-11T06:49:32.688 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.688 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:32.688 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459566+0000", 2024-09-11T06:49:32.688 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003757271 2024-09-11T06:49:32.688 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.688 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:32.688 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459640+0000", 2024-09-11T06:49:32.688 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3949000000000006e-05 2024-09-11T06:49:32.688 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.688 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:32.689 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459701+0000", 2024-09-11T06:49:32.689 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1117000000000005e-05 2024-09-11T06:49:32.689 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.689 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:32.689 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.689 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:32.689 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:49:32.689 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 26.202200196, 2024-09-11T06:49:32.689 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018760915, 2024-09-11T06:49:32.689 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:32.689 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:32.690 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:32.690 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:32.690 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:32.690 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:32.690 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 113 2024-09-11T06:49:32.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.690 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:32.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.690 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:32.690 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:49:32.690 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.690 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:32.691 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:49:32.691 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.691 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:32.691 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441096+0000", 2024-09-11T06:49:32.691 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4130000000000001e-06 2024-09-11T06:49:32.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.691 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:32.691 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441110+0000", 2024-09-11T06:49:32.691 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3900000000000001e-05 2024-09-11T06:49:32.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.692 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:32.692 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441131+0000", 2024-09-11T06:49:32.692 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0636999999999999e-05 2024-09-11T06:49:32.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.692 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:32.692 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441147+0000", 2024-09-11T06:49:32.692 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6512999999999999e-05 2024-09-11T06:49:32.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.692 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:32.692 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453274+0000", 2024-09-11T06:49:32.692 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012126764999999999 2024-09-11T06:49:32.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.693 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:32.693 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453498+0000", 2024-09-11T06:49:32.693 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000223568 2024-09-11T06:49:32.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.693 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:32.693 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453872+0000", 2024-09-11T06:49:32.693 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037467099999999998 2024-09-11T06:49:32.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.693 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:32.693 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458709+0000", 2024-09-11T06:49:32.693 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0048365680000000003 2024-09-11T06:49:32.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.694 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:32.694 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459716+0000", 2024-09-11T06:49:32.694 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010075240000000001 2024-09-11T06:49:32.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.694 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:32.694 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459779+0000", 2024-09-11T06:49:32.694 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2336999999999996e-05 2024-09-11T06:49:32.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.694 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:32.694 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459856+0000", 2024-09-11T06:49:32.695 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7019000000000002e-05 2024-09-11T06:49:32.695 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.695 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:32.695 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.695 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.695 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.695 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:32.695 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:32.695 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 26.202123456999999, 2024-09-11T06:49:32.696 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018963114, 2024-09-11T06:49:32.696 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:32.696 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:32.696 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:32.696 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:32.696 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:32.696 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:32.696 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 114 2024-09-11T06:49:32.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.697 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:32.697 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.697 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:32.697 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:32.697 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.697 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.697 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:32.697 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:32.697 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.697 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.697 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:32.697 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:32.698 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9299999999999996e-07 2024-09-11T06:49:32.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.698 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.698 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:32.698 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441180+0000", 2024-09-11T06:49:32.698 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7200000000000006e-06 2024-09-11T06:49:32.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.698 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.698 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:32.698 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441265+0000", 2024-09-11T06:49:32.698 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4753000000000007e-05 2024-09-11T06:49:32.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.699 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:32.699 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441288+0000", 2024-09-11T06:49:32.699 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2860000000000001e-05 2024-09-11T06:49:32.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.699 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:32.699 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454585+0000", 2024-09-11T06:49:32.699 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013297286 2024-09-11T06:49:32.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.699 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:32.699 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454714+0000", 2024-09-11T06:49:32.699 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000128465 2024-09-11T06:49:32.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.700 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.700 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:32.700 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454909+0000", 2024-09-11T06:49:32.700 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019522499999999999 2024-09-11T06:49:32.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.700 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.700 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:32.700 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459868+0000", 2024-09-11T06:49:32.700 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0049591239999999997 2024-09-11T06:49:32.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.700 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.700 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:32.700 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459988+0000", 2024-09-11T06:49:32.701 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000120532 2024-09-11T06:49:32.701 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.701 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.701 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:32.701 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460071+0000", 2024-09-11T06:49:32.701 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2256000000000007e-05 2024-09-11T06:49:32.701 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.701 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.701 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:32.701 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460135+0000", 2024-09-11T06:49:32.701 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3999999999999997e-05 2024-09-11T06:49:32.701 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.701 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:32.702 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.702 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.702 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.702 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:32.702 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:49:32.702 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 26.201965827999999, 2024-09-11T06:49:32.702 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.019036849000000002, 2024-09-11T06:49:32.702 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:32.702 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:32.702 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:32.702 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:32.702 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:32.702 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:32.702 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 115 2024-09-11T06:49:32.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.703 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:32.703 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.703 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:32.703 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:49:32.703 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.703 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.703 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:32.703 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:49:32.703 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.703 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.704 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:32.704 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441330+0000", 2024-09-11T06:49:32.704 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1930000000000001e-06 2024-09-11T06:49:32.704 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.704 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.704 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:32.704 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441544+0000", 2024-09-11T06:49:32.704 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021405799999999999 2024-09-11T06:49:32.704 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.704 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.704 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:32.704 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441572+0000", 2024-09-11T06:49:32.704 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7317e-05 2024-09-11T06:49:32.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.705 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.705 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:32.705 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441593+0000", 2024-09-11T06:49:32.705 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1702999999999999e-05 2024-09-11T06:49:32.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.705 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.705 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:32.705 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455891+0000", 2024-09-11T06:49:32.705 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01429708 2024-09-11T06:49:32.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.705 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.705 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:32.705 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456046+0000", 2024-09-11T06:49:32.705 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000155682 2024-09-11T06:49:32.706 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.706 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.706 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:32.706 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456242+0000", 2024-09-11T06:49:32.706 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000195692 2024-09-11T06:49:32.706 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.706 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.706 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:32.706 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459876+0000", 2024-09-11T06:49:32.706 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036336480000000002 2024-09-11T06:49:32.706 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.706 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.706 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:32.707 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460245+0000", 2024-09-11T06:49:32.707 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036901700000000002 2024-09-11T06:49:32.707 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.707 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.707 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:32.707 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460316+0000", 2024-09-11T06:49:32.707 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1846000000000003e-05 2024-09-11T06:49:32.707 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.707 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.707 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:32.707 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460366+0000", 2024-09-11T06:49:32.707 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9613000000000001e-05 2024-09-11T06:49:32.707 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.707 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:32.707 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.708 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.708 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.708 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:32.708 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:49:32.708 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 26.201643747999999, 2024-09-11T06:49:32.708 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01904432, 2024-09-11T06:49:32.708 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:32.708 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:32.708 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:32.708 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:32.708 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:32.708 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:32.708 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 116 2024-09-11T06:49:32.708 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.709 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:32.709 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.709 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:32.709 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:49:32.709 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.709 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.709 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:32.709 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:49:32.709 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.709 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.709 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:32.709 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441653+0000", 2024-09-11T06:49:32.709 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:49:32.710 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.710 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.710 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:32.710 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441665+0000", 2024-09-11T06:49:32.710 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1653e-05 2024-09-11T06:49:32.710 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.710 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.710 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:32.710 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441684+0000", 2024-09-11T06:49:32.710 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9490000000000001e-05 2024-09-11T06:49:32.710 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.710 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.710 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:32.710 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441699+0000", 2024-09-11T06:49:32.710 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5044e-05 2024-09-11T06:49:32.711 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.711 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.711 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:32.711 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457297+0000", 2024-09-11T06:49:32.711 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015598241000000001 2024-09-11T06:49:32.711 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.711 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.711 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:32.711 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457577+0000", 2024-09-11T06:49:32.711 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027930799999999998 2024-09-11T06:49:32.711 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.711 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.711 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:32.711 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457957+0000", 2024-09-11T06:49:32.711 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00038088 2024-09-11T06:49:32.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.712 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.712 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:32.712 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460432+0000", 2024-09-11T06:49:32.712 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024743349999999998 2024-09-11T06:49:32.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.712 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.712 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:32.712 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460584+0000", 2024-09-11T06:49:32.712 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015232900000000001 2024-09-11T06:49:32.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.712 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.712 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:32.712 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460646+0000", 2024-09-11T06:49:32.713 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1883000000000004e-05 2024-09-11T06:49:32.713 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.713 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.713 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:32.713 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460696+0000", 2024-09-11T06:49:32.713 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9589999999999998e-05 2024-09-11T06:49:32.713 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.713 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:32.713 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.713 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.713 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.713 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:32.713 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:49:32.714 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 26.201571682000001, 2024-09-11T06:49:32.714 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.021834473, 2024-09-11T06:49:32.714 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:32.714 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:32.714 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:32.714 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:32.714 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:32.714 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:32.714 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 117 2024-09-11T06:49:32.714 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.714 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:32.714 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.714 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:32.714 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:49:32.715 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.715 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.715 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.715 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:32.715 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:49:32.715 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.715 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.715 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.715 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:32.715 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441725+0000", 2024-09-11T06:49:32.715 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5069999999999999e-06 2024-09-11T06:49:32.715 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.715 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.715 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:32.715 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441738+0000", 2024-09-11T06:49:32.716 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2813000000000001e-05 2024-09-11T06:49:32.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.716 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.716 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:32.716 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441759+0000", 2024-09-11T06:49:32.716 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1246999999999998e-05 2024-09-11T06:49:32.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.716 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.716 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:32.716 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441779+0000", 2024-09-11T06:49:32.716 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0392999999999999e-05 2024-09-11T06:49:32.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.716 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.716 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:32.717 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458804+0000", 2024-09-11T06:49:32.717 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.017024182999999998 2024-09-11T06:49:32.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.717 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.717 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:32.717 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458977+0000", 2024-09-11T06:49:32.717 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000173926 2024-09-11T06:49:32.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.717 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.717 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:32.717 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459170+0000", 2024-09-11T06:49:32.717 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000192184 2024-09-11T06:49:32.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.717 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.718 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:32.718 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.462300+0000", 2024-09-11T06:49:32.718 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031304969999999999 2024-09-11T06:49:32.718 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.718 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.718 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:32.718 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463408+0000", 2024-09-11T06:49:32.718 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011077839999999999 2024-09-11T06:49:32.718 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.718 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.718 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:32.718 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463497+0000", 2024-09-11T06:49:32.718 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8828999999999999e-05 2024-09-11T06:49:32.718 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.719 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.719 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:32.719 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463558+0000", 2024-09-11T06:49:32.719 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1110000000000003e-05 2024-09-11T06:49:32.719 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.719 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:32.719 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.719 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.719 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.719 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:32.719 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:49:32.719 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 25.825063994000001, 2024-09-11T06:49:32.719 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012216898, 2024-09-11T06:49:32.719 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:32.720 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:32.720 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:32.720 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:32.720 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:32.720 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:32.720 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 137 2024-09-11T06:49:32.720 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.720 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:32.720 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.720 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:32.720 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:49:32.720 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.720 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.720 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.721 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:32.721 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:49:32.721 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.721 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.721 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:32.721 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818232+0000", 2024-09-11T06:49:32.721 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2839999999999999e-06 2024-09-11T06:49:32.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.721 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.721 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:32.721 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818326+0000", 2024-09-11T06:49:32.721 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3531999999999998e-05 2024-09-11T06:49:32.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.722 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.722 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:32.722 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818337+0000", 2024-09-11T06:49:32.722 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0655999999999999e-05 2024-09-11T06:49:32.722 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.722 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.722 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:32.722 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818342+0000", 2024-09-11T06:49:32.722 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4600000000000002e-06 2024-09-11T06:49:32.722 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.722 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.722 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:32.722 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823890+0000", 2024-09-11T06:49:32.722 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0055480700000000004 2024-09-11T06:49:32.723 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.723 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.723 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:32.723 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823953+0000", 2024-09-11T06:49:32.723 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2848999999999994e-05 2024-09-11T06:49:32.723 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.723 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.723 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:32.723 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824074+0000", 2024-09-11T06:49:32.723 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121309 2024-09-11T06:49:32.723 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.723 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.723 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:32.723 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828043+0000", 2024-09-11T06:49:32.724 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0039685240000000002 2024-09-11T06:49:32.724 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.724 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.724 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:32.724 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830293+0000", 2024-09-11T06:49:32.724 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0022503179999999998 2024-09-11T06:49:32.724 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.724 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.724 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:32.724 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830366+0000", 2024-09-11T06:49:32.724 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2539999999999993e-05 2024-09-11T06:49:32.724 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.724 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.724 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:32.725 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830448+0000", 2024-09-11T06:49:32.725 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2355999999999996e-05 2024-09-11T06:49:32.725 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.725 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:32.725 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.725 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.725 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.725 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:32.725 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:32.725 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 25.824930179999999, 2024-09-11T06:49:32.725 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013136297, 2024-09-11T06:49:32.725 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:32.725 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:32.726 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:32.726 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:32.726 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:32.726 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:32.726 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 138 2024-09-11T06:49:32.726 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.726 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:32.726 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.726 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:32.726 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:32.726 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.726 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.726 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.726 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:32.726 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:32.727 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.727 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.727 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.727 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:32.727 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:32.727 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.89e-07 2024-09-11T06:49:32.727 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.727 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.727 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:32.727 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818369+0000", 2024-09-11T06:49:32.727 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4170000000000001e-06 2024-09-11T06:49:32.727 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.727 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.727 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:32.728 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818376+0000", 2024-09-11T06:49:32.728 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4270000000000002e-06 2024-09-11T06:49:32.728 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.728 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.728 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:32.728 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818383+0000", 2024-09-11T06:49:32.728 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3060000000000003e-06 2024-09-11T06:49:32.728 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.728 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.728 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:32.728 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824783+0000", 2024-09-11T06:49:32.728 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064003020000000001 2024-09-11T06:49:32.728 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.728 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.728 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:32.729 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825013+0000", 2024-09-11T06:49:32.729 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023039499999999999 2024-09-11T06:49:32.729 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.729 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.729 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:32.729 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825484+0000", 2024-09-11T06:49:32.729 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000470562 2024-09-11T06:49:32.729 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.729 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.729 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:32.729 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828633+0000", 2024-09-11T06:49:32.729 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031496689999999999 2024-09-11T06:49:32.729 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.729 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.730 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:32.730 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831413+0000", 2024-09-11T06:49:32.730 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027793219999999999 2024-09-11T06:49:32.730 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.730 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.730 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:32.730 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831450+0000", 2024-09-11T06:49:32.730 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7370999999999998e-05 2024-09-11T06:49:32.730 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.730 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.730 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:32.730 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831501+0000", 2024-09-11T06:49:32.731 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1036999999999999e-05 2024-09-11T06:49:32.731 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.731 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:32.731 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.731 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.731 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.731 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:32.731 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:49:32.731 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 25.824905699999999, 2024-09-11T06:49:32.731 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013223011, 2024-09-11T06:49:32.731 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:32.731 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:32.731 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:32.731 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:32.732 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:32.732 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:32.732 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 139 2024-09-11T06:49:32.732 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.732 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:32.732 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.732 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:32.732 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:49:32.732 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.732 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.732 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.732 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:32.732 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:49:32.733 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.733 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.733 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.733 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:32.733 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818390+0000", 2024-09-11T06:49:32.733 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5600000000000001e-07 2024-09-11T06:49:32.733 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.733 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.733 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:32.733 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818394+0000", 2024-09-11T06:49:32.733 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5929999999999999e-06 2024-09-11T06:49:32.733 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.733 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.733 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:32.733 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818399+0000", 2024-09-11T06:49:32.734 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7740000000000004e-06 2024-09-11T06:49:32.734 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.734 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.734 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:32.734 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818412+0000", 2024-09-11T06:49:32.734 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2625e-05 2024-09-11T06:49:32.734 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.734 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.734 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:32.734 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826205+0000", 2024-09-11T06:49:32.734 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0077930730000000002 2024-09-11T06:49:32.734 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.734 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.734 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:32.735 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826281+0000", 2024-09-11T06:49:32.735 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5876000000000003e-05 2024-09-11T06:49:32.735 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.735 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.735 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:32.735 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826384+0000", 2024-09-11T06:49:32.735 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103838 2024-09-11T06:49:32.735 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.735 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.735 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:32.735 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830642+0000", 2024-09-11T06:49:32.735 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004257083 2024-09-11T06:49:32.735 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.735 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.736 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:32.736 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831509+0000", 2024-09-11T06:49:32.736 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086730900000000003 2024-09-11T06:49:32.736 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.736 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.736 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:32.736 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831547+0000", 2024-09-11T06:49:32.736 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7793000000000002e-05 2024-09-11T06:49:32.736 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.736 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.736 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:32.736 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831612+0000", 2024-09-11T06:49:32.736 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5691000000000005e-05 2024-09-11T06:49:32.736 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.737 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:32.737 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.737 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.737 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.737 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:32.737 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:32.737 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 25.824877006000001, 2024-09-11T06:49:32.737 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016098024999999998, 2024-09-11T06:49:32.737 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:32.737 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:32.737 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:32.737 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:32.737 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:32.737 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:32.738 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 140 2024-09-11T06:49:32.738 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.738 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:32.738 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.738 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:32.738 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:32.738 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.738 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.738 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.738 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:32.739 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:32.739 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.739 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.739 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:32.739 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:32.739 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5199999999999998e-07 2024-09-11T06:49:32.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.739 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.739 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:32.739 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818479+0000", 2024-09-11T06:49:32.739 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0368000000000002e-05 2024-09-11T06:49:32.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.739 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.740 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:32.740 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818487+0000", 2024-09-11T06:49:32.740 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2339999999999994e-06 2024-09-11T06:49:32.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.740 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.740 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:32.740 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818491+0000", 2024-09-11T06:49:32.740 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4329999999999997e-06 2024-09-11T06:49:32.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.740 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.740 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:32.740 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828080+0000", 2024-09-11T06:49:32.740 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0095890449999999992 2024-09-11T06:49:32.741 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.741 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.741 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:32.741 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828130+0000", 2024-09-11T06:49:32.741 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9239000000000001e-05 2024-09-11T06:49:32.741 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.741 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.741 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:32.741 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828208+0000", 2024-09-11T06:49:32.741 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.852e-05 2024-09-11T06:49:32.741 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.741 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.741 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:32.742 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831625+0000", 2024-09-11T06:49:32.742 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034169410000000002 2024-09-11T06:49:32.742 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.742 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.742 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:32.742 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834411+0000", 2024-09-11T06:49:32.742 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002786111 2024-09-11T06:49:32.742 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.742 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.742 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:32.742 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834457+0000", 2024-09-11T06:49:32.742 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6041000000000002e-05 2024-09-11T06:49:32.742 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.743 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.743 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:32.743 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834516+0000", 2024-09-11T06:49:32.743 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8740999999999999e-05 2024-09-11T06:49:32.743 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.743 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:32.743 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.743 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.743 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.743 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:32.743 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:49:32.743 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 25.824795939000001, 2024-09-11T06:49:32.743 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016588772000000002, 2024-09-11T06:49:32.743 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:32.744 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:32.744 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:32.744 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:32.744 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:32.744 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:32.744 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 141 2024-09-11T06:49:32.744 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.744 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:32.744 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.744 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:32.744 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:49:32.744 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.744 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.744 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.745 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:32.745 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:49:32.745 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.745 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.745 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.745 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:32.745 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818500+0000", 2024-09-11T06:49:32.745 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0800000000000005e-07 2024-09-11T06:49:32.745 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.745 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.745 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:32.745 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818505+0000", 2024-09-11T06:49:32.745 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2429999999999996e-06 2024-09-11T06:49:32.745 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.746 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.746 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:32.746 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818510+0000", 2024-09-11T06:49:32.746 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9339999999999997e-06 2024-09-11T06:49:32.746 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.746 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.746 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:32.746 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818514+0000", 2024-09-11T06:49:32.746 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7270000000000001e-06 2024-09-11T06:49:32.746 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.746 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.746 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:32.746 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828711+0000", 2024-09-11T06:49:32.746 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010197031 2024-09-11T06:49:32.747 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.747 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.747 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:32.747 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828853+0000", 2024-09-11T06:49:32.747 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014201200000000001 2024-09-11T06:49:32.747 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.747 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.747 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:32.747 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.829115+0000", 2024-09-11T06:49:32.747 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026257500000000003 2024-09-11T06:49:32.747 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.747 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.747 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:32.747 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831629+0000", 2024-09-11T06:49:32.748 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025138169999999998 2024-09-11T06:49:32.748 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.748 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.748 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:32.748 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834991+0000", 2024-09-11T06:49:32.748 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033624459999999998 2024-09-11T06:49:32.748 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.748 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.748 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:32.748 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835032+0000", 2024-09-11T06:49:32.748 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1001000000000003e-05 2024-09-11T06:49:32.748 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.748 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.748 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:32.748 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835088+0000", 2024-09-11T06:49:32.749 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5478e-05 2024-09-11T06:49:32.749 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.749 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:32.749 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.749 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.749 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.749 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:49:32.749 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:49:32.749 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 25.201161035999998, 2024-09-11T06:49:32.749 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011161912, 2024-09-11T06:49:32.749 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:32.749 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:32.749 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:32.749 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:32.750 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:32.750 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:32.750 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 158 2024-09-11T06:49:32.750 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.750 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:32.750 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.750 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:32.750 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:49:32.750 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.750 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.750 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.750 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:32.750 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:49:32.750 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.751 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.751 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.751 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:32.751 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442135+0000", 2024-09-11T06:49:32.751 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3e-07 2024-09-11T06:49:32.751 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.751 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.751 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:32.751 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442259+0000", 2024-09-11T06:49:32.751 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000123903 2024-09-11T06:49:32.751 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.751 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.751 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:32.751 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442289+0000", 2024-09-11T06:49:32.751 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0833000000000001e-05 2024-09-11T06:49:32.752 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.752 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.752 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:32.752 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442301+0000", 2024-09-11T06:49:32.752 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1433e-05 2024-09-11T06:49:32.752 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.752 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.752 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:32.752 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449796+0000", 2024-09-11T06:49:32.752 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0074954959999999999 2024-09-11T06:49:32.752 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.752 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.752 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:32.753 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449917+0000", 2024-09-11T06:49:32.753 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121033 2024-09-11T06:49:32.753 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.753 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.753 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:32.753 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.450166+0000", 2024-09-11T06:49:32.753 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00024832400000000002 2024-09-11T06:49:32.753 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.753 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.753 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:32.753 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.452630+0000", 2024-09-11T06:49:32.753 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002464188 2024-09-11T06:49:32.753 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.753 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.754 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:32.754 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453149+0000", 2024-09-11T06:49:32.754 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00051860999999999997 2024-09-11T06:49:32.754 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.754 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.754 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:32.754 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453234+0000", 2024-09-11T06:49:32.754 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.5865000000000005e-05 2024-09-11T06:49:32.754 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.754 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.754 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:32.754 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453296+0000", 2024-09-11T06:49:32.754 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1496999999999998e-05 2024-09-11T06:49:32.754 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.755 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:32.755 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.755 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.755 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.755 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:49:32.755 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:49:32.755 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 25.200974934000001, 2024-09-11T06:49:32.755 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011561900999999999, 2024-09-11T06:49:32.755 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:32.755 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:32.755 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:32.755 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:32.755 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:32.755 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:32.756 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 159 2024-09-11T06:49:32.756 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.756 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:32.756 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.756 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:32.756 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:49:32.756 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.756 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.756 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.756 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:32.756 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:49:32.756 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:32.756 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.757 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.757 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:32.757 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442321+0000", 2024-09-11T06:49:32.757 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8400000000000003e-07 2024-09-11T06:49:32.757 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.757 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.757 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:32.757 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442328+0000", 2024-09-11T06:49:32.757 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5130000000000002e-06 2024-09-11T06:49:32.757 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.757 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.757 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:32.757 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442341+0000", 2024-09-11T06:49:32.757 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.363e-05 2024-09-11T06:49:32.757 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.758 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.758 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:32.758 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442349+0000", 2024-09-11T06:49:32.758 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4760000000000001e-06 2024-09-11T06:49:32.758 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.758 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.758 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:32.758 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451257+0000", 2024-09-11T06:49:32.758 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089088569999999992 2024-09-11T06:49:32.758 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.758 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.758 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:32.758 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451389+0000", 2024-09-11T06:49:32.758 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013110899999999999 2024-09-11T06:49:32.759 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.759 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.759 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:32.759 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451608+0000", 2024-09-11T06:49:32.759 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000219588 2024-09-11T06:49:32.759 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.759 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.759 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:32.759 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453517+0000", 2024-09-11T06:49:32.759 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0019088779999999999 2024-09-11T06:49:32.759 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.759 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.759 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:32.759 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453727+0000", 2024-09-11T06:49:32.760 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020955100000000001 2024-09-11T06:49:32.760 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.760 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.760 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:32.760 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453815+0000", 2024-09-11T06:49:32.760 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8858999999999997e-05 2024-09-11T06:49:32.760 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:32.760 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:32.760 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:32.760 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453882+0000", 2024-09-11T06:49:32.760 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.6556000000000004e-05 2024-09-11T06:49:32.760 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.760 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:32.760 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.761 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:32.761 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:32.761 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:32.762 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:49:32.763 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:32.762+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:32.801 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:49:32.863 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:32.862+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:32.927 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:32.927 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:32.927 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:49:32.927 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:32.939 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:49:32.940 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:49:32.941 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:49:32.956 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:49:32.956 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:49:32.971 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:49:32.971 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:49:32.988 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:49:32.988 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:49:32.990 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:32.990+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:33.006 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:49:33.006 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:49:33.022 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:49:33.023 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:49:33.061 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:33.061 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:33.064 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:33.062+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:33.074 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:49:33.166 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:33.162+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:33.235 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:33.235 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:33.248 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:49:33.266 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:33.266+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:33.346 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.2 to 1 (10000) 2024-09-11T06:49:33.357 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:33.357 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:33.367 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:33.366+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:33.368 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-11T06:49:33.369 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:49:33.468 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:33.466+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:33.490 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:33.490 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:33.504 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:49:33.595 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:33.594+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:33.669 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:33.666+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:33.673 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:33.673 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:33.687 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:49:33.770 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:33.766+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:33.814 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:33.814 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:33.828 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:49:33.871 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:33.870+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:33.972 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:33.970+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:33.989 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:33.990 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:34.004 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:49:34.073 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:34.074+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:34.126 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:34.127 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:34.141 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:49:34.174 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:34.170+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:34.262 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:34.262 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:34.275 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:34.274+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:34.276 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:49:34.352 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.3 to 1 (10000) 2024-09-11T06:49:34.363 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T06:49:34.372 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-11T06:49:34.377 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:34.374+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:34.398 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:34.398 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:34.414 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:49:34.477 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:34.474+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:34.575 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:34.575 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:34.578 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:34.578+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:34.589 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:49:34.679 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:34.678+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:34.699 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:34.700 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:34.713 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:49:34.779 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:34.778+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:34.818 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:34.818 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:34.830 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:49:34.880 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:34.882+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:34.948 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:34.948 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:34.962 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:49:34.981 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:34.978+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:35.038 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:49:35.083 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:35.082+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:35.095 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:35.095 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:35.110 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:49:35.183 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:35.182+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:35.184 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:49:35.185 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:49:35.201 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:49:35.201 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:49:35.217 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:49:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:49:35.226 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:35.227 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:35.238 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:49:35.238 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:49:35.242 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:49:35.256 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:49:35.256 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:49:35.273 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:49:35.273 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:49:35.286 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:35.286+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:35.363 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is set 2024-09-11T06:49:35.363 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.4 to 1 (10000) 2024-09-11T06:49:35.369 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:35.369 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:49:35.369 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:49:35.369 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:49:35.369 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.369 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:35.369 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:49:35.370 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 29.199290850000001, 2024-09-11T06:49:35.370 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011420669, 2024-09-11T06:49:35.370 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:35.370 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:35.370 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:35.370 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:35.370 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:35.370 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:35.370 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 37 2024-09-11T06:49:35.370 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.370 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:35.370 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.370 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:35.370 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:49:35.371 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.371 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.371 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.371 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:35.371 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:49:35.371 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.371 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.371 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.371 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:35.371 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170509+0000", 2024-09-11T06:49:35.371 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.13e-06 2024-09-11T06:49:35.371 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.371 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.371 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:35.371 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170521+0000", 2024-09-11T06:49:35.372 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1776999999999999e-05 2024-09-11T06:49:35.372 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.372 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.372 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:35.372 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170538+0000", 2024-09-11T06:49:35.372 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7413000000000001e-05 2024-09-11T06:49:35.372 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.372 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.372 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:35.372 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170570+0000", 2024-09-11T06:49:35.372 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2313e-05 2024-09-11T06:49:35.372 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.372 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.372 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:35.373 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179768+0000", 2024-09-11T06:49:35.373 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0091978010000000002 2024-09-11T06:49:35.373 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.373 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.373 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:35.373 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179951+0000", 2024-09-11T06:49:35.373 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018227900000000001 2024-09-11T06:49:35.373 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.373 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.373 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:35.373 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.180175+0000", 2024-09-11T06:49:35.373 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022432800000000001 2024-09-11T06:49:35.373 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.374 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.374 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:35.374 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181112+0000", 2024-09-11T06:49:35.374 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00093712500000000002 2024-09-11T06:49:35.374 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.374 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.374 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:35.374 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181789+0000", 2024-09-11T06:49:35.374 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00067705100000000002 2024-09-11T06:49:35.374 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.374 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.374 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:35.374 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181871+0000", 2024-09-11T06:49:35.375 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1816000000000004e-05 2024-09-11T06:49:35.375 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.375 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.375 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:35.375 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181929+0000", 2024-09-11T06:49:35.375 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7636000000000002e-05 2024-09-11T06:49:35.375 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.375 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:35.375 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.375 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.375 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.375 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:35.375 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:49:35.375 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 29.098031889000001, 2024-09-11T06:49:35.375 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283336, 2024-09-11T06:49:35.376 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:35.376 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:35.376 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:35.376 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:35.376 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:35.376 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:35.376 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 87 2024-09-11T06:49:35.376 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.376 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:35.376 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.376 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:35.376 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:49:35.376 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.376 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.377 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.377 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:35.377 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:49:35.377 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.377 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.377 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.377 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:35.377 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271768+0000", 2024-09-11T06:49:35.377 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7599999999999996e-07 2024-09-11T06:49:35.377 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.377 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.377 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:35.377 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271860+0000", 2024-09-11T06:49:35.378 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2639999999999994e-05 2024-09-11T06:49:35.378 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.378 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.378 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:35.378 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271873+0000", 2024-09-11T06:49:35.378 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3139000000000001e-05 2024-09-11T06:49:35.378 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.378 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.378 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:35.378 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271882+0000", 2024-09-11T06:49:35.378 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2369999999999992e-06 2024-09-11T06:49:35.378 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.378 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.379 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:35.379 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277150+0000", 2024-09-11T06:49:35.379 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052684419999999999 2024-09-11T06:49:35.379 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.379 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.379 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:35.379 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277273+0000", 2024-09-11T06:49:35.379 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012278899999999999 2024-09-11T06:49:35.379 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.379 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.379 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:35.379 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277466+0000", 2024-09-11T06:49:35.379 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193578 2024-09-11T06:49:35.379 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.379 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.380 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:35.380 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.280808+0000", 2024-09-11T06:49:35.380 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003341971 2024-09-11T06:49:35.380 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.380 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.380 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:35.380 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284419+0000", 2024-09-11T06:49:35.380 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003610982 2024-09-11T06:49:35.380 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.380 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.380 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:35.380 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284531+0000", 2024-09-11T06:49:35.380 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011121299999999999 2024-09-11T06:49:35.380 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.381 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.381 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:35.381 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284600+0000", 2024-09-11T06:49:35.381 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9592999999999996e-05 2024-09-11T06:49:35.381 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.381 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:35.381 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.381 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.381 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.381 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:35.381 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:49:35.381 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 29.097901637, 2024-09-11T06:49:35.381 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283314, 2024-09-11T06:49:35.382 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:35.382 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:35.382 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:35.382 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:35.382 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:35.382 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:35.382 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 88 2024-09-11T06:49:35.382 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.382 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:35.382 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.382 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:35.382 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:49:35.382 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.382 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.382 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.383 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:35.383 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:49:35.383 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.383 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.383 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.383 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:35.383 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271898+0000", 2024-09-11T06:49:35.383 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2999999999999999e-07 2024-09-11T06:49:35.383 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.383 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.383 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:35.383 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271904+0000", 2024-09-11T06:49:35.383 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.767e-06 2024-09-11T06:49:35.384 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.384 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.384 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:35.384 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271913+0000", 2024-09-11T06:49:35.384 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7200000000000001e-06 2024-09-11T06:49:35.384 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.384 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.384 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:35.384 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271920+0000", 2024-09-11T06:49:35.384 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7100000000000001e-06 2024-09-11T06:49:35.384 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.384 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.384 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:35.384 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278684+0000", 2024-09-11T06:49:35.385 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0067642650000000002 2024-09-11T06:49:35.385 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.385 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.385 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:35.385 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278838+0000", 2024-09-11T06:49:35.385 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015371499999999999 2024-09-11T06:49:35.385 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.385 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.385 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:35.385 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.279040+0000", 2024-09-11T06:49:35.385 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020207499999999999 2024-09-11T06:49:35.385 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.385 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.385 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:35.386 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.281448+0000", 2024-09-11T06:49:35.386 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002407686 2024-09-11T06:49:35.386 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.386 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.386 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:35.386 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284615+0000", 2024-09-11T06:49:35.386 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031675499999999999 2024-09-11T06:49:35.386 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.386 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.386 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:35.386 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284682+0000", 2024-09-11T06:49:35.386 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7032000000000005e-05 2024-09-11T06:49:35.386 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.386 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.387 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:35.387 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284730+0000", 2024-09-11T06:49:35.387 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7790000000000002e-05 2024-09-11T06:49:35.387 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.387 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:35.387 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.387 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.387 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.387 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:35.387 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:49:35.387 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 28.930000414999999, 2024-09-11T06:49:35.387 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011114877, 2024-09-11T06:49:35.387 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:35.388 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:35.388 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:35.388 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:35.388 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:35.388 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:35.388 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 107 2024-09-11T06:49:35.388 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.388 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:35.388 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.388 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:35.388 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:49:35.388 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.388 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.388 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.389 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:35.389 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:49:35.389 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.389 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.389 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.389 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:35.389 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439799+0000", 2024-09-11T06:49:35.389 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.16e-07 2024-09-11T06:49:35.389 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.389 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.389 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:35.389 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439806+0000", 2024-09-11T06:49:35.389 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3640000000000004e-06 2024-09-11T06:49:35.389 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.390 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.390 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:35.390 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439816+0000", 2024-09-11T06:49:35.390 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0485999999999999e-05 2024-09-11T06:49:35.390 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.390 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.390 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:35.390 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439824+0000", 2024-09-11T06:49:35.390 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.6699999999999994e-06 2024-09-11T06:49:35.390 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.390 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.391 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:35.391 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443328+0000", 2024-09-11T06:49:35.391 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035044329999999999 2024-09-11T06:49:35.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.391 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.391 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:35.391 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443461+0000", 2024-09-11T06:49:35.391 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013272199999999999 2024-09-11T06:49:35.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.391 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.392 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:35.392 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443678+0000", 2024-09-11T06:49:35.392 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021674500000000001 2024-09-11T06:49:35.392 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.392 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.392 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:35.392 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449635+0000", 2024-09-11T06:49:35.392 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0059572119999999999 2024-09-11T06:49:35.392 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.392 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.392 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:35.392 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450750+0000", 2024-09-11T06:49:35.392 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001115654 2024-09-11T06:49:35.392 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.393 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.393 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:35.393 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450832+0000", 2024-09-11T06:49:35.393 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1188999999999998e-05 2024-09-11T06:49:35.393 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.393 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.393 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:35.393 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450913+0000", 2024-09-11T06:49:35.393 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1485999999999999e-05 2024-09-11T06:49:35.393 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.393 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:35.393 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.393 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.394 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.394 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:35.394 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:49:35.394 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 28.929409440000001, 2024-09-11T06:49:35.394 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011236556, 2024-09-11T06:49:35.394 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:35.394 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:35.394 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:35.394 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:35.394 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:35.394 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:35.394 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 109 2024-09-11T06:49:35.394 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.395 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:35.395 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.395 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:35.395 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:49:35.395 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.395 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.395 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:35.395 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:49:35.395 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.395 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.395 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:35.395 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440391+0000", 2024-09-11T06:49:35.396 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8360000000000001e-06 2024-09-11T06:49:35.396 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.396 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:35.396 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440560+0000", 2024-09-11T06:49:35.396 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000169085 2024-09-11T06:49:35.396 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.396 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:35.396 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440585+0000", 2024-09-11T06:49:35.396 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4530000000000001e-05 2024-09-11T06:49:35.396 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.396 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:35.397 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440606+0000", 2024-09-11T06:49:35.397 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1277e-05 2024-09-11T06:49:35.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.397 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.397 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:35.397 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446479+0000", 2024-09-11T06:49:35.397 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0058734989999999999 2024-09-11T06:49:35.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.397 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.397 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:35.397 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446649+0000", 2024-09-11T06:49:35.397 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016948199999999999 2024-09-11T06:49:35.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.397 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.398 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:35.398 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446854+0000", 2024-09-11T06:49:35.398 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020491500000000001 2024-09-11T06:49:35.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.398 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.398 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:35.398 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451380+0000", 2024-09-11T06:49:35.398 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0045264110000000001 2024-09-11T06:49:35.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.398 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.398 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:35.398 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451479+0000", 2024-09-11T06:49:35.398 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.8975999999999996e-05 2024-09-11T06:49:35.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.398 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.399 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:35.399 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451562+0000", 2024-09-11T06:49:35.399 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3213000000000004e-05 2024-09-11T06:49:35.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.399 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.399 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:35.399 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451626+0000", 2024-09-11T06:49:35.399 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3331999999999996e-05 2024-09-11T06:49:35.399 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.399 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:35.399 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.399 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.400 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:35.400 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:49:35.400 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 28.929160312, 2024-09-11T06:49:35.400 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011252866, 2024-09-11T06:49:35.400 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:35.400 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:35.400 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:35.400 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:35.400 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:35.400 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:35.400 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 110 2024-09-11T06:49:35.400 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.400 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:35.400 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.401 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:35.401 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:49:35.401 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.401 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:35.401 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:49:35.401 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.401 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:35.401 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440640+0000", 2024-09-11T06:49:35.401 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5430000000000001e-06 2024-09-11T06:49:35.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.402 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:35.402 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440651+0000", 2024-09-11T06:49:35.402 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0672999999999999e-05 2024-09-11T06:49:35.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.402 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:35.402 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440670+0000", 2024-09-11T06:49:35.402 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9726999999999999e-05 2024-09-11T06:49:35.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.402 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:35.402 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440691+0000", 2024-09-11T06:49:35.403 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0526e-05 2024-09-11T06:49:35.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.403 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.403 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:35.403 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448191+0000", 2024-09-11T06:49:35.403 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0075005990000000002 2024-09-11T06:49:35.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.403 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.403 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:35.403 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448390+0000", 2024-09-11T06:49:35.403 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000198815 2024-09-11T06:49:35.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.403 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.403 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:35.403 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448735+0000", 2024-09-11T06:49:35.404 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034424999999999999 2024-09-11T06:49:35.404 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.404 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.404 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:35.404 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451742+0000", 2024-09-11T06:49:35.404 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030077440000000001 2024-09-11T06:49:35.404 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.404 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.404 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:35.404 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451761+0000", 2024-09-11T06:49:35.404 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9196999999999999e-05 2024-09-11T06:49:35.404 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.404 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.404 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:35.405 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451834+0000", 2024-09-11T06:49:35.405 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2071999999999997e-05 2024-09-11T06:49:35.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.405 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.405 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:35.405 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451891+0000", 2024-09-11T06:49:35.405 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7720000000000003e-05 2024-09-11T06:49:35.405 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.405 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:35.405 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.405 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.405 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:35.405 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:49:35.406 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 28.929052436999999, 2024-09-11T06:49:35.406 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014944941, 2024-09-11T06:49:35.406 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:35.406 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:35.406 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:35.406 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:35.406 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:35.406 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:35.406 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 111 2024-09-11T06:49:35.406 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.406 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:35.406 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.406 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:35.406 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:49:35.407 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.407 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.407 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:35.407 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:49:35.407 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.407 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.407 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:35.408 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440748+0000", 2024-09-11T06:49:35.408 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:49:35.408 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.408 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.408 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:35.408 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440760+0000", 2024-09-11T06:49:35.408 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1897e-05 2024-09-11T06:49:35.408 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.408 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.408 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:35.408 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440782+0000", 2024-09-11T06:49:35.408 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2552999999999999e-05 2024-09-11T06:49:35.408 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.408 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.409 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:35.409 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440793+0000", 2024-09-11T06:49:35.409 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0643e-05 2024-09-11T06:49:35.409 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.409 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.409 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:35.409 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449755+0000", 2024-09-11T06:49:35.409 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089622620000000004 2024-09-11T06:49:35.409 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.409 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.409 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:35.409 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449918+0000", 2024-09-11T06:49:35.409 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163086 2024-09-11T06:49:35.409 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.410 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.410 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:35.410 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450150+0000", 2024-09-11T06:49:35.410 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023204500000000001 2024-09-11T06:49:35.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.410 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.410 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:35.410 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454533+0000", 2024-09-11T06:49:35.410 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0043831349999999998 2024-09-11T06:49:35.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.410 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.410 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:35.410 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455511+0000", 2024-09-11T06:49:35.410 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0009776050000000001 2024-09-11T06:49:35.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.411 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.411 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:35.411 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455597+0000", 2024-09-11T06:49:35.411 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6446000000000005e-05 2024-09-11T06:49:35.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.411 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.411 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:35.411 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455691+0000", 2024-09-11T06:49:35.411 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3832000000000006e-05 2024-09-11T06:49:35.411 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.411 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:35.411 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.411 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.412 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:35.412 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:49:35.412 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 28.928982674, 2024-09-11T06:49:35.412 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018885411000000001, 2024-09-11T06:49:35.412 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:35.412 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:35.412 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:35.412 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:35.412 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:35.412 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:35.412 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 112 2024-09-11T06:49:35.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.412 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:35.412 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.413 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:35.413 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:49:35.413 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.413 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:35.413 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:49:35.413 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.413 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:35.413 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440817+0000", 2024-09-11T06:49:35.413 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:49:35.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.414 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.414 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:35.414 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441003+0000", 2024-09-11T06:49:35.414 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018661200000000001 2024-09-11T06:49:35.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.414 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.414 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:35.414 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441027+0000", 2024-09-11T06:49:35.414 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3920000000000001e-05 2024-09-11T06:49:35.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.414 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.414 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:35.414 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441062+0000", 2024-09-11T06:49:35.414 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4663000000000003e-05 2024-09-11T06:49:35.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.415 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.415 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:35.415 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451981+0000", 2024-09-11T06:49:35.415 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010918926000000001 2024-09-11T06:49:35.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.415 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.415 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:35.415 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452111+0000", 2024-09-11T06:49:35.415 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129529 2024-09-11T06:49:35.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.415 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.415 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:35.415 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452311+0000", 2024-09-11T06:49:35.415 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020013799999999999 2024-09-11T06:49:35.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.416 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.416 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:35.416 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455809+0000", 2024-09-11T06:49:35.416 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003498443 2024-09-11T06:49:35.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.416 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.416 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:35.416 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459566+0000", 2024-09-11T06:49:35.416 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003757271 2024-09-11T06:49:35.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.416 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.416 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:35.417 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459640+0000", 2024-09-11T06:49:35.417 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3949000000000006e-05 2024-09-11T06:49:35.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.417 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.417 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:35.417 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459701+0000", 2024-09-11T06:49:35.417 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1117000000000005e-05 2024-09-11T06:49:35.417 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.417 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:35.417 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.417 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.417 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:35.417 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:49:35.417 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 28.928703822999999, 2024-09-11T06:49:35.418 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018760915, 2024-09-11T06:49:35.418 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:35.418 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:35.418 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:35.418 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:35.418 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:35.418 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:35.418 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 113 2024-09-11T06:49:35.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.418 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:35.418 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.418 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:35.418 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:49:35.418 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.418 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.419 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:35.419 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:49:35.419 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.419 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.419 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:35.419 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441096+0000", 2024-09-11T06:49:35.419 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4130000000000001e-06 2024-09-11T06:49:35.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.419 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.419 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:35.419 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441110+0000", 2024-09-11T06:49:35.419 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3900000000000001e-05 2024-09-11T06:49:35.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.419 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.419 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:35.420 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441131+0000", 2024-09-11T06:49:35.420 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0636999999999999e-05 2024-09-11T06:49:35.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.420 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.420 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:35.420 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441147+0000", 2024-09-11T06:49:35.420 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6512999999999999e-05 2024-09-11T06:49:35.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.420 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.420 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:35.420 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453274+0000", 2024-09-11T06:49:35.420 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012126764999999999 2024-09-11T06:49:35.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.420 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.420 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:35.421 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453498+0000", 2024-09-11T06:49:35.421 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000223568 2024-09-11T06:49:35.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.421 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.421 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:35.421 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453872+0000", 2024-09-11T06:49:35.421 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037467099999999998 2024-09-11T06:49:35.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.421 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.421 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:35.421 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458709+0000", 2024-09-11T06:49:35.421 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0048365680000000003 2024-09-11T06:49:35.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.421 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.422 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:35.422 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459716+0000", 2024-09-11T06:49:35.422 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010075240000000001 2024-09-11T06:49:35.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.422 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.422 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:35.422 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459779+0000", 2024-09-11T06:49:35.422 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2336999999999996e-05 2024-09-11T06:49:35.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.422 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.422 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:35.422 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459856+0000", 2024-09-11T06:49:35.422 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7019000000000002e-05 2024-09-11T06:49:35.422 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.422 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:35.422 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.423 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.423 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:35.423 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:35.423 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 28.928627083999999, 2024-09-11T06:49:35.423 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018963114, 2024-09-11T06:49:35.423 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:35.423 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:35.423 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:35.423 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:35.423 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:35.423 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:35.423 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 114 2024-09-11T06:49:35.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.423 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:35.423 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.424 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:35.424 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:35.424 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.424 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.424 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.424 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:35.424 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:35.424 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.424 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.424 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.424 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:35.424 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:35.424 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9299999999999996e-07 2024-09-11T06:49:35.424 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.424 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.425 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:35.425 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441180+0000", 2024-09-11T06:49:35.425 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7200000000000006e-06 2024-09-11T06:49:35.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.425 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.425 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:35.425 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441265+0000", 2024-09-11T06:49:35.425 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4753000000000007e-05 2024-09-11T06:49:35.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.425 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.425 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:35.425 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441288+0000", 2024-09-11T06:49:35.425 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2860000000000001e-05 2024-09-11T06:49:35.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.426 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.426 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:35.426 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454585+0000", 2024-09-11T06:49:35.426 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013297286 2024-09-11T06:49:35.426 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.426 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.426 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:35.426 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454714+0000", 2024-09-11T06:49:35.426 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000128465 2024-09-11T06:49:35.426 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.426 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.426 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:35.426 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454909+0000", 2024-09-11T06:49:35.426 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019522499999999999 2024-09-11T06:49:35.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.427 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.427 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:35.427 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459868+0000", 2024-09-11T06:49:35.427 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0049591239999999997 2024-09-11T06:49:35.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.427 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.427 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:35.427 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459988+0000", 2024-09-11T06:49:35.427 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000120532 2024-09-11T06:49:35.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.427 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.427 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:35.427 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460071+0000", 2024-09-11T06:49:35.428 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2256000000000007e-05 2024-09-11T06:49:35.428 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.428 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.428 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:35.428 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460135+0000", 2024-09-11T06:49:35.428 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3999999999999997e-05 2024-09-11T06:49:35.428 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.428 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:35.428 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.428 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.428 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.428 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:35.428 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:49:35.429 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 28.928469454999998, 2024-09-11T06:49:35.429 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.019036849000000002, 2024-09-11T06:49:35.429 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:35.429 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:35.429 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:35.429 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:35.429 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:35.429 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:35.429 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 115 2024-09-11T06:49:35.429 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.429 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:35.429 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.429 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:35.429 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:49:35.430 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.430 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.430 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.430 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:35.430 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:49:35.430 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.430 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.430 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.430 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:35.430 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441330+0000", 2024-09-11T06:49:35.430 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1930000000000001e-06 2024-09-11T06:49:35.430 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.430 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.430 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:35.431 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441544+0000", 2024-09-11T06:49:35.431 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021405799999999999 2024-09-11T06:49:35.431 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.431 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.431 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:35.431 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441572+0000", 2024-09-11T06:49:35.431 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7317e-05 2024-09-11T06:49:35.431 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.431 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.431 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:35.431 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441593+0000", 2024-09-11T06:49:35.431 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1702999999999999e-05 2024-09-11T06:49:35.431 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.431 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.432 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:35.432 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455891+0000", 2024-09-11T06:49:35.432 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01429708 2024-09-11T06:49:35.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.432 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.432 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:35.432 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456046+0000", 2024-09-11T06:49:35.432 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000155682 2024-09-11T06:49:35.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.432 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.432 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:35.432 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456242+0000", 2024-09-11T06:49:35.432 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000195692 2024-09-11T06:49:35.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.433 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.433 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:35.433 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459876+0000", 2024-09-11T06:49:35.433 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036336480000000002 2024-09-11T06:49:35.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.433 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.433 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:35.433 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460245+0000", 2024-09-11T06:49:35.433 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036901700000000002 2024-09-11T06:49:35.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.433 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.433 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:35.433 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460316+0000", 2024-09-11T06:49:35.433 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1846000000000003e-05 2024-09-11T06:49:35.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.434 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.434 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:35.434 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460366+0000", 2024-09-11T06:49:35.434 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9613000000000001e-05 2024-09-11T06:49:35.434 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.434 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:35.434 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.434 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.434 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:35.434 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:49:35.434 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 28.928147375000002, 2024-09-11T06:49:35.434 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01904432, 2024-09-11T06:49:35.434 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:35.434 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:35.435 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:35.435 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:35.435 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:35.435 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:35.435 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 116 2024-09-11T06:49:35.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.435 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:35.435 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.435 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:35.435 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:49:35.435 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.435 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.435 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:35.435 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:49:35.436 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.436 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.436 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:35.436 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441653+0000", 2024-09-11T06:49:35.436 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:49:35.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.436 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.436 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:35.436 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441665+0000", 2024-09-11T06:49:35.436 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1653e-05 2024-09-11T06:49:35.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.436 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.436 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:35.436 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441684+0000", 2024-09-11T06:49:35.437 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9490000000000001e-05 2024-09-11T06:49:35.437 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.437 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.437 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:35.437 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441699+0000", 2024-09-11T06:49:35.437 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5044e-05 2024-09-11T06:49:35.437 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.437 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.437 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:35.437 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457297+0000", 2024-09-11T06:49:35.437 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015598241000000001 2024-09-11T06:49:35.437 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.437 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.437 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:35.438 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457577+0000", 2024-09-11T06:49:35.438 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027930799999999998 2024-09-11T06:49:35.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.438 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.438 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:35.438 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457957+0000", 2024-09-11T06:49:35.438 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00038088 2024-09-11T06:49:35.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.438 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.438 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:35.438 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460432+0000", 2024-09-11T06:49:35.438 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024743349999999998 2024-09-11T06:49:35.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.438 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.439 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:35.439 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460584+0000", 2024-09-11T06:49:35.439 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015232900000000001 2024-09-11T06:49:35.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.439 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.439 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:35.439 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460646+0000", 2024-09-11T06:49:35.439 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1883000000000004e-05 2024-09-11T06:49:35.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.439 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.439 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:35.439 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460696+0000", 2024-09-11T06:49:35.439 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9589999999999998e-05 2024-09-11T06:49:35.440 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.440 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:35.440 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.440 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.440 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:35.440 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:49:35.440 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 28.928075309, 2024-09-11T06:49:35.440 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.021834473, 2024-09-11T06:49:35.440 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:35.440 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:35.440 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:35.440 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:35.440 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:35.441 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:35.441 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 117 2024-09-11T06:49:35.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.441 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:35.441 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.441 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:35.441 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:49:35.441 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.441 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.441 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:35.441 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:49:35.441 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.442 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.442 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:35.442 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441725+0000", 2024-09-11T06:49:35.442 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5069999999999999e-06 2024-09-11T06:49:35.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.442 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.442 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:35.442 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441738+0000", 2024-09-11T06:49:35.442 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2813000000000001e-05 2024-09-11T06:49:35.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.442 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.442 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:35.442 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441759+0000", 2024-09-11T06:49:35.442 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1246999999999998e-05 2024-09-11T06:49:35.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.442 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.443 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:35.443 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441779+0000", 2024-09-11T06:49:35.443 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0392999999999999e-05 2024-09-11T06:49:35.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.443 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:35.443 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458804+0000", 2024-09-11T06:49:35.443 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.017024182999999998 2024-09-11T06:49:35.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.443 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:35.443 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458977+0000", 2024-09-11T06:49:35.443 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000173926 2024-09-11T06:49:35.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.444 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:35.444 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459170+0000", 2024-09-11T06:49:35.444 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000192184 2024-09-11T06:49:35.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.444 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.444 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:35.444 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.462300+0000", 2024-09-11T06:49:35.444 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031304969999999999 2024-09-11T06:49:35.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.444 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.444 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:35.444 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463408+0000", 2024-09-11T06:49:35.444 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011077839999999999 2024-09-11T06:49:35.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.444 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.444 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:35.445 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463497+0000", 2024-09-11T06:49:35.445 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8828999999999999e-05 2024-09-11T06:49:35.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.445 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:35.445 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463558+0000", 2024-09-11T06:49:35.445 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1110000000000003e-05 2024-09-11T06:49:35.445 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.445 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:35.445 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.445 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:35.446 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:49:35.446 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 28.551567621, 2024-09-11T06:49:35.446 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012216898, 2024-09-11T06:49:35.446 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:35.446 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:35.446 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:35.446 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:35.446 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:35.446 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:35.446 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 137 2024-09-11T06:49:35.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.446 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:35.446 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.446 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:35.446 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:49:35.446 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.447 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:35.447 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:49:35.447 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.447 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:35.447 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818232+0000", 2024-09-11T06:49:35.447 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2839999999999999e-06 2024-09-11T06:49:35.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.447 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:35.447 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818326+0000", 2024-09-11T06:49:35.447 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3531999999999998e-05 2024-09-11T06:49:35.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.448 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:35.448 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818337+0000", 2024-09-11T06:49:35.448 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0655999999999999e-05 2024-09-11T06:49:35.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.448 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:35.448 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818342+0000", 2024-09-11T06:49:35.448 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4600000000000002e-06 2024-09-11T06:49:35.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.448 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:35.448 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823890+0000", 2024-09-11T06:49:35.448 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0055480700000000004 2024-09-11T06:49:35.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.449 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:35.449 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823953+0000", 2024-09-11T06:49:35.449 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2848999999999994e-05 2024-09-11T06:49:35.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.449 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:35.449 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824074+0000", 2024-09-11T06:49:35.449 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121309 2024-09-11T06:49:35.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.449 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:35.449 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828043+0000", 2024-09-11T06:49:35.449 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0039685240000000002 2024-09-11T06:49:35.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.450 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.450 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:35.450 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830293+0000", 2024-09-11T06:49:35.450 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0022503179999999998 2024-09-11T06:49:35.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.450 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.450 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:35.450 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830366+0000", 2024-09-11T06:49:35.450 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2539999999999993e-05 2024-09-11T06:49:35.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.450 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.451 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:35.451 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830448+0000", 2024-09-11T06:49:35.451 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2355999999999996e-05 2024-09-11T06:49:35.451 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.451 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:35.451 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.451 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.451 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:35.451 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:35.451 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 28.551433806999999, 2024-09-11T06:49:35.451 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013136297, 2024-09-11T06:49:35.451 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:35.452 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:35.452 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:35.452 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:35.452 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:35.452 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:35.452 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 138 2024-09-11T06:49:35.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.452 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:35.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.452 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:35.452 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:35.452 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.453 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:35.453 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:35.453 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.453 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:35.453 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:35.453 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.89e-07 2024-09-11T06:49:35.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.453 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:35.453 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818369+0000", 2024-09-11T06:49:35.453 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4170000000000001e-06 2024-09-11T06:49:35.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.454 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:35.454 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818376+0000", 2024-09-11T06:49:35.454 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4270000000000002e-06 2024-09-11T06:49:35.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.454 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:35.454 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818383+0000", 2024-09-11T06:49:35.454 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3060000000000003e-06 2024-09-11T06:49:35.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.454 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:35.454 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824783+0000", 2024-09-11T06:49:35.454 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064003020000000001 2024-09-11T06:49:35.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.454 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:35.455 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825013+0000", 2024-09-11T06:49:35.455 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023039499999999999 2024-09-11T06:49:35.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.455 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:35.455 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825484+0000", 2024-09-11T06:49:35.455 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000470562 2024-09-11T06:49:35.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.455 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:35.455 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828633+0000", 2024-09-11T06:49:35.455 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031496689999999999 2024-09-11T06:49:35.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.455 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:35.455 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831413+0000", 2024-09-11T06:49:35.456 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027793219999999999 2024-09-11T06:49:35.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.456 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:35.456 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831450+0000", 2024-09-11T06:49:35.456 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7370999999999998e-05 2024-09-11T06:49:35.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.456 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:35.456 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831501+0000", 2024-09-11T06:49:35.456 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1036999999999999e-05 2024-09-11T06:49:35.456 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.456 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:35.456 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.457 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:35.457 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:49:35.457 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 28.551409327000002, 2024-09-11T06:49:35.457 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013223011, 2024-09-11T06:49:35.457 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:35.457 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:35.457 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:35.457 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:35.457 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:35.457 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:35.457 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 139 2024-09-11T06:49:35.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.458 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:35.458 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.458 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:35.458 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:49:35.458 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.458 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.458 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:35.458 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:49:35.458 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.458 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.458 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:35.459 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818390+0000", 2024-09-11T06:49:35.459 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5600000000000001e-07 2024-09-11T06:49:35.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.459 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:35.459 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818394+0000", 2024-09-11T06:49:35.459 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5929999999999999e-06 2024-09-11T06:49:35.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.459 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:35.459 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818399+0000", 2024-09-11T06:49:35.459 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7740000000000004e-06 2024-09-11T06:49:35.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.459 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:35.460 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818412+0000", 2024-09-11T06:49:35.460 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2625e-05 2024-09-11T06:49:35.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.460 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.460 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:35.460 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826205+0000", 2024-09-11T06:49:35.460 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0077930730000000002 2024-09-11T06:49:35.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.460 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.460 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:35.460 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826281+0000", 2024-09-11T06:49:35.460 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5876000000000003e-05 2024-09-11T06:49:35.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.460 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.461 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:35.461 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826384+0000", 2024-09-11T06:49:35.461 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103838 2024-09-11T06:49:35.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.461 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.461 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:35.461 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830642+0000", 2024-09-11T06:49:35.461 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004257083 2024-09-11T06:49:35.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.461 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.461 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:35.461 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831509+0000", 2024-09-11T06:49:35.461 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086730900000000003 2024-09-11T06:49:35.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.462 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.462 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:35.462 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831547+0000", 2024-09-11T06:49:35.462 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7793000000000002e-05 2024-09-11T06:49:35.462 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.462 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.462 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:35.462 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831612+0000", 2024-09-11T06:49:35.462 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5691000000000005e-05 2024-09-11T06:49:35.462 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.462 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:35.462 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.462 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.462 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.463 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:35.463 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:35.463 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 28.551380633000001, 2024-09-11T06:49:35.463 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016098024999999998, 2024-09-11T06:49:35.463 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:35.463 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:35.463 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:35.463 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:35.463 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:35.463 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:35.463 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 140 2024-09-11T06:49:35.463 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.463 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:35.463 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.463 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:35.463 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:35.464 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.464 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.464 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.464 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:35.464 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:35.464 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.464 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.464 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.464 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:35.464 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:35.464 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5199999999999998e-07 2024-09-11T06:49:35.464 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.464 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.465 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:35.465 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818479+0000", 2024-09-11T06:49:35.465 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0368000000000002e-05 2024-09-11T06:49:35.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.465 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.465 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:35.465 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818487+0000", 2024-09-11T06:49:35.465 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2339999999999994e-06 2024-09-11T06:49:35.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.465 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.465 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:35.465 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818491+0000", 2024-09-11T06:49:35.465 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4329999999999997e-06 2024-09-11T06:49:35.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.465 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.466 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:35.466 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828080+0000", 2024-09-11T06:49:35.466 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0095890449999999992 2024-09-11T06:49:35.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.466 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.466 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:35.466 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828130+0000", 2024-09-11T06:49:35.466 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9239000000000001e-05 2024-09-11T06:49:35.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.466 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.466 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:35.466 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828208+0000", 2024-09-11T06:49:35.466 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.852e-05 2024-09-11T06:49:35.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.466 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.466 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:35.467 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831625+0000", 2024-09-11T06:49:35.467 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034169410000000002 2024-09-11T06:49:35.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.467 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.467 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:35.467 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834411+0000", 2024-09-11T06:49:35.467 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002786111 2024-09-11T06:49:35.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.467 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.467 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:35.467 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834457+0000", 2024-09-11T06:49:35.467 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6041000000000002e-05 2024-09-11T06:49:35.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.467 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.467 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:35.467 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834516+0000", 2024-09-11T06:49:35.468 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8740999999999999e-05 2024-09-11T06:49:35.468 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.468 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:35.468 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.468 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.468 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.468 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:35.468 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:49:35.468 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 28.551299566000001, 2024-09-11T06:49:35.468 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016588772000000002, 2024-09-11T06:49:35.468 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:35.468 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:35.468 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:35.469 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:35.469 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:35.469 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:35.469 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 141 2024-09-11T06:49:35.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.469 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:35.469 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.469 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:35.469 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:49:35.469 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.469 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.469 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:35.469 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:49:35.470 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.470 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.470 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:35.470 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818500+0000", 2024-09-11T06:49:35.470 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0800000000000005e-07 2024-09-11T06:49:35.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.470 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.470 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:35.470 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818505+0000", 2024-09-11T06:49:35.470 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2429999999999996e-06 2024-09-11T06:49:35.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.470 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.471 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:35.471 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818510+0000", 2024-09-11T06:49:35.471 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9339999999999997e-06 2024-09-11T06:49:35.471 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.471 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.471 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:35.471 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818514+0000", 2024-09-11T06:49:35.471 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7270000000000001e-06 2024-09-11T06:49:35.471 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.471 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.471 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:35.471 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828711+0000", 2024-09-11T06:49:35.471 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010197031 2024-09-11T06:49:35.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.472 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.472 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:35.472 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828853+0000", 2024-09-11T06:49:35.472 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014201200000000001 2024-09-11T06:49:35.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.472 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.472 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:35.472 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.829115+0000", 2024-09-11T06:49:35.472 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026257500000000003 2024-09-11T06:49:35.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.472 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.472 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:35.472 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831629+0000", 2024-09-11T06:49:35.473 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025138169999999998 2024-09-11T06:49:35.473 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.473 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.473 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:35.473 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834991+0000", 2024-09-11T06:49:35.473 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033624459999999998 2024-09-11T06:49:35.473 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.473 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.473 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:35.473 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835032+0000", 2024-09-11T06:49:35.473 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1001000000000003e-05 2024-09-11T06:49:35.473 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.473 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.473 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:35.474 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835088+0000", 2024-09-11T06:49:35.474 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5478e-05 2024-09-11T06:49:35.474 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.474 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:35.474 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.474 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.474 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:49:35.474 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:49:35.474 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 27.927664663000002, 2024-09-11T06:49:35.474 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011161912, 2024-09-11T06:49:35.474 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:35.474 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:35.474 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:35.475 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:35.475 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:35.475 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:35.475 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 158 2024-09-11T06:49:35.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.475 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:35.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.475 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:35.475 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:49:35.475 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.475 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:35.476 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:49:35.476 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.476 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:35.476 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442135+0000", 2024-09-11T06:49:35.476 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3e-07 2024-09-11T06:49:35.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.476 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:35.476 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442259+0000", 2024-09-11T06:49:35.476 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000123903 2024-09-11T06:49:35.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.477 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:35.477 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442289+0000", 2024-09-11T06:49:35.477 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0833000000000001e-05 2024-09-11T06:49:35.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.477 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:35.477 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442301+0000", 2024-09-11T06:49:35.477 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1433e-05 2024-09-11T06:49:35.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.477 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:35.477 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449796+0000", 2024-09-11T06:49:35.477 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0074954959999999999 2024-09-11T06:49:35.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.477 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:35.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449917+0000", 2024-09-11T06:49:35.478 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121033 2024-09-11T06:49:35.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.478 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:35.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.450166+0000", 2024-09-11T06:49:35.478 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00024832400000000002 2024-09-11T06:49:35.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.478 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:35.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.452630+0000", 2024-09-11T06:49:35.478 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002464188 2024-09-11T06:49:35.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.478 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:35.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453149+0000", 2024-09-11T06:49:35.479 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00051860999999999997 2024-09-11T06:49:35.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.479 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.479 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:35.479 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453234+0000", 2024-09-11T06:49:35.479 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.5865000000000005e-05 2024-09-11T06:49:35.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.479 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.479 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:35.479 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453296+0000", 2024-09-11T06:49:35.479 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1496999999999998e-05 2024-09-11T06:49:35.479 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.479 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:35.479 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.480 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:49:35.480 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:49:35.480 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 27.927478561000001, 2024-09-11T06:49:35.480 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011561900999999999, 2024-09-11T06:49:35.480 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:35.480 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:35.480 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:35.480 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:35.480 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:35.480 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:35.480 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 159 2024-09-11T06:49:35.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.480 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:35.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.481 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:35.481 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:49:35.481 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.481 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:35.481 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:49:35.481 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:35.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.481 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:35.481 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442321+0000", 2024-09-11T06:49:35.481 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8400000000000003e-07 2024-09-11T06:49:35.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.482 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:35.482 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442328+0000", 2024-09-11T06:49:35.482 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5130000000000002e-06 2024-09-11T06:49:35.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.482 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:35.482 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442341+0000", 2024-09-11T06:49:35.482 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.363e-05 2024-09-11T06:49:35.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.482 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:35.482 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442349+0000", 2024-09-11T06:49:35.482 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4760000000000001e-06 2024-09-11T06:49:35.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.483 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:35.483 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451257+0000", 2024-09-11T06:49:35.483 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089088569999999992 2024-09-11T06:49:35.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.483 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.483 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:35.483 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451389+0000", 2024-09-11T06:49:35.483 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013110899999999999 2024-09-11T06:49:35.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.483 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.483 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:35.483 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451608+0000", 2024-09-11T06:49:35.483 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000219588 2024-09-11T06:49:35.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.484 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.484 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:35.484 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453517+0000", 2024-09-11T06:49:35.484 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0019088779999999999 2024-09-11T06:49:35.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.484 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.484 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:35.484 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453727+0000", 2024-09-11T06:49:35.484 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020955100000000001 2024-09-11T06:49:35.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.484 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.484 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:35.484 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453815+0000", 2024-09-11T06:49:35.484 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8858999999999997e-05 2024-09-11T06:49:35.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:35.484 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:35.485 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:35.485 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453882+0000", 2024-09-11T06:49:35.485 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.6556000000000004e-05 2024-09-11T06:49:35.485 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.485 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:35.485 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.485 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:35.485 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:35.485 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:35.487 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-11T06:49:35.488 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:49:35.488 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:35.482+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:35.588 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:35.586+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:35.657 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:35.657 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:35.658 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:49:35.658 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:35.672 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:49:35.689 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:35.686+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:35.790 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:35.786+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:35.794 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:35.794 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:35.794 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:49:35.795 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:49:35.795 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:35.809 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:49:35.891 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:35.890+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:35.973 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:35.973 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:49:35.973 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:49:35.973 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [] 2024-09-11T06:49:35.973 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:35.988 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:49:35.992 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:35.990+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:36.092 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:36.090+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:36.113 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:36.113 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:36.113 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:49:36.114 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:36.126 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:49:36.193 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:36.190+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:36.294 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:36.290+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:36.296 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:36.296 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:36.296 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:49:36.296 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:49:36.296 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:36.311 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:49:36.360 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.5 to 1 (10000) 2024-09-11T06:49:36.385 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-11T06:49:36.385 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:49:36.427 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:36.426+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:36.437 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:36.437 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:49:36.437 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:49:36.437 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:49:36.437 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.437 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:49:36.437 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:49:36.437 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 25.208707615000002, 2024-09-11T06:49:36.437 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010553425999999999, 2024-09-11T06:49:36.438 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:36.438 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:36.438 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:36.438 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:36.438 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:49:36.438 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:49:36.438 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:49:36.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.438 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:36.438 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.438 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:36.438 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:49:36.439 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:36.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.439 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.439 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:36.439 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:49:36.439 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:36.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.439 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.439 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:36.439 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230274+0000", 2024-09-11T06:49:36.439 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.533e-06 2024-09-11T06:49:36.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.439 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.439 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:36.440 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230285+0000", 2024-09-11T06:49:36.440 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0540000000000001e-05 2024-09-11T06:49:36.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.440 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.440 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:36.440 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230309+0000", 2024-09-11T06:49:36.440 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4223000000000002e-05 2024-09-11T06:49:36.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.440 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.440 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:36.440 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230327+0000", 2024-09-11T06:49:36.440 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:49:36.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.441 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.441 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:36.441 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230478+0000", 2024-09-11T06:49:36.441 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000151459 2024-09-11T06:49:36.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.441 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.441 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:49:36.441 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230490+0000", 2024-09-11T06:49:36.441 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.147e-05 2024-09-11T06:49:36.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.441 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.441 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:36.441 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231803+0000", 2024-09-11T06:49:36.441 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001313158 2024-09-11T06:49:36.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.442 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.442 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:49:36.442 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231817+0000", 2024-09-11T06:49:36.442 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4197e-05 2024-09-11T06:49:36.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.442 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.442 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:36.442 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.235945+0000", 2024-09-11T06:49:36.442 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0041282480000000002 2024-09-11T06:49:36.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.442 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.442 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:36.443 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236312+0000", 2024-09-11T06:49:36.443 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036652700000000001 2024-09-11T06:49:36.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.443 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:49:36.443 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236689+0000", 2024-09-11T06:49:36.443 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000377117 2024-09-11T06:49:36.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.443 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:36.443 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238460+0000", 2024-09-11T06:49:36.443 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001770918 2024-09-11T06:49:36.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.443 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:36.444 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240627+0000", 2024-09-11T06:49:36.444 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021670779999999998 2024-09-11T06:49:36.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.444 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.444 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:36.444 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240751+0000", 2024-09-11T06:49:36.444 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012384900000000001 2024-09-11T06:49:36.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.444 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.444 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:36.444 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240821+0000", 2024-09-11T06:49:36.444 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0516e-05 2024-09-11T06:49:36.444 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:36.444 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:36.445 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:36.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.445 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop_reply(client.4230.0:1 2.3 e16/15 ondisk, result = 0)", 2024-09-11T06:49:36.445 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.238282+0000", 2024-09-11T06:49:36.445 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 25.20069329, 2024-09-11T06:49:36.445 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018970799999999999, 2024-09-11T06:49:36.445 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:36.445 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:36.445 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:49:36.446 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:36.446 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.446 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:36.446 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238282+0000", 2024-09-11T06:49:36.446 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:36.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.446 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.446 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:36.446 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238282+0000", 2024-09-11T06:49:36.446 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:36.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.446 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.446 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:36.447 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238284+0000", 2024-09-11T06:49:36.447 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:49:36.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.447 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:36.447 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238292+0000", 2024-09-11T06:49:36.447 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2390000000000002e-06 2024-09-11T06:49:36.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.447 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:36.447 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238314+0000", 2024-09-11T06:49:36.447 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1776999999999998e-05 2024-09-11T06:49:36.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.448 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:36.448 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238339+0000", 2024-09-11T06:49:36.448 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5119000000000001e-05 2024-09-11T06:49:36.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.448 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:36.448 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238438+0000", 2024-09-11T06:49:36.448 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9553e-05 2024-09-11T06:49:36.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.448 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:36.448 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238451+0000", 2024-09-11T06:49:36.448 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2286000000000001e-05 2024-09-11T06:49:36.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.449 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:36.449 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238472+0000", 2024-09-11T06:49:36.449 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1166999999999999e-05 2024-09-11T06:49:36.449 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:36.449 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:36.449 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:36.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.449 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:49:36.449 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.244627+0000", 2024-09-11T06:49:36.449 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 25.194348592000001, 2024-09-11T06:49:36.449 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00046141999999999998, 2024-09-11T06:49:36.450 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:36.450 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:36.450 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:49:36.450 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:36.450 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:49:36.450 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:49:36.450 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 2 2024-09-11T06:49:36.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.450 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:36.450 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.450 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:36.450 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244627+0000", 2024-09-11T06:49:36.450 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:36.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.451 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.451 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:36.451 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244627+0000", 2024-09-11T06:49:36.451 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:36.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.451 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.451 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:36.451 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244629+0000", 2024-09-11T06:49:36.451 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.95e-06 2024-09-11T06:49:36.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.451 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.451 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:36.451 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244640+0000", 2024-09-11T06:49:36.451 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1494e-05 2024-09-11T06:49:36.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.452 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:36.452 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244664+0000", 2024-09-11T06:49:36.452 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4006e-05 2024-09-11T06:49:36.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.452 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:36.452 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244684+0000", 2024-09-11T06:49:36.452 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.961e-05 2024-09-11T06:49:36.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.452 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:36.453 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244862+0000", 2024-09-11T06:49:36.453 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017863500000000001 2024-09-11T06:49:36.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.453 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:36.453 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245088+0000", 2024-09-11T06:49:36.453 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000225725 2024-09-11T06:49:36.453 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:36.453 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:36.453 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:36.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.454 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:49:36.454 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.245492+0000", 2024-09-11T06:49:36.454 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 25.193483649000001, 2024-09-11T06:49:36.454 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00042092600000000001, 2024-09-11T06:49:36.454 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:36.454 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:36.454 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:49:36.454 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:36.454 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:49:36.454 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:49:36.454 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 3 2024-09-11T06:49:36.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.455 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:36.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.455 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:36.455 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245492+0000", 2024-09-11T06:49:36.455 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:36.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.455 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:36.455 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245492+0000", 2024-09-11T06:49:36.455 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:36.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.456 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:36.456 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245493+0000", 2024-09-11T06:49:36.456 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.657e-06 2024-09-11T06:49:36.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.456 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:36.456 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245501+0000", 2024-09-11T06:49:36.456 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2490000000000004e-06 2024-09-11T06:49:36.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.456 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:36.457 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245523+0000", 2024-09-11T06:49:36.457 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2016999999999999e-05 2024-09-11T06:49:36.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.457 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.457 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:36.457 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245548+0000", 2024-09-11T06:49:36.457 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5060000000000001e-05 2024-09-11T06:49:36.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.457 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.458 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:36.458 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245720+0000", 2024-09-11T06:49:36.458 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017190200000000001 2024-09-11T06:49:36.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.458 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.458 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:36.458 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245913+0000", 2024-09-11T06:49:36.458 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193041 2024-09-11T06:49:36.458 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:36.458 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:36.458 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:36.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.458 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.458 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-11T06:49:36.459 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:49:36.459 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 24.206536772, 2024-09-11T06:49:36.459 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069904770000000001, 2024-09-11T06:49:36.459 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:36.459 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:36.459 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:36.459 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:36.459 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:49:36.459 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:49:36.459 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 5 2024-09-11T06:49:36.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.459 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:36.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.459 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:36.460 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:49:36.460 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:36.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.460 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.460 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:36.460 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:49:36.460 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:36.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.460 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.460 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:36.460 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232442+0000", 2024-09-11T06:49:36.460 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.044e-06 2024-09-11T06:49:36.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.461 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.461 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:36.461 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232460+0000", 2024-09-11T06:49:36.461 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7863000000000002e-05 2024-09-11T06:49:36.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.461 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.461 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:36.461 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232492+0000", 2024-09-11T06:49:36.461 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2812999999999999e-05 2024-09-11T06:49:36.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.461 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.461 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:36.461 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232510+0000", 2024-09-11T06:49:36.461 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7552999999999999e-05 2024-09-11T06:49:36.462 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.462 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.462 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:36.462 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232640+0000", 2024-09-11T06:49:36.462 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013029900000000001 2024-09-11T06:49:36.462 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.462 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.462 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:49:36.462 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232650+0000", 2024-09-11T06:49:36.462 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0023999999999999e-05 2024-09-11T06:49:36.462 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.462 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.462 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:36.463 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235641+0000", 2024-09-11T06:49:36.463 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00299089 2024-09-11T06:49:36.463 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.463 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.463 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:36.463 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235980+0000", 2024-09-11T06:49:36.463 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000338907 2024-09-11T06:49:36.463 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.463 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.463 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:49:36.463 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.236351+0000", 2024-09-11T06:49:36.463 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037087699999999998 2024-09-11T06:49:36.463 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.463 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.464 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:36.464 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238074+0000", 2024-09-11T06:49:36.464 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001722859 2024-09-11T06:49:36.464 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.464 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.464 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:36.464 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239160+0000", 2024-09-11T06:49:36.464 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001086074 2024-09-11T06:49:36.464 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.464 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.465 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:36.465 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239299+0000", 2024-09-11T06:49:36.465 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013933500000000001 2024-09-11T06:49:36.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.465 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.465 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:36.465 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239429+0000", 2024-09-11T06:49:36.465 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129939 2024-09-11T06:49:36.465 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:36.465 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:36.465 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:36.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.466 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.466 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop_reply(client.4230.0:5 2.2 e16/15 ondisk, result = 0)", 2024-09-11T06:49:36.466 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:12.238911+0000", 2024-09-11T06:49:36.466 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 24.200064483999999, 2024-09-11T06:49:36.466 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00053624900000000004, 2024-09-11T06:49:36.466 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:36.466 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:36.466 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:49:36.466 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:36.466 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.466 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:36.466 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238911+0000", 2024-09-11T06:49:36.466 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:36.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.467 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.467 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:36.467 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238911+0000", 2024-09-11T06:49:36.467 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:36.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.467 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.467 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:36.467 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238913+0000", 2024-09-11T06:49:36.467 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2069999999999998e-06 2024-09-11T06:49:36.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.467 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.467 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:36.468 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238923+0000", 2024-09-11T06:49:36.468 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0360000000000001e-05 2024-09-11T06:49:36.468 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.468 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.468 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:36.468 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238953+0000", 2024-09-11T06:49:36.468 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9258999999999999e-05 2024-09-11T06:49:36.468 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.468 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.468 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:36.468 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238967+0000", 2024-09-11T06:49:36.468 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3797e-05 2024-09-11T06:49:36.468 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.469 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.469 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:36.469 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239117+0000", 2024-09-11T06:49:36.469 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000150765 2024-09-11T06:49:36.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.469 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.469 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:36.469 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239143+0000", 2024-09-11T06:49:36.469 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5514e-05 2024-09-11T06:49:36.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.469 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.469 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:36.469 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239447+0000", 2024-09-11T06:49:36.470 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00030434700000000001 2024-09-11T06:49:36.470 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:36.470 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:36.470 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:36.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.470 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.470 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-11T06:49:36.470 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:49:36.470 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 23.192448819999999, 2024-09-11T06:49:36.470 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069253250000000004, 2024-09-11T06:49:36.470 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:36.470 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:36.470 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:36.470 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:36.471 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:49:36.471 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:49:36.471 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 7 2024-09-11T06:49:36.471 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.471 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:36.471 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.471 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:36.471 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:49:36.471 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:36.471 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.471 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.471 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:36.471 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:49:36.472 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:36.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.472 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.472 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:36.472 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246530+0000", 2024-09-11T06:49:36.472 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9469999999999999e-06 2024-09-11T06:49:36.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.472 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.472 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:36.472 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246545+0000", 2024-09-11T06:49:36.472 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5973000000000001e-05 2024-09-11T06:49:36.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.472 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.472 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:36.473 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246578+0000", 2024-09-11T06:49:36.473 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2759999999999998e-05 2024-09-11T06:49:36.473 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.473 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.473 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:36.473 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246595+0000", 2024-09-11T06:49:36.473 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6939000000000002e-05 2024-09-11T06:49:36.473 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.473 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.473 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:36.473 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246770+0000", 2024-09-11T06:49:36.473 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000174723 2024-09-11T06:49:36.473 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.473 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.473 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:49:36.474 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246789+0000", 2024-09-11T06:49:36.474 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9159000000000001e-05 2024-09-11T06:49:36.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.474 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.474 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:36.474 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250337+0000", 2024-09-11T06:49:36.474 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035480759999999998 2024-09-11T06:49:36.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.474 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.474 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:36.474 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250549+0000", 2024-09-11T06:49:36.474 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021192199999999999 2024-09-11T06:49:36.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.475 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:49:36.475 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250750+0000", 2024-09-11T06:49:36.475 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020104199999999999 2024-09-11T06:49:36.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.475 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:36.475 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.252455+0000", 2024-09-11T06:49:36.475 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001705249 2024-09-11T06:49:36.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.475 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:36.475 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253305+0000", 2024-09-11T06:49:36.475 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00084941200000000002 2024-09-11T06:49:36.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.476 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:36.476 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253388+0000", 2024-09-11T06:49:36.476 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3642999999999999e-05 2024-09-11T06:49:36.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.476 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:36.476 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253452+0000", 2024-09-11T06:49:36.476 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3479999999999995e-05 2024-09-11T06:49:36.476 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:36.476 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:36.476 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:36.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.477 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop_reply(client.4230.0:7 2.2 e18/15 ondisk, result = 0)", 2024-09-11T06:49:36.477 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:13.253128+0000", 2024-09-11T06:49:36.477 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 23.185847246000002, 2024-09-11T06:49:36.477 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034023700000000002, 2024-09-11T06:49:36.477 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:36.477 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:36.477 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:49:36.477 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:36.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.477 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:36.477 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253128+0000", 2024-09-11T06:49:36.477 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:36.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.478 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:36.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253128+0000", 2024-09-11T06:49:36.478 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:36.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.478 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:36.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253135+0000", 2024-09-11T06:49:36.478 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.657e-06 2024-09-11T06:49:36.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.478 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:36.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253144+0000", 2024-09-11T06:49:36.478 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0869999999999996e-06 2024-09-11T06:49:36.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.479 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.479 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:36.479 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253167+0000", 2024-09-11T06:49:36.479 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3252e-05 2024-09-11T06:49:36.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.479 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.479 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:36.479 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253186+0000", 2024-09-11T06:49:36.479 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9239999999999999e-05 2024-09-11T06:49:36.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.479 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.479 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:36.480 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253282+0000", 2024-09-11T06:49:36.480 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.6093000000000003e-05 2024-09-11T06:49:36.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.480 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:36.480 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253294+0000", 2024-09-11T06:49:36.480 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1986e-05 2024-09-11T06:49:36.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:36.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:36.480 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:36.480 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253468+0000", 2024-09-11T06:49:36.480 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017392199999999999 2024-09-11T06:49:36.480 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:36.481 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:36.481 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:36.481 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:36.481 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:36.481 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:36.482 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:49:36.527 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:36.526+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:36.596 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:36.594+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:36.600 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:36.601 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:36.601 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:49:36.601 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:36.615 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:49:36.703 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:36.702+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:36.724 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:49:36.725 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:49:36.743 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2024-09-11T06:49:36.362862+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":137660,"kb_used_data":2512,"kb_used_omap":7,"kb_used_meta":135032,"kb_avail":471721540,"statfs":{"total":483183820800,"available":483042856960,"internally_reserved":0,"allocated":2572288,"data_stored":1657570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7971,"internal_metadata":138272989},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.020284"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T06:49:33.346727+0000","last_change":"2024-09-11T06:49:13.393788+0000","last_active":"2024-09-11T06:49:33.346727+0000","last_peered":"2024-09-11T06:49:33.346727+0000","last_clean":"2024-09-11T06:49:33.346727+0000","last_became_active":"2024-09-11T06:49:11.433437+0000","last_became_peered":"2024-09-11T06:49:11.433437+0000","last_unstale":"2024-09-11T06:49:33.346727+0000","last_undegraded":"2024-09-11T06:49:33.346727+0000","last_fullsized":"2024-09-11T06:49:33.346727+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:20:04.339208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001403348,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T06:49:33.346640+0000","last_change":"2024-09-11T06:49:21.882377+0000","last_active":"2024-09-11T06:49:33.346640+0000","last_peered":"2024-09-11T06:49:33.346640+0000","last_clean":"2024-09-11T06:49:33.346640+0000","last_became_active":"2024-09-11T06:49:21.881858+0000","last_became_peered":"2024-09-11T06:49:21.881858+0000","last_unstale":"2024-09-11T06:49:33.346640+0000","last_undegraded":"2024-09-11T06:49:33.346640+0000","last_fullsized":"2024-09-11T06:49:33.346640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:33:42.789131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T06:49:33.346605+0000","last_change":"2024-09-11T06:49:13.393298+0000","last_active":"2024-09-11T06:49:33.346605+0000","last_peered":"2024-09-11T06:49:33.346605+0000","last_clean":"2024-09-11T06:49:33.346605+0000","last_became_active":"2024-09-11T06:49:11.315089+0000","last_became_peered":"2024-09-11T06:49:11.315089+0000","last_unstale":"2024-09-11T06:49:33.346605+0000","last_undegraded":"2024-09-11T06:49:33.346605+0000","last_fullsized":"2024-09-11T06:49:33.346605+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:31.769443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095172900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T06:49:33.916273+0000","last_change":"2024-09-11T06:49:28.017449+0000","last_active":"2024-09-11T06:49:33.916273+0000","last_peered":"2024-09-11T06:49:33.916273+0000","last_clean":"2024-09-11T06:49:33.916273+0000","last_became_active":"2024-09-11T06:49:28.016860+0000","last_became_peered":"2024-09-11T06:49:28.016860+0000","last_unstale":"2024-09-11T06:49:33.916273+0000","last_undegraded":"2024-09-11T06:49:33.916273+0000","last_fullsized":"2024-09-11T06:49:33.916273+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:32:34.900467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T06:49:33.907207+0000","last_change":"2024-09-11T06:49:21.283656+0000","last_active":"2024-09-11T06:49:33.907207+0000","last_peered":"2024-09-11T06:49:33.907207+0000","last_clean":"2024-09-11T06:49:33.907207+0000","last_became_active":"2024-09-11T06:49:21.278679+0000","last_became_peered":"2024-09-11T06:49:21.278679+0000","last_unstale":"2024-09-11T06:49:33.907207+0000","last_undegraded":"2024-09-11T06:49:33.907207+0000","last_fullsized":"2024-09-11T06:49:33.907207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:20:11.653333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001910244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T06:49:33.346454+0000","last_change":"2024-09-11T06:49:21.274656+0000","last_active":"2024-09-11T06:49:33.346454+0000","last_peered":"2024-09-11T06:49:33.346454+0000","last_clean":"2024-09-11T06:49:33.346454+0000","last_became_active":"2024-09-11T06:49:21.273719+0000","last_became_peered":"2024-09-11T06:49:21.273719+0000","last_unstale":"2024-09-11T06:49:33.346454+0000","last_undegraded":"2024-09-11T06:49:33.346454+0000","last_fullsized":"2024-09-11T06:49:33.346454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:59.900889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010163780000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T06:49:33.346704+0000","last_change":"2024-09-11T06:49:21.274276+0000","last_active":"2024-09-11T06:49:33.346704+0000","last_peered":"2024-09-11T06:49:33.346704+0000","last_clean":"2024-09-11T06:49:33.346704+0000","last_became_active":"2024-09-11T06:49:21.273899+0000","last_became_peered":"2024-09-11T06:49:21.273899+0000","last_unstale":"2024-09-11T06:49:33.346704+0000","last_undegraded":"2024-09-11T06:49:33.346704+0000","last_fullsized":"2024-09-11T06:49:33.346704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:37:31.862801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077615599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T06:49:33.907224+0000","last_change":"2024-09-11T06:49:13.251115+0000","last_active":"2024-09-11T06:49:33.907224+0000","last_peered":"2024-09-11T06:49:33.907224+0000","last_clean":"2024-09-11T06:49:33.907224+0000","last_became_active":"2024-09-11T06:49:11.235231+0000","last_became_peered":"2024-09-11T06:49:11.235231+0000","last_unstale":"2024-09-11T06:49:33.907224+0000","last_undegraded":"2024-09-11T06:49:33.907224+0000","last_fullsized":"2024-09-11T06:49:33.907224+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:01:21.028466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011212469999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":188,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T06:49:33.346694+0000","last_change":"2024-09-11T06:49:06.112585+0000","last_active":"2024-09-11T06:49:33.346694+0000","last_peered":"2024-09-11T06:49:33.346694+0000","last_clean":"2024-09-11T06:49:33.346694+0000","last_became_active":"2024-09-11T06:49:06.110690+0000","last_became_peered":"2024-09-11T06:49:06.110690+0000","last_unstale":"2024-09-11T06:49:33.346694+0000","last_undegraded":"2024-09-11T06:49:33.346694+0000","last_fullsized":"2024-09-11T06:49:33.346694+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_clean_scrub_stamp":"2024-09-11T06:49:05.088839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:51.167301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640264,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":93547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640265,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":93528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640265,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":876544,"data_stored":688474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640266,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":98106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":47244640266,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640267,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":683915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:49:36.744 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:49:36.778 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:36.778 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:36.778 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:49:36.778 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:49:36.778 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:36.792 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:49:36.839 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:36.838+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:36.899 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:36.898+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:36.957 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:36.957 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:49:36.957 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:49:36.957 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:49:36.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.957 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:54 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'32, mlcod=13'29)", 2024-09-11T06:49:36.957 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:49:36.957 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.745907965000001, 2024-09-11T06:49:36.957 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034271670000000001, 2024-09-11T06:49:36.957 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:36.957 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:36.957 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:36.957 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:49:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:49:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210361+0000", 2024-09-11T06:49:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1260000000000001e-06 2024-09-11T06:49:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210370+0000", 2024-09-11T06:49:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8200000000000003e-06 2024-09-11T06:49:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210390+0000", 2024-09-11T06:49:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0126e-05 2024-09-11T06:49:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210404+0000", 2024-09-11T06:49:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4589999999999999e-05 2024-09-11T06:49:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.960 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:36.960 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210538+0000", 2024-09-11T06:49:36.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000133719 2024-09-11T06:49:36.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.960 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:36.960 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210593+0000", 2024-09-11T06:49:36.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5016999999999999e-05 2024-09-11T06:49:36.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.960 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:36.960 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213695+0000", 2024-09-11T06:49:36.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003101866 2024-09-11T06:49:36.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.961 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.961 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:36.961 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213787+0000", 2024-09-11T06:49:36.961 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1903000000000004e-05 2024-09-11T06:49:36.961 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:36.961 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:36.961 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:36.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.961 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.961 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:55 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'33, mlcod=13'29)", 2024-09-11T06:49:36.961 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:49:36.961 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.744888726999999, 2024-09-11T06:49:36.961 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.005086091, 2024-09-11T06:49:36.961 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:36.962 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:36.962 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:36.962 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:36.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:36.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:49:36.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:36.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:36.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:49:36.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:36.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211380+0000", 2024-09-11T06:49:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.097e-06 2024-09-11T06:49:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211537+0000", 2024-09-11T06:49:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000157369 2024-09-11T06:49:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211559+0000", 2024-09-11T06:49:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000001e-05 2024-09-11T06:49:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.964 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:36.964 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211573+0000", 2024-09-11T06:49:36.964 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4946000000000001e-05 2024-09-11T06:49:36.964 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.964 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.964 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:36.964 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211676+0000", 2024-09-11T06:49:36.964 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0001026 2024-09-11T06:49:36.964 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.964 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.964 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:36.964 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211709+0000", 2024-09-11T06:49:36.964 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3071999999999998e-05 2024-09-11T06:49:36.964 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.964 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216379+0000", 2024-09-11T06:49:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0046693780000000001 2024-09-11T06:49:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216465+0000", 2024-09-11T06:49:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6469000000000002e-05 2024-09-11T06:49:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:56 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'34, mlcod=13'29)", 2024-09-11T06:49:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:49:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.743454868000001, 2024-09-11T06:49:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0060310629999999997, 2024-09-11T06:49:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:49:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:49:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.967 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:36.967 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212814+0000", 2024-09-11T06:49:36.967 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0869999999999999e-06 2024-09-11T06:49:36.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.967 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:36.967 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212822+0000", 2024-09-11T06:49:36.967 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0870000000000007e-06 2024-09-11T06:49:36.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.967 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:36.967 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212840+0000", 2024-09-11T06:49:36.967 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:49:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212854+0000", 2024-09-11T06:49:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4003e-05 2024-09-11T06:49:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213026+0000", 2024-09-11T06:49:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017207499999999999 2024-09-11T06:49:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213063+0000", 2024-09-11T06:49:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7187000000000002e-05 2024-09-11T06:49:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218762+0000", 2024-09-11T06:49:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0056993290000000004 2024-09-11T06:49:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218844+0000", 2024-09-11T06:49:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1701999999999998e-05 2024-09-11T06:49:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.970 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:60 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'38, mlcod=13'36)", 2024-09-11T06:49:36.970 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:49:36.970 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.729513519000001, 2024-09-11T06:49:36.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039254270000000004, 2024-09-11T06:49:36.970 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:36.970 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:36.970 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:36.970 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:36.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.970 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:36.970 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:49:36.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:36.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:49:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226756+0000", 2024-09-11T06:49:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-11T06:49:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226768+0000", 2024-09-11T06:49:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1919999999999999e-05 2024-09-11T06:49:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226797+0000", 2024-09-11T06:49:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9573e-05 2024-09-11T06:49:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226818+0000", 2024-09-11T06:49:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0432999999999999e-05 2024-09-11T06:49:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226909+0000", 2024-09-11T06:49:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0920000000000004e-05 2024-09-11T06:49:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226936+0000", 2024-09-11T06:49:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7472999999999999e-05 2024-09-11T06:49:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230596+0000", 2024-09-11T06:49:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0036602520000000001 2024-09-11T06:49:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230680+0000", 2024-09-11T06:49:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3106e-05 2024-09-11T06:49:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:107 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'69, mlcod=13'64)", 2024-09-11T06:49:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:49:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.513066062, 2024-09-11T06:49:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040728830000000002, 2024-09-11T06:49:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:49:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:49:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443203+0000", 2024-09-11T06:49:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1480000000000001e-06 2024-09-11T06:49:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443222+0000", 2024-09-11T06:49:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9040000000000001e-05 2024-09-11T06:49:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443246+0000", 2024-09-11T06:49:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4606000000000001e-05 2024-09-11T06:49:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443277+0000", 2024-09-11T06:49:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0207999999999999e-05 2024-09-11T06:49:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443340+0000", 2024-09-11T06:49:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3602000000000005e-05 2024-09-11T06:49:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443363+0000", 2024-09-11T06:49:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2331e-05 2024-09-11T06:49:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447216+0000", 2024-09-11T06:49:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0038539690000000001 2024-09-11T06:49:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447274+0000", 2024-09-11T06:49:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7979000000000002e-05 2024-09-11T06:49:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:111 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'73, mlcod=13'67)", 2024-09-11T06:49:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:49:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.506554432000001, 2024-09-11T06:49:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034372360000000002, 2024-09-11T06:49:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:49:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:49:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449714+0000", 2024-09-11T06:49:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5000000000000001e-07 2024-09-11T06:49:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449721+0000", 2024-09-11T06:49:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8530000000000003e-06 2024-09-11T06:49:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449736+0000", 2024-09-11T06:49:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5114e-05 2024-09-11T06:49:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449743+0000", 2024-09-11T06:49:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0210000000000002e-06 2024-09-11T06:49:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449804+0000", 2024-09-11T06:49:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0479999999999997e-05 2024-09-11T06:49:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449826+0000", 2024-09-11T06:49:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1922000000000001e-05 2024-09-11T06:49:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453104+0000", 2024-09-11T06:49:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003278652 2024-09-11T06:49:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453150+0000", 2024-09-11T06:49:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6244e-05 2024-09-11T06:49:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:114 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'76, mlcod=13'71)", 2024-09-11T06:49:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:49:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.501876361000001, 2024-09-11T06:49:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0042576660000000002, 2024-09-11T06:49:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:49:36.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:36.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:36.983 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:49:36.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:36.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:36.983 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454392+0000", 2024-09-11T06:49:36.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5999999999999991e-07 2024-09-11T06:49:36.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454399+0000", 2024-09-11T06:49:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.878e-06 2024-09-11T06:49:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454415+0000", 2024-09-11T06:49:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5659999999999999e-05 2024-09-11T06:49:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454422+0000", 2024-09-11T06:49:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:49:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:36.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454474+0000", 2024-09-11T06:49:36.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1860999999999997e-05 2024-09-11T06:49:36.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:36.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454497+0000", 2024-09-11T06:49:36.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3686e-05 2024-09-11T06:49:36.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:36.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458552+0000", 2024-09-11T06:49:36.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040544550000000002 2024-09-11T06:49:36.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458649+0000", 2024-09-11T06:49:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7005999999999999e-05 2024-09-11T06:49:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:115 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'77, mlcod=13'72)", 2024-09-11T06:49:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.500535749000001, 2024-09-11T06:49:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003355584, 2024-09-11T06:49:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:36.987 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:36.988 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:36.988 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.988 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:36.988 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:36.988 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:36.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.988 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.988 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:36.988 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:36.988 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:36.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.988 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.988 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:36.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:36.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5999999999999999e-07 2024-09-11T06:49:36.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:36.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455803+0000", 2024-09-11T06:49:36.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0310999999999998e-05 2024-09-11T06:49:36.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:36.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455815+0000", 2024-09-11T06:49:36.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2787e-05 2024-09-11T06:49:36.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:36.990 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455823+0000", 2024-09-11T06:49:36.990 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5959999999999998e-06 2024-09-11T06:49:36.990 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.990 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:36.990 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455875+0000", 2024-09-11T06:49:36.990 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2054e-05 2024-09-11T06:49:36.990 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.990 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:36.990 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455896+0000", 2024-09-11T06:49:36.990 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0554999999999999e-05 2024-09-11T06:49:36.990 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.990 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:36.990 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459000+0000", 2024-09-11T06:49:36.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003104842 2024-09-11T06:49:36.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.991 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:36.991 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459087+0000", 2024-09-11T06:49:36.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6978999999999995e-05 2024-09-11T06:49:36.991 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:36.991 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:36.991 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:36.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.991 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:120 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'82, mlcod=13'78)", 2024-09-11T06:49:36.991 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:49:36.991 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.489858464000001, 2024-09-11T06:49:36.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0064612549999999999, 2024-09-11T06:49:36.992 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:36.992 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:36.992 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:36.992 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:36.992 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.992 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:36.992 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:49:36.992 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:36.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.992 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.992 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:36.992 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:49:36.992 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:36.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.993 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:36.993 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466410+0000", 2024-09-11T06:49:36.993 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0389999999999999e-06 2024-09-11T06:49:36.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.993 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:36.993 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466496+0000", 2024-09-11T06:49:36.993 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5519000000000006e-05 2024-09-11T06:49:36.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.993 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:36.993 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466510+0000", 2024-09-11T06:49:36.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3842999999999999e-05 2024-09-11T06:49:36.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.994 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:36.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466517+0000", 2024-09-11T06:49:36.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5229999999999998e-06 2024-09-11T06:49:36.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.994 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:36.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466595+0000", 2024-09-11T06:49:36.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8019e-05 2024-09-11T06:49:36.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.994 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:36.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466612+0000", 2024-09-11T06:49:36.995 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7013000000000001e-05 2024-09-11T06:49:36.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.995 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:36.995 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472787+0000", 2024-09-11T06:49:36.995 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0061750269999999996 2024-09-11T06:49:36.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.995 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:36.995 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472870+0000", 2024-09-11T06:49:36.995 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3271999999999998e-05 2024-09-11T06:49:36.995 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:36.995 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:36.995 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:36.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.996 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:121 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'83, mlcod=13'78)", 2024-09-11T06:49:36.996 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:49:36.996 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.488633667999999, 2024-09-11T06:49:36.996 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055613069999999997, 2024-09-11T06:49:36.996 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:36.996 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:36.996 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:36.996 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:36.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:36.996 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:49:36.996 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:36.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.997 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:36.997 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:49:36.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:36.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.997 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.997 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:36.997 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467635+0000", 2024-09-11T06:49:36.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6899999999999996e-07 2024-09-11T06:49:36.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.997 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.997 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:36.997 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467727+0000", 2024-09-11T06:49:36.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1800999999999997e-05 2024-09-11T06:49:36.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:36.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467740+0000", 2024-09-11T06:49:36.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3747999999999999e-05 2024-09-11T06:49:36.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:36.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467753+0000", 2024-09-11T06:49:36.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2807e-05 2024-09-11T06:49:36.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:36.999 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467812+0000", 2024-09-11T06:49:36.999 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8381999999999998e-05 2024-09-11T06:49:36.999 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.999 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:36.999 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467828+0000", 2024-09-11T06:49:36.999 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6728e-05 2024-09-11T06:49:36.999 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.999 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:36.999 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473112+0000", 2024-09-11T06:49:36.999 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052833619999999998 2024-09-11T06:49:36.999 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:36.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:36.999 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:37.000 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473195+0000", 2024-09-11T06:49:37.000 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3609999999999994e-05 2024-09-11T06:49:37.000 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:37.000 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:37.000 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:37.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.000 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:131 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'89, mlcod=13'85)", 2024-09-11T06:49:37.000 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:49:37.000 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.137227953, 2024-09-11T06:49:37.000 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052492260000000001, 2024-09-11T06:49:37.000 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:37.000 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:37.000 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:37.001 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:37.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.001 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:37.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:49:37.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:37.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.001 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:37.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:49:37.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:37.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.001 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:37.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819041+0000", 2024-09-11T06:49:37.002 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4360000000000001e-06 2024-09-11T06:49:37.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:37.002 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819100+0000", 2024-09-11T06:49:37.002 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9225999999999998e-05 2024-09-11T06:49:37.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:37.002 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819128+0000", 2024-09-11T06:49:37.002 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.779e-05 2024-09-11T06:49:37.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:37.002 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819142+0000", 2024-09-11T06:49:37.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3573e-05 2024-09-11T06:49:37.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:37.003 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819263+0000", 2024-09-11T06:49:37.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000121086 2024-09-11T06:49:37.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:37.003 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819292+0000", 2024-09-11T06:49:37.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9363e-05 2024-09-11T06:49:37.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:37.004 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824185+0000", 2024-09-11T06:49:37.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0048924620000000002 2024-09-11T06:49:37.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.004 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:37.004 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824289+0000", 2024-09-11T06:49:37.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010429000000000001 2024-09-11T06:49:37.004 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:37.004 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:37.004 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:37.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.004 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:132 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'90, mlcod=13'85)", 2024-09-11T06:49:37.005 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:49:37.005 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.136453263, 2024-09-11T06:49:37.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.004532342, 2024-09-11T06:49:37.005 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:37.005 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:37.005 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:37.005 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:37.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:37.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:49:37.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:37.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.006 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:37.006 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:49:37.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:37.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.006 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:37.006 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819816+0000", 2024-09-11T06:49:37.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.61e-06 2024-09-11T06:49:37.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.006 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:37.006 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819827+0000", 2024-09-11T06:49:37.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1e-05 2024-09-11T06:49:37.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.007 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:37.007 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819851+0000", 2024-09-11T06:49:37.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4267e-05 2024-09-11T06:49:37.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.007 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:37.007 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819864+0000", 2024-09-11T06:49:37.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2767e-05 2024-09-11T06:49:37.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.007 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:37.007 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820071+0000", 2024-09-11T06:49:37.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000207061 2024-09-11T06:49:37.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.008 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:37.008 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820103+0000", 2024-09-11T06:49:37.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2326999999999997e-05 2024-09-11T06:49:37.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.008 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:37.008 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824302+0000", 2024-09-11T06:49:37.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0041986000000000002 2024-09-11T06:49:37.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.008 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:37.009 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824347+0000", 2024-09-11T06:49:37.009 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4709999999999997e-05 2024-09-11T06:49:37.009 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:37.009 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:37.009 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:37.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.009 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:133 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'91, mlcod=13'86)", 2024-09-11T06:49:37.009 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:49:37.009 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.135821604, 2024-09-11T06:49:37.009 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039421459999999997, 2024-09-11T06:49:37.009 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:37.009 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:37.010 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:37.010 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:37.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:37.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:49:37.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:37.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:37.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:49:37.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:37.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:37.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820448+0000", 2024-09-11T06:49:37.011 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.84e-06 2024-09-11T06:49:37.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.011 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:37.011 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820463+0000", 2024-09-11T06:49:37.011 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5356e-05 2024-09-11T06:49:37.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.011 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:37.011 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820499+0000", 2024-09-11T06:49:37.011 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.625e-05 2024-09-11T06:49:37.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.011 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:37.011 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820546+0000", 2024-09-11T06:49:37.012 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7017000000000001e-05 2024-09-11T06:49:37.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.012 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.012 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:37.012 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820882+0000", 2024-09-11T06:49:37.012 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033512 2024-09-11T06:49:37.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.012 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.012 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:37.012 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820941+0000", 2024-09-11T06:49:37.012 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8922999999999998e-05 2024-09-11T06:49:37.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.012 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.012 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:37.012 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824351+0000", 2024-09-11T06:49:37.013 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034101940000000001 2024-09-11T06:49:37.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.013 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:37.013 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824388+0000", 2024-09-11T06:49:37.013 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7446e-05 2024-09-11T06:49:37.013 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:37.013 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:37.013 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:37.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.013 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:134 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'92, mlcod=13'86)", 2024-09-11T06:49:37.013 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:49:37.013 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.135162485999999, 2024-09-11T06:49:37.014 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033248179999999998, 2024-09-11T06:49:37.014 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:37.014 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:37.014 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:37.014 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:37.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.014 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:37.014 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:49:37.014 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:37.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.014 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:37.014 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:49:37.014 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:37.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.015 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:37.015 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821107+0000", 2024-09-11T06:49:37.015 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7430000000000001e-06 2024-09-11T06:49:37.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.015 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:37.015 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821205+0000", 2024-09-11T06:49:37.015 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7666999999999998e-05 2024-09-11T06:49:37.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.015 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:37.015 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821234+0000", 2024-09-11T06:49:37.015 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9756000000000001e-05 2024-09-11T06:49:37.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.016 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:37.016 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821254+0000", 2024-09-11T06:49:37.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9446e-05 2024-09-11T06:49:37.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.016 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.016 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:37.016 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821846+0000", 2024-09-11T06:49:37.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00059187599999999997 2024-09-11T06:49:37.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.016 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.016 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:37.016 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821894+0000", 2024-09-11T06:49:37.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8766e-05 2024-09-11T06:49:37.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.016 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.017 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:37.017 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824392+0000", 2024-09-11T06:49:37.017 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0024977509999999999 2024-09-11T06:49:37.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.017 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:37.017 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824430+0000", 2024-09-11T06:49:37.017 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7812999999999999e-05 2024-09-11T06:49:37.017 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:37.017 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:37.017 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:37.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.017 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:137 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'95, mlcod=13'86)", 2024-09-11T06:49:37.017 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:49:37.017 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.132700345, 2024-09-11T06:49:37.018 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003084299, 2024-09-11T06:49:37.018 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:37.018 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:37.018 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:37.018 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:37.018 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.018 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:37.018 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:49:37.018 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:37.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.018 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.018 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:37.018 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:49:37.018 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:37.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:37.019 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823568+0000", 2024-09-11T06:49:37.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1030000000000001e-06 2024-09-11T06:49:37.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:37.019 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823662+0000", 2024-09-11T06:49:37.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3529999999999994e-05 2024-09-11T06:49:37.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:37.020 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823686+0000", 2024-09-11T06:49:37.020 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4358999999999998e-05 2024-09-11T06:49:37.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.020 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:37.020 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823701+0000", 2024-09-11T06:49:37.020 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4847e-05 2024-09-11T06:49:37.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.020 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:37.020 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824518+0000", 2024-09-11T06:49:37.020 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00081716700000000005 2024-09-11T06:49:37.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.020 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:37.021 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824546+0000", 2024-09-11T06:49:37.021 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7860000000000001e-05 2024-09-11T06:49:37.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.021 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.021 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:37.021 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826558+0000", 2024-09-11T06:49:37.021 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0020120009999999998 2024-09-11T06:49:37.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.021 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.021 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:37.021 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826652+0000", 2024-09-11T06:49:37.021 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3431999999999996e-05 2024-09-11T06:49:37.021 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:37.022 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:37.022 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:37.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.022 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:139 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'97, mlcod=13'86)", 2024-09-11T06:49:37.022 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:49:37.022 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.130411626000001, 2024-09-11T06:49:37.022 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003732221, 2024-09-11T06:49:37.022 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:37.022 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:37.022 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:37.022 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:37.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.022 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:37.023 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:49:37.023 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:37.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.023 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.023 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:37.023 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:49:37.023 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:37.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.023 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.023 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:37.023 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825858+0000", 2024-09-11T06:49:37.023 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5829999999999999e-06 2024-09-11T06:49:37.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.023 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.023 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:37.024 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825874+0000", 2024-09-11T06:49:37.024 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5967000000000001e-05 2024-09-11T06:49:37.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.024 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.024 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:37.024 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825909+0000", 2024-09-11T06:49:37.024 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5692999999999999e-05 2024-09-11T06:49:37.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.024 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.024 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:37.024 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825923+0000", 2024-09-11T06:49:37.024 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4080000000000001e-05 2024-09-11T06:49:37.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.024 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.024 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:37.025 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826010+0000", 2024-09-11T06:49:37.025 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6248999999999994e-05 2024-09-11T06:49:37.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.025 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.025 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:37.025 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826038+0000", 2024-09-11T06:49:37.025 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8010000000000001e-05 2024-09-11T06:49:37.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.025 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.025 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:37.025 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829499+0000", 2024-09-11T06:49:37.025 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003461783 2024-09-11T06:49:37.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.025 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.026 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:37.026 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829588+0000", 2024-09-11T06:49:37.026 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8856000000000004e-05 2024-09-11T06:49:37.026 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:37.026 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:37.026 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:37.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.026 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:140 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'98, mlcod=13'93)", 2024-09-11T06:49:37.026 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:49:37.026 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.128560897, 2024-09-11T06:49:37.026 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0057995929999999996, 2024-09-11T06:49:37.026 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:37.026 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:37.027 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:37.027 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:37.027 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.027 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:37.027 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:49:37.027 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:37.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.027 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.027 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:37.027 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:49:37.027 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:37.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.027 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.027 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:37.027 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827708+0000", 2024-09-11T06:49:37.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.547e-06 2024-09-11T06:49:37.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.028 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:37.028 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827765+0000", 2024-09-11T06:49:37.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6266e-05 2024-09-11T06:49:37.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.028 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:37.028 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827791+0000", 2024-09-11T06:49:37.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6098999999999999e-05 2024-09-11T06:49:37.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.028 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:37.029 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827806+0000", 2024-09-11T06:49:37.029 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5537e-05 2024-09-11T06:49:37.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.029 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:37.029 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827928+0000", 2024-09-11T06:49:37.029 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000122012 2024-09-11T06:49:37.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.029 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:37.029 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827961+0000", 2024-09-11T06:49:37.029 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2663999999999998e-05 2024-09-11T06:49:37.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:37.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833409+0000", 2024-09-11T06:49:37.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054479350000000001 2024-09-11T06:49:37.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:37.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833506+0000", 2024-09-11T06:49:37.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7533000000000003e-05 2024-09-11T06:49:37.030 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:37.030 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:37.030 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:37.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.031 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:141 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'99, mlcod=13'93)", 2024-09-11T06:49:37.031 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:49:37.031 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.127667513999999, 2024-09-11T06:49:37.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055394709999999998, 2024-09-11T06:49:37.031 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:37.031 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:37.031 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:37.031 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:37.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.031 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:37.031 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:49:37.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:37.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:37.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:49:37.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:37.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:37.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828602+0000", 2024-09-11T06:49:37.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.46e-06 2024-09-11T06:49:37.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:37.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828618+0000", 2024-09-11T06:49:37.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6592999999999999e-05 2024-09-11T06:49:37.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:37.033 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828646+0000", 2024-09-11T06:49:37.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7552999999999999e-05 2024-09-11T06:49:37.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:37.033 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828663+0000", 2024-09-11T06:49:37.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.719e-05 2024-09-11T06:49:37.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:37.034 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829023+0000", 2024-09-11T06:49:37.034 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036054799999999998 2024-09-11T06:49:37.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.034 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.034 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:37.034 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829062+0000", 2024-09-11T06:49:37.034 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8865999999999997e-05 2024-09-11T06:49:37.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.034 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.034 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:37.034 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834078+0000", 2024-09-11T06:49:37.034 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050157480000000004 2024-09-11T06:49:37.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.035 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:37.035 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834140+0000", 2024-09-11T06:49:37.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1513000000000005e-05 2024-09-11T06:49:37.035 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:37.035 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:37.035 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:37.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.035 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:144 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'102, mlcod=13'98)", 2024-09-11T06:49:37.036 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:49:37.036 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.118506901, 2024-09-11T06:49:37.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003210348, 2024-09-11T06:49:37.036 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:37.036 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:37.036 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:37.036 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:37.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:37.036 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:49:37.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:37.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:37.037 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:49:37.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:37.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.037 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:37.037 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837768+0000", 2024-09-11T06:49:37.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9330000000000004e-06 2024-09-11T06:49:37.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.037 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:37.037 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837858+0000", 2024-09-11T06:49:37.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0185999999999993e-05 2024-09-11T06:49:37.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:37.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837890+0000", 2024-09-11T06:49:37.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2437000000000001e-05 2024-09-11T06:49:37.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:37.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837905+0000", 2024-09-11T06:49:37.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:49:37.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:37.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838019+0000", 2024-09-11T06:49:37.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000114336 2024-09-11T06:49:37.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.039 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.039 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:37.039 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838052+0000", 2024-09-11T06:49:37.039 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3139000000000002e-05 2024-09-11T06:49:37.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.039 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.039 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:37.039 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840914+0000", 2024-09-11T06:49:37.039 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028612780000000001 2024-09-11T06:49:37.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.040 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.040 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:37.040 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840971+0000", 2024-09-11T06:49:37.040 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7402999999999999e-05 2024-09-11T06:49:37.040 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:37.040 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:37.040 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:37.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.040 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.040 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:145 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'103, mlcod=13'98)", 2024-09-11T06:49:37.040 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:49:37.040 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.117715016999998, 2024-09-11T06:49:37.040 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054455409999999999, 2024-09-11T06:49:37.041 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:37.041 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:37.041 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:37.041 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:37.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:37.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:49:37.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:37.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:37.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:49:37.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:37.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:37.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838554+0000", 2024-09-11T06:49:37.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3430000000000001e-06 2024-09-11T06:49:37.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:37.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838640+0000", 2024-09-11T06:49:37.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5569999999999996e-05 2024-09-11T06:49:37.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:37.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838660+0000", 2024-09-11T06:49:37.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0576000000000001e-05 2024-09-11T06:49:37.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.043 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:37.043 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838674+0000", 2024-09-11T06:49:37.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3529999999999999e-05 2024-09-11T06:49:37.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.043 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:37.043 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838838+0000", 2024-09-11T06:49:37.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016468799999999999 2024-09-11T06:49:37.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.043 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:37.043 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838885+0000", 2024-09-11T06:49:37.044 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6884e-05 2024-09-11T06:49:37.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.044 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.044 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:37.044 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843901+0000", 2024-09-11T06:49:37.044 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050162310000000003 2024-09-11T06:49:37.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:37.044 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:37.044 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:37.044 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843998+0000", 2024-09-11T06:49:37.044 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.6718999999999993e-05 2024-09-11T06:49:37.044 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:37.045 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:37.045 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:37.045 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:37.045 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:37.045 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:37.046 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:49:37.047 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:37.042+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:37.081 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:49:37.082 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:49:37.100 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-09-11T06:49:37.018351+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":137660,"kb_used_data":2512,"kb_used_omap":7,"kb_used_meta":135032,"kb_avail":471721540,"statfs":{"total":483183820800,"available":483042856960,"internally_reserved":0,"allocated":2572288,"data_stored":1657570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7971,"internal_metadata":138272989},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000742"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T06:49:33.346727+0000","last_change":"2024-09-11T06:49:13.393788+0000","last_active":"2024-09-11T06:49:33.346727+0000","last_peered":"2024-09-11T06:49:33.346727+0000","last_clean":"2024-09-11T06:49:33.346727+0000","last_became_active":"2024-09-11T06:49:11.433437+0000","last_became_peered":"2024-09-11T06:49:11.433437+0000","last_unstale":"2024-09-11T06:49:33.346727+0000","last_undegraded":"2024-09-11T06:49:33.346727+0000","last_fullsized":"2024-09-11T06:49:33.346727+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:20:04.339208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001403348,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T06:49:33.346640+0000","last_change":"2024-09-11T06:49:21.882377+0000","last_active":"2024-09-11T06:49:33.346640+0000","last_peered":"2024-09-11T06:49:33.346640+0000","last_clean":"2024-09-11T06:49:33.346640+0000","last_became_active":"2024-09-11T06:49:21.881858+0000","last_became_peered":"2024-09-11T06:49:21.881858+0000","last_unstale":"2024-09-11T06:49:33.346640+0000","last_undegraded":"2024-09-11T06:49:33.346640+0000","last_fullsized":"2024-09-11T06:49:33.346640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:33:42.789131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T06:49:33.346605+0000","last_change":"2024-09-11T06:49:13.393298+0000","last_active":"2024-09-11T06:49:33.346605+0000","last_peered":"2024-09-11T06:49:33.346605+0000","last_clean":"2024-09-11T06:49:33.346605+0000","last_became_active":"2024-09-11T06:49:11.315089+0000","last_became_peered":"2024-09-11T06:49:11.315089+0000","last_unstale":"2024-09-11T06:49:33.346605+0000","last_undegraded":"2024-09-11T06:49:33.346605+0000","last_fullsized":"2024-09-11T06:49:33.346605+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:31.769443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095172900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T06:49:33.916273+0000","last_change":"2024-09-11T06:49:28.017449+0000","last_active":"2024-09-11T06:49:33.916273+0000","last_peered":"2024-09-11T06:49:33.916273+0000","last_clean":"2024-09-11T06:49:33.916273+0000","last_became_active":"2024-09-11T06:49:28.016860+0000","last_became_peered":"2024-09-11T06:49:28.016860+0000","last_unstale":"2024-09-11T06:49:33.916273+0000","last_undegraded":"2024-09-11T06:49:33.916273+0000","last_fullsized":"2024-09-11T06:49:33.916273+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:32:34.900467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T06:49:33.907207+0000","last_change":"2024-09-11T06:49:21.283656+0000","last_active":"2024-09-11T06:49:33.907207+0000","last_peered":"2024-09-11T06:49:33.907207+0000","last_clean":"2024-09-11T06:49:33.907207+0000","last_became_active":"2024-09-11T06:49:21.278679+0000","last_became_peered":"2024-09-11T06:49:21.278679+0000","last_unstale":"2024-09-11T06:49:33.907207+0000","last_undegraded":"2024-09-11T06:49:33.907207+0000","last_fullsized":"2024-09-11T06:49:33.907207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:20:11.653333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001910244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T06:49:33.346454+0000","last_change":"2024-09-11T06:49:21.274656+0000","last_active":"2024-09-11T06:49:33.346454+0000","last_peered":"2024-09-11T06:49:33.346454+0000","last_clean":"2024-09-11T06:49:33.346454+0000","last_became_active":"2024-09-11T06:49:21.273719+0000","last_became_peered":"2024-09-11T06:49:21.273719+0000","last_unstale":"2024-09-11T06:49:33.346454+0000","last_undegraded":"2024-09-11T06:49:33.346454+0000","last_fullsized":"2024-09-11T06:49:33.346454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:59.900889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010163780000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T06:49:33.346704+0000","last_change":"2024-09-11T06:49:21.274276+0000","last_active":"2024-09-11T06:49:33.346704+0000","last_peered":"2024-09-11T06:49:33.346704+0000","last_clean":"2024-09-11T06:49:33.346704+0000","last_became_active":"2024-09-11T06:49:21.273899+0000","last_became_peered":"2024-09-11T06:49:21.273899+0000","last_unstale":"2024-09-11T06:49:33.346704+0000","last_undegraded":"2024-09-11T06:49:33.346704+0000","last_fullsized":"2024-09-11T06:49:33.346704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:37:31.862801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077615599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T06:49:33.907224+0000","last_change":"2024-09-11T06:49:13.251115+0000","last_active":"2024-09-11T06:49:33.907224+0000","last_peered":"2024-09-11T06:49:33.907224+0000","last_clean":"2024-09-11T06:49:33.907224+0000","last_became_active":"2024-09-11T06:49:11.235231+0000","last_became_peered":"2024-09-11T06:49:11.235231+0000","last_unstale":"2024-09-11T06:49:33.907224+0000","last_undegraded":"2024-09-11T06:49:33.907224+0000","last_fullsized":"2024-09-11T06:49:33.907224+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:01:21.028466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011212469999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":188,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T06:49:33.346694+0000","last_change":"2024-09-11T06:49:06.112585+0000","last_active":"2024-09-11T06:49:33.346694+0000","last_peered":"2024-09-11T06:49:33.346694+0000","last_clean":"2024-09-11T06:49:33.346694+0000","last_became_active":"2024-09-11T06:49:06.110690+0000","last_became_peered":"2024-09-11T06:49:06.110690+0000","last_unstale":"2024-09-11T06:49:33.346694+0000","last_undegraded":"2024-09-11T06:49:33.346694+0000","last_fullsized":"2024-09-11T06:49:33.346694+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_clean_scrub_stamp":"2024-09-11T06:49:05.088839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:51.167301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640264,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":93547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640265,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":93528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640265,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":876544,"data_stored":688474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640266,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":98106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":47244640266,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640267,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":683915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:49:37.101 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-11T06:49:37.101 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T06:49:37.101 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:49:37.147 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:37.146+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:37.204 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:37.204 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:37.204 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:49:37.205 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:37.220 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:49:37.247 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:37.246+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:37.291 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:49:37.347 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:37.348 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:37.348 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:49:37.348 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:49:37.348 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:37.348 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:37.346+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:37.363 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:49:37.425 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:49:37.425 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:49:37.442 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:49:37.442 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:49:37.444 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:49:37.445 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:49:37.449 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:37.446+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:37.460 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:49:37.460 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:49:37.463 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-09-11T06:49:37.018351+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":137660,"kb_used_data":2512,"kb_used_omap":7,"kb_used_meta":135032,"kb_avail":471721540,"statfs":{"total":483183820800,"available":483042856960,"internally_reserved":0,"allocated":2572288,"data_stored":1657570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7971,"internal_metadata":138272989},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000742"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T06:49:33.346727+0000","last_change":"2024-09-11T06:49:13.393788+0000","last_active":"2024-09-11T06:49:33.346727+0000","last_peered":"2024-09-11T06:49:33.346727+0000","last_clean":"2024-09-11T06:49:33.346727+0000","last_became_active":"2024-09-11T06:49:11.433437+0000","last_became_peered":"2024-09-11T06:49:11.433437+0000","last_unstale":"2024-09-11T06:49:33.346727+0000","last_undegraded":"2024-09-11T06:49:33.346727+0000","last_fullsized":"2024-09-11T06:49:33.346727+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:20:04.339208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001403348,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T06:49:33.346640+0000","last_change":"2024-09-11T06:49:21.882377+0000","last_active":"2024-09-11T06:49:33.346640+0000","last_peered":"2024-09-11T06:49:33.346640+0000","last_clean":"2024-09-11T06:49:33.346640+0000","last_became_active":"2024-09-11T06:49:21.881858+0000","last_became_peered":"2024-09-11T06:49:21.881858+0000","last_unstale":"2024-09-11T06:49:33.346640+0000","last_undegraded":"2024-09-11T06:49:33.346640+0000","last_fullsized":"2024-09-11T06:49:33.346640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:33:42.789131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T06:49:33.346605+0000","last_change":"2024-09-11T06:49:13.393298+0000","last_active":"2024-09-11T06:49:33.346605+0000","last_peered":"2024-09-11T06:49:33.346605+0000","last_clean":"2024-09-11T06:49:33.346605+0000","last_became_active":"2024-09-11T06:49:11.315089+0000","last_became_peered":"2024-09-11T06:49:11.315089+0000","last_unstale":"2024-09-11T06:49:33.346605+0000","last_undegraded":"2024-09-11T06:49:33.346605+0000","last_fullsized":"2024-09-11T06:49:33.346605+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:31.769443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095172900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T06:49:33.916273+0000","last_change":"2024-09-11T06:49:28.017449+0000","last_active":"2024-09-11T06:49:33.916273+0000","last_peered":"2024-09-11T06:49:33.916273+0000","last_clean":"2024-09-11T06:49:33.916273+0000","last_became_active":"2024-09-11T06:49:28.016860+0000","last_became_peered":"2024-09-11T06:49:28.016860+0000","last_unstale":"2024-09-11T06:49:33.916273+0000","last_undegraded":"2024-09-11T06:49:33.916273+0000","last_fullsized":"2024-09-11T06:49:33.916273+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:32:34.900467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T06:49:33.907207+0000","last_change":"2024-09-11T06:49:21.283656+0000","last_active":"2024-09-11T06:49:33.907207+0000","last_peered":"2024-09-11T06:49:33.907207+0000","last_clean":"2024-09-11T06:49:33.907207+0000","last_became_active":"2024-09-11T06:49:21.278679+0000","last_became_peered":"2024-09-11T06:49:21.278679+0000","last_unstale":"2024-09-11T06:49:33.907207+0000","last_undegraded":"2024-09-11T06:49:33.907207+0000","last_fullsized":"2024-09-11T06:49:33.907207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:20:11.653333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001910244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T06:49:33.346454+0000","last_change":"2024-09-11T06:49:21.274656+0000","last_active":"2024-09-11T06:49:33.346454+0000","last_peered":"2024-09-11T06:49:33.346454+0000","last_clean":"2024-09-11T06:49:33.346454+0000","last_became_active":"2024-09-11T06:49:21.273719+0000","last_became_peered":"2024-09-11T06:49:21.273719+0000","last_unstale":"2024-09-11T06:49:33.346454+0000","last_undegraded":"2024-09-11T06:49:33.346454+0000","last_fullsized":"2024-09-11T06:49:33.346454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:59.900889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010163780000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T06:49:33.346704+0000","last_change":"2024-09-11T06:49:21.274276+0000","last_active":"2024-09-11T06:49:33.346704+0000","last_peered":"2024-09-11T06:49:33.346704+0000","last_clean":"2024-09-11T06:49:33.346704+0000","last_became_active":"2024-09-11T06:49:21.273899+0000","last_became_peered":"2024-09-11T06:49:21.273899+0000","last_unstale":"2024-09-11T06:49:33.346704+0000","last_undegraded":"2024-09-11T06:49:33.346704+0000","last_fullsized":"2024-09-11T06:49:33.346704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:37:31.862801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077615599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T06:49:33.907224+0000","last_change":"2024-09-11T06:49:13.251115+0000","last_active":"2024-09-11T06:49:33.907224+0000","last_peered":"2024-09-11T06:49:33.907224+0000","last_clean":"2024-09-11T06:49:33.907224+0000","last_became_active":"2024-09-11T06:49:11.235231+0000","last_became_peered":"2024-09-11T06:49:11.235231+0000","last_unstale":"2024-09-11T06:49:33.907224+0000","last_undegraded":"2024-09-11T06:49:33.907224+0000","last_fullsized":"2024-09-11T06:49:33.907224+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:01:21.028466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011212469999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":188,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-11T06:49:33.346694+0000","last_change":"2024-09-11T06:49:06.112585+0000","last_active":"2024-09-11T06:49:33.346694+0000","last_peered":"2024-09-11T06:49:33.346694+0000","last_clean":"2024-09-11T06:49:33.346694+0000","last_became_active":"2024-09-11T06:49:06.110690+0000","last_became_peered":"2024-09-11T06:49:06.110690+0000","last_unstale":"2024-09-11T06:49:33.346694+0000","last_undegraded":"2024-09-11T06:49:33.346694+0000","last_fullsized":"2024-09-11T06:49:33.346694+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_clean_scrub_stamp":"2024-09-11T06:49:05.088839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:51.167301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640264,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":93547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640265,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":93528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640265,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":876544,"data_stored":688474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640266,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":98106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":47244640266,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640267,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":683915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:49:37.481 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:49:37.481 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:49:37.489 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T06:49:37.502 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:49:37.502 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:49:37.522 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:49:37.522 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:49:37.523 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:37.523 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:37.540 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:49:37.550 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:37.546+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:37.650 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:37.646+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:37.698 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:37.698 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:37.711 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:49:37.752 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:37.750+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:37.853 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:37.850+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:37.874 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:37.874 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:37.889 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:49:37.954 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:37.950+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:38.050 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:38.050 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:38.055 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:38.054+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:38.064 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:49:38.156 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:38.154+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:38.196 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:38.197 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:38.211 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:49:38.256 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:38.254+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:38.357 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:38.358+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:38.375 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:38.376 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:38.384 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is unset 2024-09-11T06:49:38.391 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:49:38.458 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:38.458+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:38.510 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:38.510 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:38.524 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:49:38.566 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:38.566+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:38.661 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:38.658+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:38.687 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:38.688 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:38.702 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:49:38.762 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:38.762+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:38.860 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:38.860 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:38.863 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:38.862+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:38.872 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:49:38.964 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:38.962+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:38.991 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:38.991 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:39.005 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:49:39.065 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:39.066+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:39.166 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:39.166+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:39.167 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:39.167 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:39.180 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:49:39.266 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:39.266+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:39.340 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:39.340 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:39.353 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:49:39.367 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:39.366+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:39.468 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:39.466+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:39.511 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:39.511 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:39.525 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:49:39.542 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:49:39.571 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:39.570+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:39.658 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:39.658 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:39.669 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:39.666+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:39.674 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:49:39.713 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:49:39.714 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:49:39.730 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:49:39.730 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:49:39.746 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:49:39.746 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:49:39.767 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:49:39.767 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:49:39.770 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:39.766+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:39.788 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:49:39.788 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:49:39.808 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:49:39.808 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:49:39.826 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:39.826 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:39.826 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:49:39.826 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:49:39.826 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:39.839 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:49:39.871 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:39.870+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:39.972 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:39.970+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:40.002 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:40.002 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:49:40.002 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:49:40.003 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:49:40.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.003 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:54 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'32, mlcod=13'29)", 2024-09-11T06:49:40.003 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:49:40.003 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.792119872999997, 2024-09-11T06:49:40.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034271670000000001, 2024-09-11T06:49:40.003 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:40.003 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:40.003 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:40.003 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:40.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:40.004 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:49:40.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.004 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:40.004 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:49:40.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.004 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:40.004 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210361+0000", 2024-09-11T06:49:40.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1260000000000001e-06 2024-09-11T06:49:40.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:40.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210370+0000", 2024-09-11T06:49:40.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8200000000000003e-06 2024-09-11T06:49:40.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:40.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210390+0000", 2024-09-11T06:49:40.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0126e-05 2024-09-11T06:49:40.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:40.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210404+0000", 2024-09-11T06:49:40.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4589999999999999e-05 2024-09-11T06:49:40.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.006 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:40.006 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210538+0000", 2024-09-11T06:49:40.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000133719 2024-09-11T06:49:40.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.006 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:40.006 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210593+0000", 2024-09-11T06:49:40.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5016999999999999e-05 2024-09-11T06:49:40.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.006 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:40.007 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213695+0000", 2024-09-11T06:49:40.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003101866 2024-09-11T06:49:40.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.007 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:40.007 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213787+0000", 2024-09-11T06:49:40.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1903000000000004e-05 2024-09-11T06:49:40.007 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.007 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:40.007 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.007 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:55 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'33, mlcod=13'29)", 2024-09-11T06:49:40.007 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:49:40.008 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.791100634999999, 2024-09-11T06:49:40.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.005086091, 2024-09-11T06:49:40.008 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:40.008 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:40.008 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:40.008 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:40.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.008 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:40.008 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:49:40.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.008 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:40.009 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:49:40.009 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.009 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:40.009 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211380+0000", 2024-09-11T06:49:40.009 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.097e-06 2024-09-11T06:49:40.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.009 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:40.009 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211537+0000", 2024-09-11T06:49:40.009 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000157369 2024-09-11T06:49:40.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:40.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211559+0000", 2024-09-11T06:49:40.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000001e-05 2024-09-11T06:49:40.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:40.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211573+0000", 2024-09-11T06:49:40.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4946000000000001e-05 2024-09-11T06:49:40.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:40.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211676+0000", 2024-09-11T06:49:40.011 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0001026 2024-09-11T06:49:40.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.011 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:40.011 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211709+0000", 2024-09-11T06:49:40.011 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3071999999999998e-05 2024-09-11T06:49:40.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.011 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:40.011 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216379+0000", 2024-09-11T06:49:40.011 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0046693780000000001 2024-09-11T06:49:40.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.012 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.012 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:40.012 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216465+0000", 2024-09-11T06:49:40.012 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6469000000000002e-05 2024-09-11T06:49:40.012 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.012 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:40.013 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.013 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:56 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'34, mlcod=13'29)", 2024-09-11T06:49:40.013 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:49:40.013 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.789666775999997, 2024-09-11T06:49:40.013 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0060310629999999997, 2024-09-11T06:49:40.013 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:40.013 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:40.013 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:40.013 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:40.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.013 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:40.014 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:49:40.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.016 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.016 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:40.016 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:49:40.017 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.017 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:40.017 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212814+0000", 2024-09-11T06:49:40.017 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0869999999999999e-06 2024-09-11T06:49:40.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.017 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:40.017 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212822+0000", 2024-09-11T06:49:40.017 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0870000000000007e-06 2024-09-11T06:49:40.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.018 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:40.018 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212840+0000", 2024-09-11T06:49:40.018 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:49:40.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.018 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.018 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:40.018 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212854+0000", 2024-09-11T06:49:40.018 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4003e-05 2024-09-11T06:49:40.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.018 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.018 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:40.018 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213026+0000", 2024-09-11T06:49:40.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017207499999999999 2024-09-11T06:49:40.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:40.019 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213063+0000", 2024-09-11T06:49:40.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7187000000000002e-05 2024-09-11T06:49:40.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:40.019 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218762+0000", 2024-09-11T06:49:40.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0056993290000000004 2024-09-11T06:49:40.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.020 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:40.020 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218844+0000", 2024-09-11T06:49:40.020 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1701999999999998e-05 2024-09-11T06:49:40.020 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.020 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:40.020 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.020 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:60 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'38, mlcod=13'36)", 2024-09-11T06:49:40.020 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:49:40.020 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.775725426999998, 2024-09-11T06:49:40.021 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039254270000000004, 2024-09-11T06:49:40.021 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:40.021 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:40.021 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:40.021 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:40.021 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.021 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:40.021 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:49:40.021 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.027 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:40.027 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:49:40.027 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.027 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.028 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:40.028 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226756+0000", 2024-09-11T06:49:40.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-11T06:49:40.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.028 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:40.028 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226768+0000", 2024-09-11T06:49:40.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1919999999999999e-05 2024-09-11T06:49:40.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.028 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:40.028 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226797+0000", 2024-09-11T06:49:40.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9573e-05 2024-09-11T06:49:40.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.029 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:40.029 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226818+0000", 2024-09-11T06:49:40.029 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0432999999999999e-05 2024-09-11T06:49:40.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.029 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:40.029 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226909+0000", 2024-09-11T06:49:40.029 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0920000000000004e-05 2024-09-11T06:49:40.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.029 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:40.029 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226936+0000", 2024-09-11T06:49:40.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7472999999999999e-05 2024-09-11T06:49:40.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:40.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230596+0000", 2024-09-11T06:49:40.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0036602520000000001 2024-09-11T06:49:40.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:40.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230680+0000", 2024-09-11T06:49:40.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3106e-05 2024-09-11T06:49:40.030 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.030 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:40.031 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.031 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:107 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'69, mlcod=13'64)", 2024-09-11T06:49:40.031 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:49:40.031 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.559277969999997, 2024-09-11T06:49:40.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040728830000000002, 2024-09-11T06:49:40.031 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:40.031 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:40.031 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:40.031 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:40.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.031 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:40.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:49:40.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:40.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:49:40.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:40.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443203+0000", 2024-09-11T06:49:40.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1480000000000001e-06 2024-09-11T06:49:40.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:40.033 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443222+0000", 2024-09-11T06:49:40.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9040000000000001e-05 2024-09-11T06:49:40.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:40.033 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443246+0000", 2024-09-11T06:49:40.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4606000000000001e-05 2024-09-11T06:49:40.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:40.033 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443277+0000", 2024-09-11T06:49:40.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0207999999999999e-05 2024-09-11T06:49:40.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.034 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.034 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:40.034 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443340+0000", 2024-09-11T06:49:40.034 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3602000000000005e-05 2024-09-11T06:49:40.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.034 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.034 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:40.034 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443363+0000", 2024-09-11T06:49:40.034 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2331e-05 2024-09-11T06:49:40.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.034 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.034 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:40.035 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447216+0000", 2024-09-11T06:49:40.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0038539690000000001 2024-09-11T06:49:40.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.035 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:40.035 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447274+0000", 2024-09-11T06:49:40.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7979000000000002e-05 2024-09-11T06:49:40.035 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.035 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:40.035 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.035 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:111 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'73, mlcod=13'67)", 2024-09-11T06:49:40.035 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:49:40.036 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.552766339999998, 2024-09-11T06:49:40.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034372360000000002, 2024-09-11T06:49:40.036 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:40.036 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:40.036 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:40.036 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:40.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:40.036 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:49:40.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:40.036 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:49:40.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.037 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:40.037 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449714+0000", 2024-09-11T06:49:40.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5000000000000001e-07 2024-09-11T06:49:40.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.037 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:40.037 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449721+0000", 2024-09-11T06:49:40.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8530000000000003e-06 2024-09-11T06:49:40.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.037 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:40.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449736+0000", 2024-09-11T06:49:40.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5114e-05 2024-09-11T06:49:40.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:40.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449743+0000", 2024-09-11T06:49:40.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0210000000000002e-06 2024-09-11T06:49:40.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:40.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449804+0000", 2024-09-11T06:49:40.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0479999999999997e-05 2024-09-11T06:49:40.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.039 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.039 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:40.039 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449826+0000", 2024-09-11T06:49:40.039 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1922000000000001e-05 2024-09-11T06:49:40.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.039 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.039 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:40.039 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453104+0000", 2024-09-11T06:49:40.039 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003278652 2024-09-11T06:49:40.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.039 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.039 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:40.039 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453150+0000", 2024-09-11T06:49:40.039 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6244e-05 2024-09-11T06:49:40.040 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.040 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:40.040 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.040 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.040 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:114 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'76, mlcod=13'71)", 2024-09-11T06:49:40.040 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:49:40.040 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.548088268999997, 2024-09-11T06:49:40.040 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0042576660000000002, 2024-09-11T06:49:40.040 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:40.040 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:40.040 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:40.041 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:40.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:40.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:49:40.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:40.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:49:40.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:40.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454392+0000", 2024-09-11T06:49:40.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5999999999999991e-07 2024-09-11T06:49:40.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:40.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454399+0000", 2024-09-11T06:49:40.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.878e-06 2024-09-11T06:49:40.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:40.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454415+0000", 2024-09-11T06:49:40.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5659999999999999e-05 2024-09-11T06:49:40.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.043 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:40.043 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454422+0000", 2024-09-11T06:49:40.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:49:40.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.043 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:40.043 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454474+0000", 2024-09-11T06:49:40.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1860999999999997e-05 2024-09-11T06:49:40.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.043 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:40.043 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454497+0000", 2024-09-11T06:49:40.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3686e-05 2024-09-11T06:49:40.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.044 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:40.044 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458552+0000", 2024-09-11T06:49:40.044 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040544550000000002 2024-09-11T06:49:40.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.044 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.044 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:40.044 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458649+0000", 2024-09-11T06:49:40.044 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7005999999999999e-05 2024-09-11T06:49:40.044 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.044 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:40.044 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.044 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.045 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:115 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'77, mlcod=13'72)", 2024-09-11T06:49:40.045 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:40.045 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.546747656999997, 2024-09-11T06:49:40.045 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003355584, 2024-09-11T06:49:40.045 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:40.045 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:40.045 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:40.045 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:40.045 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.045 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:40.045 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:40.045 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:40.046 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:40.046 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:40.046 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:40.046 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5999999999999999e-07 2024-09-11T06:49:40.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:40.046 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455803+0000", 2024-09-11T06:49:40.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0310999999999998e-05 2024-09-11T06:49:40.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.047 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:40.047 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455815+0000", 2024-09-11T06:49:40.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2787e-05 2024-09-11T06:49:40.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.047 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:40.047 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455823+0000", 2024-09-11T06:49:40.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5959999999999998e-06 2024-09-11T06:49:40.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.048 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:40.048 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455875+0000", 2024-09-11T06:49:40.048 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2054e-05 2024-09-11T06:49:40.048 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.048 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.048 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:40.048 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455896+0000", 2024-09-11T06:49:40.048 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0554999999999999e-05 2024-09-11T06:49:40.048 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.048 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.048 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:40.048 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459000+0000", 2024-09-11T06:49:40.048 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003104842 2024-09-11T06:49:40.048 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.049 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.049 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:40.049 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459087+0000", 2024-09-11T06:49:40.049 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6978999999999995e-05 2024-09-11T06:49:40.049 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.049 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:40.049 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.049 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.049 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.049 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:120 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'82, mlcod=13'78)", 2024-09-11T06:49:40.049 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:49:40.049 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.536070371999998, 2024-09-11T06:49:40.049 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0064612549999999999, 2024-09-11T06:49:40.049 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:40.050 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:40.050 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:40.050 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:40.050 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.050 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:40.050 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:49:40.050 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.050 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.050 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:40.050 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:49:40.050 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:40.051 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466410+0000", 2024-09-11T06:49:40.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0389999999999999e-06 2024-09-11T06:49:40.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:40.051 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466496+0000", 2024-09-11T06:49:40.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5519000000000006e-05 2024-09-11T06:49:40.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:40.051 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466510+0000", 2024-09-11T06:49:40.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3842999999999999e-05 2024-09-11T06:49:40.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.052 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.052 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:40.052 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466517+0000", 2024-09-11T06:49:40.052 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5229999999999998e-06 2024-09-11T06:49:40.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.052 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.052 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:40.052 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466595+0000", 2024-09-11T06:49:40.052 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8019e-05 2024-09-11T06:49:40.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.052 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.052 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:40.052 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466612+0000", 2024-09-11T06:49:40.053 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7013000000000001e-05 2024-09-11T06:49:40.053 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.053 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.053 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:40.053 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472787+0000", 2024-09-11T06:49:40.053 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0061750269999999996 2024-09-11T06:49:40.053 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.053 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.053 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:40.053 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472870+0000", 2024-09-11T06:49:40.053 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3271999999999998e-05 2024-09-11T06:49:40.053 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.053 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:40.054 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.054 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.054 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.054 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:121 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'83, mlcod=13'78)", 2024-09-11T06:49:40.054 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:49:40.054 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.534845576000002, 2024-09-11T06:49:40.054 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055613069999999997, 2024-09-11T06:49:40.054 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:40.054 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:40.054 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:40.054 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:40.054 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.054 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:40.054 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:49:40.055 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.055 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:40.055 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:49:40.055 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.055 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:40.055 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467635+0000", 2024-09-11T06:49:40.055 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6899999999999996e-07 2024-09-11T06:49:40.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.056 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:40.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467727+0000", 2024-09-11T06:49:40.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1800999999999997e-05 2024-09-11T06:49:40.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.056 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.056 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:40.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467740+0000", 2024-09-11T06:49:40.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3747999999999999e-05 2024-09-11T06:49:40.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.056 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.056 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:40.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467753+0000", 2024-09-11T06:49:40.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2807e-05 2024-09-11T06:49:40.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.057 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.057 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:40.057 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467812+0000", 2024-09-11T06:49:40.057 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8381999999999998e-05 2024-09-11T06:49:40.057 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.057 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.057 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:40.057 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467828+0000", 2024-09-11T06:49:40.057 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6728e-05 2024-09-11T06:49:40.057 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.057 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.057 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:40.057 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473112+0000", 2024-09-11T06:49:40.058 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052833619999999998 2024-09-11T06:49:40.058 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.058 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.058 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:40.058 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473195+0000", 2024-09-11T06:49:40.058 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3609999999999994e-05 2024-09-11T06:49:40.058 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.058 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:40.058 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.058 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.060 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.060 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:131 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'89, mlcod=13'85)", 2024-09-11T06:49:40.060 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:49:40.060 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.183439860999997, 2024-09-11T06:49:40.060 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052492260000000001, 2024-09-11T06:49:40.060 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:40.060 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:40.061 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:40.061 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:40.061 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.061 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:40.061 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:49:40.061 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.061 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.061 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.061 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:40.061 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:49:40.061 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.061 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.061 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.062 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:40.062 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819041+0000", 2024-09-11T06:49:40.062 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4360000000000001e-06 2024-09-11T06:49:40.062 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.062 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.062 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:40.062 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819100+0000", 2024-09-11T06:49:40.062 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9225999999999998e-05 2024-09-11T06:49:40.062 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.062 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.062 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:40.062 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819128+0000", 2024-09-11T06:49:40.062 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.779e-05 2024-09-11T06:49:40.062 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.063 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.063 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:40.063 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819142+0000", 2024-09-11T06:49:40.063 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3573e-05 2024-09-11T06:49:40.063 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.063 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.063 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:40.063 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819263+0000", 2024-09-11T06:49:40.063 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000121086 2024-09-11T06:49:40.063 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.063 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.063 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:40.063 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819292+0000", 2024-09-11T06:49:40.064 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9363e-05 2024-09-11T06:49:40.064 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.064 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.064 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:40.064 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824185+0000", 2024-09-11T06:49:40.064 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0048924620000000002 2024-09-11T06:49:40.064 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.064 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.064 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:40.064 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824289+0000", 2024-09-11T06:49:40.064 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010429000000000001 2024-09-11T06:49:40.064 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.064 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:40.065 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.065 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.065 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:132 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'90, mlcod=13'85)", 2024-09-11T06:49:40.065 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:49:40.065 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.182665170999996, 2024-09-11T06:49:40.065 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.004532342, 2024-09-11T06:49:40.065 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:40.065 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:40.065 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:40.065 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:40.065 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.065 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:40.065 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:49:40.066 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.066 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.066 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.066 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:40.066 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:49:40.066 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.066 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.066 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.066 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:40.066 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819816+0000", 2024-09-11T06:49:40.066 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.61e-06 2024-09-11T06:49:40.066 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.066 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.066 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:40.067 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819827+0000", 2024-09-11T06:49:40.067 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1e-05 2024-09-11T06:49:40.067 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.067 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.067 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:40.067 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819851+0000", 2024-09-11T06:49:40.067 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4267e-05 2024-09-11T06:49:40.067 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.067 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.067 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:40.067 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819864+0000", 2024-09-11T06:49:40.067 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2767e-05 2024-09-11T06:49:40.068 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.068 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.068 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:40.068 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820071+0000", 2024-09-11T06:49:40.068 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000207061 2024-09-11T06:49:40.068 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.068 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.068 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:40.068 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820103+0000", 2024-09-11T06:49:40.068 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2326999999999997e-05 2024-09-11T06:49:40.068 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.068 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.068 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:40.068 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824302+0000", 2024-09-11T06:49:40.069 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0041986000000000002 2024-09-11T06:49:40.069 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.069 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.069 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:40.069 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824347+0000", 2024-09-11T06:49:40.069 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4709999999999997e-05 2024-09-11T06:49:40.069 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.069 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:40.069 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.069 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.069 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.069 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:133 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'91, mlcod=13'86)", 2024-09-11T06:49:40.069 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:49:40.069 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.182033511999997, 2024-09-11T06:49:40.070 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039421459999999997, 2024-09-11T06:49:40.070 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:40.070 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:40.070 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:40.070 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:40.070 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.070 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:40.070 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:49:40.070 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.070 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.070 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.070 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:40.070 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:49:40.071 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.071 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.071 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.071 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:40.071 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820448+0000", 2024-09-11T06:49:40.071 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.84e-06 2024-09-11T06:49:40.071 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.071 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.071 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:40.071 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820463+0000", 2024-09-11T06:49:40.071 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5356e-05 2024-09-11T06:49:40.071 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.071 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.071 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:40.072 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820499+0000", 2024-09-11T06:49:40.072 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.625e-05 2024-09-11T06:49:40.072 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.072 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.072 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:40.072 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820546+0000", 2024-09-11T06:49:40.072 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7017000000000001e-05 2024-09-11T06:49:40.072 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.072 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.072 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:40.072 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820882+0000", 2024-09-11T06:49:40.072 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033512 2024-09-11T06:49:40.072 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.073 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.073 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:40.073 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820941+0000", 2024-09-11T06:49:40.073 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8922999999999998e-05 2024-09-11T06:49:40.073 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.073 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.073 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:40.073 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824351+0000", 2024-09-11T06:49:40.073 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034101940000000001 2024-09-11T06:49:40.073 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.073 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.073 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:40.073 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824388+0000", 2024-09-11T06:49:40.074 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7446e-05 2024-09-11T06:49:40.074 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.074 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:40.074 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.074 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.074 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.074 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:134 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'92, mlcod=13'86)", 2024-09-11T06:49:40.074 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:49:40.074 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.181374394000002, 2024-09-11T06:49:40.074 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033248179999999998, 2024-09-11T06:49:40.074 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:40.074 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:40.075 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:40.075 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:40.075 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.075 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:40.075 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:49:40.075 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.075 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.075 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.075 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:40.075 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:49:40.075 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.075 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.075 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.075 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:40.076 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821107+0000", 2024-09-11T06:49:40.076 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7430000000000001e-06 2024-09-11T06:49:40.076 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.076 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.076 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:40.076 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821205+0000", 2024-09-11T06:49:40.076 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7666999999999998e-05 2024-09-11T06:49:40.076 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.076 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.076 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:40.076 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821234+0000", 2024-09-11T06:49:40.076 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9756000000000001e-05 2024-09-11T06:49:40.076 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.076 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.077 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:40.077 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821254+0000", 2024-09-11T06:49:40.077 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9446e-05 2024-09-11T06:49:40.077 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.077 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.077 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:40.077 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821846+0000", 2024-09-11T06:49:40.077 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00059187599999999997 2024-09-11T06:49:40.077 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.077 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.077 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:40.077 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821894+0000", 2024-09-11T06:49:40.077 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8766e-05 2024-09-11T06:49:40.077 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.078 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.078 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:40.078 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824392+0000", 2024-09-11T06:49:40.078 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0024977509999999999 2024-09-11T06:49:40.078 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.078 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.078 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:40.078 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824430+0000", 2024-09-11T06:49:40.078 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7812999999999999e-05 2024-09-11T06:49:40.078 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.078 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:40.078 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.078 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.079 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.079 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:137 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'95, mlcod=13'86)", 2024-09-11T06:49:40.079 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:49:40.079 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.178912253, 2024-09-11T06:49:40.079 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003084299, 2024-09-11T06:49:40.079 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:40.079 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:40.079 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:40.079 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:40.079 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.079 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:40.079 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:49:40.079 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.079 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.079 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.080 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:40.080 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:49:40.080 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.080 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.080 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.080 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:40.080 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823568+0000", 2024-09-11T06:49:40.080 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1030000000000001e-06 2024-09-11T06:49:40.080 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.080 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.080 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:40.080 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823662+0000", 2024-09-11T06:49:40.080 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3529999999999994e-05 2024-09-11T06:49:40.081 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.081 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.081 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:40.081 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823686+0000", 2024-09-11T06:49:40.081 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4358999999999998e-05 2024-09-11T06:49:40.081 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.081 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.081 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:40.081 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823701+0000", 2024-09-11T06:49:40.081 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4847e-05 2024-09-11T06:49:40.081 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.081 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.081 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:40.081 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824518+0000", 2024-09-11T06:49:40.082 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00081716700000000005 2024-09-11T06:49:40.082 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.082 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.082 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:40.082 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824546+0000", 2024-09-11T06:49:40.082 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7860000000000001e-05 2024-09-11T06:49:40.082 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.082 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.082 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:40.082 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826558+0000", 2024-09-11T06:49:40.082 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0020120009999999998 2024-09-11T06:49:40.082 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.082 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.083 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:40.083 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826652+0000", 2024-09-11T06:49:40.083 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3431999999999996e-05 2024-09-11T06:49:40.083 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.083 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:40.083 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.083 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.083 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.083 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:139 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'97, mlcod=13'86)", 2024-09-11T06:49:40.083 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:49:40.083 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.176623534000001, 2024-09-11T06:49:40.083 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003732221, 2024-09-11T06:49:40.083 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:40.083 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:40.083 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:40.084 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:40.084 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.084 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:40.084 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:49:40.084 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.084 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.084 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.084 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:40.084 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:49:40.084 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.084 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.084 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.084 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:40.084 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825858+0000", 2024-09-11T06:49:40.085 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5829999999999999e-06 2024-09-11T06:49:40.085 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.085 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.085 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:40.085 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825874+0000", 2024-09-11T06:49:40.085 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5967000000000001e-05 2024-09-11T06:49:40.085 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.085 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.085 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:40.085 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825909+0000", 2024-09-11T06:49:40.085 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5692999999999999e-05 2024-09-11T06:49:40.085 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.085 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.086 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:40.086 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825923+0000", 2024-09-11T06:49:40.086 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4080000000000001e-05 2024-09-11T06:49:40.086 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.086 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.086 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:40.086 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826010+0000", 2024-09-11T06:49:40.086 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6248999999999994e-05 2024-09-11T06:49:40.086 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.086 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.086 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:40.086 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826038+0000", 2024-09-11T06:49:40.086 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8010000000000001e-05 2024-09-11T06:49:40.086 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.087 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.087 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:40.087 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829499+0000", 2024-09-11T06:49:40.087 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003461783 2024-09-11T06:49:40.087 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.087 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.087 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:40.087 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829588+0000", 2024-09-11T06:49:40.087 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8856000000000004e-05 2024-09-11T06:49:40.087 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.087 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:40.087 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.087 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.088 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.088 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:140 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'98, mlcod=13'93)", 2024-09-11T06:49:40.088 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:49:40.088 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.174772805000003, 2024-09-11T06:49:40.088 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0057995929999999996, 2024-09-11T06:49:40.088 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:40.088 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:40.088 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:40.088 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:40.088 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.088 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:40.088 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:49:40.088 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.089 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.089 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.089 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:40.089 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:49:40.089 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.089 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.089 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.089 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:40.089 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827708+0000", 2024-09-11T06:49:40.089 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.547e-06 2024-09-11T06:49:40.089 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.089 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.089 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:40.090 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827765+0000", 2024-09-11T06:49:40.090 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6266e-05 2024-09-11T06:49:40.090 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.090 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.090 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:40.090 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827791+0000", 2024-09-11T06:49:40.090 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6098999999999999e-05 2024-09-11T06:49:40.090 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.090 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.090 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:40.090 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827806+0000", 2024-09-11T06:49:40.090 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5537e-05 2024-09-11T06:49:40.090 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.090 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.091 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:40.091 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827928+0000", 2024-09-11T06:49:40.091 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000122012 2024-09-11T06:49:40.091 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.091 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.091 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:40.091 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827961+0000", 2024-09-11T06:49:40.091 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2663999999999998e-05 2024-09-11T06:49:40.091 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.091 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.091 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:40.091 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833409+0000", 2024-09-11T06:49:40.091 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054479350000000001 2024-09-11T06:49:40.091 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.092 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.092 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:40.092 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833506+0000", 2024-09-11T06:49:40.092 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7533000000000003e-05 2024-09-11T06:49:40.092 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.092 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:40.092 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.092 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.092 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.092 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:141 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'99, mlcod=13'93)", 2024-09-11T06:49:40.092 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:49:40.092 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.173879421999999, 2024-09-11T06:49:40.093 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055394709999999998, 2024-09-11T06:49:40.093 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:40.093 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:40.093 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:40.093 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:40.093 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.093 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:40.093 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:49:40.093 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.093 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.093 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.093 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:40.093 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:49:40.094 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.094 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.094 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.094 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:40.094 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828602+0000", 2024-09-11T06:49:40.094 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.46e-06 2024-09-11T06:49:40.094 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.094 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.094 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:40.094 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828618+0000", 2024-09-11T06:49:40.094 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6592999999999999e-05 2024-09-11T06:49:40.094 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.094 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.095 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:40.095 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828646+0000", 2024-09-11T06:49:40.095 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7552999999999999e-05 2024-09-11T06:49:40.095 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.095 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.095 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:40.095 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828663+0000", 2024-09-11T06:49:40.095 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.719e-05 2024-09-11T06:49:40.095 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.095 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.095 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:40.095 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829023+0000", 2024-09-11T06:49:40.095 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036054799999999998 2024-09-11T06:49:40.095 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.096 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.096 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:40.096 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829062+0000", 2024-09-11T06:49:40.096 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8865999999999997e-05 2024-09-11T06:49:40.096 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.096 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.096 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:40.096 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834078+0000", 2024-09-11T06:49:40.096 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050157480000000004 2024-09-11T06:49:40.096 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.096 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.096 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:40.096 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834140+0000", 2024-09-11T06:49:40.097 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1513000000000005e-05 2024-09-11T06:49:40.097 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.097 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:40.097 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.097 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.097 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.097 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:144 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'102, mlcod=13'98)", 2024-09-11T06:49:40.097 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:49:40.097 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.164718809, 2024-09-11T06:49:40.097 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003210348, 2024-09-11T06:49:40.097 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:40.097 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:40.097 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:40.098 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:40.098 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.098 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:40.098 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:49:40.098 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.098 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.098 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.098 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:40.098 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:49:40.098 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.098 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.098 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.098 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:40.099 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837768+0000", 2024-09-11T06:49:40.099 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9330000000000004e-06 2024-09-11T06:49:40.099 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.099 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.099 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:40.099 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837858+0000", 2024-09-11T06:49:40.099 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0185999999999993e-05 2024-09-11T06:49:40.099 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.099 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.099 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:40.099 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837890+0000", 2024-09-11T06:49:40.099 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2437000000000001e-05 2024-09-11T06:49:40.099 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.099 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.100 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:40.100 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837905+0000", 2024-09-11T06:49:40.100 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:49:40.100 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.100 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.100 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:40.100 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838019+0000", 2024-09-11T06:49:40.100 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000114336 2024-09-11T06:49:40.100 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.100 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.100 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:40.100 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838052+0000", 2024-09-11T06:49:40.100 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3139000000000002e-05 2024-09-11T06:49:40.101 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.101 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.101 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:40.101 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840914+0000", 2024-09-11T06:49:40.101 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028612780000000001 2024-09-11T06:49:40.101 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.101 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.101 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:40.101 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840971+0000", 2024-09-11T06:49:40.101 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7402999999999999e-05 2024-09-11T06:49:40.101 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.101 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:40.101 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.102 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.102 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.102 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:145 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'103, mlcod=13'98)", 2024-09-11T06:49:40.102 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:49:40.102 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.163926924999998, 2024-09-11T06:49:40.102 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054455409999999999, 2024-09-11T06:49:40.102 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:40.102 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:40.102 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:40.102 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:40.102 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.102 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:40.102 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:49:40.102 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.102 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.103 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.103 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:40.103 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:49:40.103 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.103 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.103 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.103 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:40.103 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838554+0000", 2024-09-11T06:49:40.103 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3430000000000001e-06 2024-09-11T06:49:40.103 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.103 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.103 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:40.103 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838640+0000", 2024-09-11T06:49:40.103 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5569999999999996e-05 2024-09-11T06:49:40.104 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.104 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.104 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:40.104 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838660+0000", 2024-09-11T06:49:40.104 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0576000000000001e-05 2024-09-11T06:49:40.104 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.104 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.104 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:40.104 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838674+0000", 2024-09-11T06:49:40.104 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3529999999999999e-05 2024-09-11T06:49:40.104 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.104 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.104 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:40.104 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838838+0000", 2024-09-11T06:49:40.105 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016468799999999999 2024-09-11T06:49:40.105 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.105 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.105 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:40.105 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838885+0000", 2024-09-11T06:49:40.105 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6884e-05 2024-09-11T06:49:40.105 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.105 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.105 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:40.105 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843901+0000", 2024-09-11T06:49:40.105 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050162310000000003 2024-09-11T06:49:40.105 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.105 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.105 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:40.105 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843998+0000", 2024-09-11T06:49:40.106 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.6718999999999993e-05 2024-09-11T06:49:40.106 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.106 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:40.106 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.106 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.106 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:40.106 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:40.107 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:49:40.108 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:40.106+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:40.208 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:40.206+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:40.278 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:40.278 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:40.278 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:49:40.279 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:40.292 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:49:40.339 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:40.338+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:40.408 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T06:49:40.409 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:40.406+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:40.410 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:40.410 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:40.411 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:49:40.411 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:49:40.411 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:40.425 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:49:40.515 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:40.514+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:40.587 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:40.588 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:49:40.588 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:49:40.588 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:49:40.588 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.588 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:5 2.2 e17/15 2:5cea7035:::rbd_info:head v 17'1, mlcod=0'0)", 2024-09-11T06:49:40.588 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:49:40.588 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 28.352848893000001, 2024-09-11T06:49:40.588 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0019446719999999999, 2024-09-11T06:49:40.588 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:40.588 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:40.588 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:40.588 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:40.588 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.589 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:40.589 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:49:40.589 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.589 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.589 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.589 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:40.589 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:49:40.589 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.589 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.589 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.589 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:40.589 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236085+0000", 2024-09-11T06:49:40.589 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6000000000000003e-07 2024-09-11T06:49:40.589 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.590 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.590 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:40.590 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236108+0000", 2024-09-11T06:49:40.590 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3113e-05 2024-09-11T06:49:40.590 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.590 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.590 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:40.590 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236133+0000", 2024-09-11T06:49:40.590 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5723000000000001e-05 2024-09-11T06:49:40.590 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.590 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.590 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:40.590 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236154+0000", 2024-09-11T06:49:40.590 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0333999999999999e-05 2024-09-11T06:49:40.591 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.591 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.591 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:40.591 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236297+0000", 2024-09-11T06:49:40.591 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014278500000000001 2024-09-11T06:49:40.591 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.591 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.591 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:40.591 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236383+0000", 2024-09-11T06:49:40.591 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5927999999999995e-05 2024-09-11T06:49:40.591 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.591 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.591 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:40.592 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.237950+0000", 2024-09-11T06:49:40.592 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015673690000000001 2024-09-11T06:49:40.592 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.592 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.592 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:40.592 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.238029+0000", 2024-09-11T06:49:40.592 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8659999999999996e-05 2024-09-11T06:49:40.592 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.592 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:40.592 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.592 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.592 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.592 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:7 2.2 e18/15 2:5cea7035:::rbd_info:head v 18'2, mlcod=0'0)", 2024-09-11T06:49:40.592 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:49:40.593 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.338523685999998, 2024-09-11T06:49:40.593 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001907192, 2024-09-11T06:49:40.593 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:40.593 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:40.593 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:40.593 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:40.593 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.593 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:40.593 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:49:40.593 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.593 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.593 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.593 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:40.594 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:49:40.594 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:40.594 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.594 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.594 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:40.594 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250410+0000", 2024-09-11T06:49:40.594 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7700000000000004e-07 2024-09-11T06:49:40.594 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.594 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.594 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:40.594 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250445+0000", 2024-09-11T06:49:40.594 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4749000000000002e-05 2024-09-11T06:49:40.594 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.595 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.595 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:40.595 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250478+0000", 2024-09-11T06:49:40.595 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3269999999999998e-05 2024-09-11T06:49:40.595 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.595 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.595 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:40.595 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250501+0000", 2024-09-11T06:49:40.595 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3632999999999999e-05 2024-09-11T06:49:40.595 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.595 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.595 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:40.595 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250633+0000", 2024-09-11T06:49:40.595 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013143200000000001 2024-09-11T06:49:40.595 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.596 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.596 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:40.596 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250700+0000", 2024-09-11T06:49:40.596 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7002999999999995e-05 2024-09-11T06:49:40.596 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.596 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.596 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:40.596 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.252172+0000", 2024-09-11T06:49:40.596 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014719550000000001 2024-09-11T06:49:40.596 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:40.596 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:40.596 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:40.596 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.252316+0000", 2024-09-11T06:49:40.597 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144473 2024-09-11T06:49:40.597 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.597 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:40.597 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.597 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:40.597 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:40.597 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:40.601 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:49:40.647 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:40.646+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:40.713 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:40.710+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:40.758 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:40.758 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:40.758 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:49:40.758 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:40.771 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:49:40.814 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:40.814+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:40.915 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:40.910+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:40.935 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:40.935 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:40.935 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:49:40.935 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:49:40.935 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:40.947 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:49:41.043 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:41.042+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:41.115 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:41.116 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:49:41.116 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:49:41.116 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:49:41.116 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:41.117 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:1 2.3 e16/15 2:c4c92e5a:::rbd_trash:head v 16'1, mlcod=0'0)", 2024-09-11T06:49:41.117 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:49:41.117 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 29.880696892, 2024-09-11T06:49:41.117 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001234761, 2024-09-11T06:49:41.117 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:41.117 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:41.117 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:41.117 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:41.117 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:41.117 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:41.117 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:49:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:49:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:49:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6000000000000003e-07 2024-09-11T06:49:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236296+0000", 2024-09-11T06:49:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6393999999999999e-05 2024-09-11T06:49:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236317+0000", 2024-09-11T06:49:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1243e-05 2024-09-11T06:49:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236333+0000", 2024-09-11T06:49:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.558e-05 2024-09-11T06:49:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236435+0000", 2024-09-11T06:49:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000102803 2024-09-11T06:49:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236474+0000", 2024-09-11T06:49:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8383999999999997e-05 2024-09-11T06:49:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.237432+0000", 2024-09-11T06:49:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00095849300000000004 2024-09-11T06:49:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:41.121 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:41.121 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:41.121 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.237513+0000", 2024-09-11T06:49:41.121 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1204000000000004e-05 2024-09-11T06:49:41.121 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:41.121 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:41.121 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:41.121 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:41.121 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:41.121 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:41.122 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:41.114+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:41.130 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:49:41.217 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:41.214+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:41.263 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:41.263 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:41.263 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:49:41.263 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:41.278 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:49:41.318 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:41.314+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:41.398 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:41.398 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:41.398 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:49:41.398 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:49:41.398 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:41.401 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is unset 2024-09-11T06:49:41.413 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:49:41.458 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:41.458+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:41.520 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:41.518+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:41.540 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:41.540 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:49:41.540 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:49:41.540 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:49:41.540 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.540 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:41.541 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:49:41.541 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 35.368203493000003, 2024-09-11T06:49:41.541 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011420669, 2024-09-11T06:49:41.541 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:41.541 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:41.541 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:41.541 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:41.541 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:41.541 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:41.541 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 37 2024-09-11T06:49:41.541 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.542 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:41.542 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.542 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:41.542 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:49:41.542 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.542 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.542 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:41.542 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:49:41.542 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.543 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.543 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:41.543 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170509+0000", 2024-09-11T06:49:41.543 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.13e-06 2024-09-11T06:49:41.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.543 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.543 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:41.543 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170521+0000", 2024-09-11T06:49:41.543 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1776999999999999e-05 2024-09-11T06:49:41.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.543 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.543 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:41.543 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170538+0000", 2024-09-11T06:49:41.544 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7413000000000001e-05 2024-09-11T06:49:41.544 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.544 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.544 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:41.544 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170570+0000", 2024-09-11T06:49:41.544 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2313e-05 2024-09-11T06:49:41.544 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.544 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.544 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:41.544 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179768+0000", 2024-09-11T06:49:41.545 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0091978010000000002 2024-09-11T06:49:41.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.545 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.545 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:41.545 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179951+0000", 2024-09-11T06:49:41.545 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018227900000000001 2024-09-11T06:49:41.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.545 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.545 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:41.545 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.180175+0000", 2024-09-11T06:49:41.545 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022432800000000001 2024-09-11T06:49:41.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.546 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.546 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:41.546 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181112+0000", 2024-09-11T06:49:41.546 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00093712500000000002 2024-09-11T06:49:41.546 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.546 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.546 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:41.546 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181789+0000", 2024-09-11T06:49:41.546 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00067705100000000002 2024-09-11T06:49:41.546 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.546 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.546 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:41.547 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181871+0000", 2024-09-11T06:49:41.547 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1816000000000004e-05 2024-09-11T06:49:41.547 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.547 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.547 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:41.547 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181929+0000", 2024-09-11T06:49:41.547 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7636000000000002e-05 2024-09-11T06:49:41.547 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.547 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:41.547 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.547 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.547 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.547 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:41.548 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:49:41.548 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 35.266944531999997, 2024-09-11T06:49:41.548 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283336, 2024-09-11T06:49:41.548 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:41.548 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:41.548 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:41.548 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:41.548 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:41.548 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:41.548 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 87 2024-09-11T06:49:41.548 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.548 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:41.548 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.549 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:41.549 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:49:41.549 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.549 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.549 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.549 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:41.549 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:49:41.549 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.549 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.550 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:41.550 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271768+0000", 2024-09-11T06:49:41.550 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7599999999999996e-07 2024-09-11T06:49:41.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.550 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:41.550 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271860+0000", 2024-09-11T06:49:41.550 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2639999999999994e-05 2024-09-11T06:49:41.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.550 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:41.550 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271873+0000", 2024-09-11T06:49:41.550 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3139000000000001e-05 2024-09-11T06:49:41.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.551 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.551 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:41.551 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271882+0000", 2024-09-11T06:49:41.551 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2369999999999992e-06 2024-09-11T06:49:41.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.551 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.551 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:41.552 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277150+0000", 2024-09-11T06:49:41.552 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052684419999999999 2024-09-11T06:49:41.552 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.552 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.552 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:41.552 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277273+0000", 2024-09-11T06:49:41.553 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012278899999999999 2024-09-11T06:49:41.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.553 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:41.553 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277466+0000", 2024-09-11T06:49:41.553 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193578 2024-09-11T06:49:41.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.554 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.554 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:41.554 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.280808+0000", 2024-09-11T06:49:41.554 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003341971 2024-09-11T06:49:41.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.554 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.554 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:41.554 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284419+0000", 2024-09-11T06:49:41.554 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003610982 2024-09-11T06:49:41.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.555 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.555 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:41.555 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284531+0000", 2024-09-11T06:49:41.555 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011121299999999999 2024-09-11T06:49:41.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.555 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.555 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:41.555 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284600+0000", 2024-09-11T06:49:41.555 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9592999999999996e-05 2024-09-11T06:49:41.555 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.555 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:41.555 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.556 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.556 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:41.556 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:49:41.556 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 35.266814279999998, 2024-09-11T06:49:41.556 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283314, 2024-09-11T06:49:41.556 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:41.556 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:41.556 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:41.556 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:41.556 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:41.556 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:41.556 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 88 2024-09-11T06:49:41.557 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.557 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:41.557 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.557 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:41.557 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:49:41.557 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.557 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.557 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.557 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:41.557 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:49:41.557 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.558 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.558 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:41.558 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271898+0000", 2024-09-11T06:49:41.558 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2999999999999999e-07 2024-09-11T06:49:41.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.558 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.558 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:41.558 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271904+0000", 2024-09-11T06:49:41.558 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.767e-06 2024-09-11T06:49:41.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.558 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.558 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:41.559 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271913+0000", 2024-09-11T06:49:41.559 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7200000000000001e-06 2024-09-11T06:49:41.559 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.559 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.559 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:41.559 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271920+0000", 2024-09-11T06:49:41.559 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7100000000000001e-06 2024-09-11T06:49:41.559 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.559 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.559 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:41.559 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278684+0000", 2024-09-11T06:49:41.559 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0067642650000000002 2024-09-11T06:49:41.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.560 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:41.560 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278838+0000", 2024-09-11T06:49:41.560 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015371499999999999 2024-09-11T06:49:41.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.560 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:41.561 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.279040+0000", 2024-09-11T06:49:41.561 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020207499999999999 2024-09-11T06:49:41.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.561 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.561 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:41.561 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.281448+0000", 2024-09-11T06:49:41.561 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002407686 2024-09-11T06:49:41.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.562 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:41.562 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284615+0000", 2024-09-11T06:49:41.562 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031675499999999999 2024-09-11T06:49:41.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.562 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:41.562 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284682+0000", 2024-09-11T06:49:41.563 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7032000000000005e-05 2024-09-11T06:49:41.563 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.563 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:41.563 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284730+0000", 2024-09-11T06:49:41.563 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7790000000000002e-05 2024-09-11T06:49:41.563 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.563 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:41.563 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.564 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:41.564 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:49:41.564 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 35.098913058000001, 2024-09-11T06:49:41.564 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011114877, 2024-09-11T06:49:41.564 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:41.564 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:41.565 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:41.565 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:41.565 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:41.565 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:41.565 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 107 2024-09-11T06:49:41.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.565 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:41.565 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.566 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:41.566 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:49:41.566 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.566 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.566 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:41.566 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:49:41.566 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.567 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.567 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.567 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:41.567 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439799+0000", 2024-09-11T06:49:41.567 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.16e-07 2024-09-11T06:49:41.567 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.567 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.567 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:41.568 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439806+0000", 2024-09-11T06:49:41.568 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3640000000000004e-06 2024-09-11T06:49:41.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.568 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.568 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:41.568 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439816+0000", 2024-09-11T06:49:41.568 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0485999999999999e-05 2024-09-11T06:49:41.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.569 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:41.569 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439824+0000", 2024-09-11T06:49:41.569 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.6699999999999994e-06 2024-09-11T06:49:41.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.569 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:41.569 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443328+0000", 2024-09-11T06:49:41.570 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035044329999999999 2024-09-11T06:49:41.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.570 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:41.570 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443461+0000", 2024-09-11T06:49:41.571 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013272199999999999 2024-09-11T06:49:41.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.571 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:41.571 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443678+0000", 2024-09-11T06:49:41.571 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021674500000000001 2024-09-11T06:49:41.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.572 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:41.572 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449635+0000", 2024-09-11T06:49:41.572 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0059572119999999999 2024-09-11T06:49:41.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.572 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:41.572 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450750+0000", 2024-09-11T06:49:41.573 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001115654 2024-09-11T06:49:41.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.573 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:41.573 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450832+0000", 2024-09-11T06:49:41.573 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1188999999999998e-05 2024-09-11T06:49:41.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.573 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:41.573 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450913+0000", 2024-09-11T06:49:41.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1485999999999999e-05 2024-09-11T06:49:41.574 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.574 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:41.574 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.574 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:41.574 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:49:41.574 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 35.098322082999999, 2024-09-11T06:49:41.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011236556, 2024-09-11T06:49:41.574 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:41.574 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:41.574 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:41.575 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:41.575 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:41.575 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:41.575 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 109 2024-09-11T06:49:41.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.575 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:41.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.575 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:41.575 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:49:41.575 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.575 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:41.576 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:49:41.576 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.576 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:41.576 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440391+0000", 2024-09-11T06:49:41.576 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8360000000000001e-06 2024-09-11T06:49:41.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.576 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:41.576 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440560+0000", 2024-09-11T06:49:41.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000169085 2024-09-11T06:49:41.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:41.577 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440585+0000", 2024-09-11T06:49:41.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4530000000000001e-05 2024-09-11T06:49:41.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:41.577 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440606+0000", 2024-09-11T06:49:41.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1277e-05 2024-09-11T06:49:41.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.578 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:41.578 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446479+0000", 2024-09-11T06:49:41.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0058734989999999999 2024-09-11T06:49:41.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.578 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:41.578 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446649+0000", 2024-09-11T06:49:41.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016948199999999999 2024-09-11T06:49:41.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.578 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:41.578 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446854+0000", 2024-09-11T06:49:41.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020491500000000001 2024-09-11T06:49:41.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.579 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:41.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451380+0000", 2024-09-11T06:49:41.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0045264110000000001 2024-09-11T06:49:41.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.579 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:41.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451479+0000", 2024-09-11T06:49:41.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.8975999999999996e-05 2024-09-11T06:49:41.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.580 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:41.580 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451562+0000", 2024-09-11T06:49:41.580 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3213000000000004e-05 2024-09-11T06:49:41.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.580 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:41.580 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451626+0000", 2024-09-11T06:49:41.580 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3331999999999996e-05 2024-09-11T06:49:41.580 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.580 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:41.580 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.581 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:41.581 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:49:41.581 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 35.098072954999999, 2024-09-11T06:49:41.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011252866, 2024-09-11T06:49:41.581 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:41.581 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:41.581 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:41.581 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:41.581 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:41.582 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:41.582 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 110 2024-09-11T06:49:41.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.582 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:41.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:41.582 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:49:41.582 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:41.582 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:49:41.582 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:41.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440640+0000", 2024-09-11T06:49:41.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5430000000000001e-06 2024-09-11T06:49:41.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:41.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440651+0000", 2024-09-11T06:49:41.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0672999999999999e-05 2024-09-11T06:49:41.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:41.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440670+0000", 2024-09-11T06:49:41.584 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9726999999999999e-05 2024-09-11T06:49:41.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.584 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:41.584 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440691+0000", 2024-09-11T06:49:41.584 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0526e-05 2024-09-11T06:49:41.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.584 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:41.584 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448191+0000", 2024-09-11T06:49:41.584 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0075005990000000002 2024-09-11T06:49:41.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.585 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:41.585 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448390+0000", 2024-09-11T06:49:41.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000198815 2024-09-11T06:49:41.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.585 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:41.585 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448735+0000", 2024-09-11T06:49:41.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034424999999999999 2024-09-11T06:49:41.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.585 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:41.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451742+0000", 2024-09-11T06:49:41.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030077440000000001 2024-09-11T06:49:41.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:41.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451761+0000", 2024-09-11T06:49:41.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9196999999999999e-05 2024-09-11T06:49:41.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:41.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451834+0000", 2024-09-11T06:49:41.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2071999999999997e-05 2024-09-11T06:49:41.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.587 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:41.587 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451891+0000", 2024-09-11T06:49:41.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7720000000000003e-05 2024-09-11T06:49:41.587 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.587 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:41.587 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.587 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:41.587 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:49:41.587 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 35.097965080000002, 2024-09-11T06:49:41.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014944941, 2024-09-11T06:49:41.588 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:41.588 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:41.588 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:41.588 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:41.588 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:41.588 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:41.588 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 111 2024-09-11T06:49:41.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.588 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:41.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.588 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:41.589 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:49:41.589 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.589 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:41.589 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:49:41.589 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.589 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:41.589 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440748+0000", 2024-09-11T06:49:41.589 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:49:41.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:41.590 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440760+0000", 2024-09-11T06:49:41.590 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1897e-05 2024-09-11T06:49:41.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:41.590 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440782+0000", 2024-09-11T06:49:41.590 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2552999999999999e-05 2024-09-11T06:49:41.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:41.591 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440793+0000", 2024-09-11T06:49:41.591 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0643e-05 2024-09-11T06:49:41.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.591 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:41.591 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449755+0000", 2024-09-11T06:49:41.591 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089622620000000004 2024-09-11T06:49:41.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.591 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:41.591 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449918+0000", 2024-09-11T06:49:41.591 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163086 2024-09-11T06:49:41.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.592 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:41.592 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450150+0000", 2024-09-11T06:49:41.592 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023204500000000001 2024-09-11T06:49:41.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.592 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:41.592 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454533+0000", 2024-09-11T06:49:41.592 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0043831349999999998 2024-09-11T06:49:41.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.592 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:41.593 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455511+0000", 2024-09-11T06:49:41.593 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0009776050000000001 2024-09-11T06:49:41.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.593 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:41.593 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455597+0000", 2024-09-11T06:49:41.593 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6446000000000005e-05 2024-09-11T06:49:41.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.593 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:41.593 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455691+0000", 2024-09-11T06:49:41.593 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3832000000000006e-05 2024-09-11T06:49:41.593 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.594 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:41.594 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.594 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:41.594 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:49:41.594 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 35.097895317000003, 2024-09-11T06:49:41.594 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018885411000000001, 2024-09-11T06:49:41.594 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:41.594 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:41.594 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:41.594 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:41.594 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:41.595 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:41.595 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 112 2024-09-11T06:49:41.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.595 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:41.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.595 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:41.595 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:49:41.595 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.595 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:41.595 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:49:41.595 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.596 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:41.596 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440817+0000", 2024-09-11T06:49:41.596 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:49:41.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.596 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:41.596 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441003+0000", 2024-09-11T06:49:41.596 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018661200000000001 2024-09-11T06:49:41.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.596 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:41.596 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441027+0000", 2024-09-11T06:49:41.597 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3920000000000001e-05 2024-09-11T06:49:41.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.597 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:41.597 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441062+0000", 2024-09-11T06:49:41.597 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4663000000000003e-05 2024-09-11T06:49:41.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.597 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:41.597 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451981+0000", 2024-09-11T06:49:41.597 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010918926000000001 2024-09-11T06:49:41.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.598 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:41.598 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452111+0000", 2024-09-11T06:49:41.598 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129529 2024-09-11T06:49:41.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.598 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:41.598 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452311+0000", 2024-09-11T06:49:41.598 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020013799999999999 2024-09-11T06:49:41.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.598 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:41.598 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455809+0000", 2024-09-11T06:49:41.598 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003498443 2024-09-11T06:49:41.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.599 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:41.599 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459566+0000", 2024-09-11T06:49:41.599 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003757271 2024-09-11T06:49:41.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.599 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:41.599 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459640+0000", 2024-09-11T06:49:41.599 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3949000000000006e-05 2024-09-11T06:49:41.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.599 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:41.600 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459701+0000", 2024-09-11T06:49:41.600 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1117000000000005e-05 2024-09-11T06:49:41.600 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.600 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:41.600 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.600 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:41.600 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:49:41.600 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 35.097616465999998, 2024-09-11T06:49:41.600 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018760915, 2024-09-11T06:49:41.600 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:41.600 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:41.601 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:41.601 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:41.601 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:41.601 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:41.601 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 113 2024-09-11T06:49:41.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.601 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:41.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.601 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:41.601 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:49:41.601 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.602 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:41.602 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:49:41.602 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.602 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:41.602 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441096+0000", 2024-09-11T06:49:41.602 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4130000000000001e-06 2024-09-11T06:49:41.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.602 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:41.602 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441110+0000", 2024-09-11T06:49:41.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3900000000000001e-05 2024-09-11T06:49:41.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.603 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:41.603 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441131+0000", 2024-09-11T06:49:41.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0636999999999999e-05 2024-09-11T06:49:41.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.603 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:41.603 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441147+0000", 2024-09-11T06:49:41.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6512999999999999e-05 2024-09-11T06:49:41.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.604 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:41.604 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453274+0000", 2024-09-11T06:49:41.604 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012126764999999999 2024-09-11T06:49:41.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.604 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:41.604 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453498+0000", 2024-09-11T06:49:41.604 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000223568 2024-09-11T06:49:41.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.605 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:41.605 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453872+0000", 2024-09-11T06:49:41.605 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037467099999999998 2024-09-11T06:49:41.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.605 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:41.605 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458709+0000", 2024-09-11T06:49:41.605 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0048365680000000003 2024-09-11T06:49:41.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.605 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:41.605 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459716+0000", 2024-09-11T06:49:41.606 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010075240000000001 2024-09-11T06:49:41.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.606 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.606 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:41.606 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459779+0000", 2024-09-11T06:49:41.606 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2336999999999996e-05 2024-09-11T06:49:41.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.606 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.606 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:41.606 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459856+0000", 2024-09-11T06:49:41.606 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7019000000000002e-05 2024-09-11T06:49:41.606 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.606 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:41.606 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.607 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:41.607 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:41.607 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 35.097539726999997, 2024-09-11T06:49:41.607 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018963114, 2024-09-11T06:49:41.607 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:41.607 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:41.607 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:41.607 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:41.607 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:41.607 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:41.607 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 114 2024-09-11T06:49:41.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.608 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:41.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.608 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:41.608 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:41.608 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.608 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:41.608 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:41.608 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.609 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:41.609 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:41.609 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9299999999999996e-07 2024-09-11T06:49:41.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.609 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:41.609 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441180+0000", 2024-09-11T06:49:41.609 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7200000000000006e-06 2024-09-11T06:49:41.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.609 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:41.609 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441265+0000", 2024-09-11T06:49:41.610 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4753000000000007e-05 2024-09-11T06:49:41.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.610 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:41.610 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441288+0000", 2024-09-11T06:49:41.610 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2860000000000001e-05 2024-09-11T06:49:41.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.611 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:41.611 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454585+0000", 2024-09-11T06:49:41.611 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013297286 2024-09-11T06:49:41.611 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.611 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.611 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:41.612 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454714+0000", 2024-09-11T06:49:41.612 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000128465 2024-09-11T06:49:41.612 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.612 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.612 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:41.613 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454909+0000", 2024-09-11T06:49:41.613 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019522499999999999 2024-09-11T06:49:41.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.613 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.613 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:41.613 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459868+0000", 2024-09-11T06:49:41.614 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0049591239999999997 2024-09-11T06:49:41.614 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.614 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.614 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:41.614 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459988+0000", 2024-09-11T06:49:41.614 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000120532 2024-09-11T06:49:41.615 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.615 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.615 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:41.615 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460071+0000", 2024-09-11T06:49:41.615 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2256000000000007e-05 2024-09-11T06:49:41.615 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.616 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.616 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:41.616 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460135+0000", 2024-09-11T06:49:41.616 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3999999999999997e-05 2024-09-11T06:49:41.616 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.617 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:41.617 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.617 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.617 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.617 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:41.618 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:49:41.618 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 35.097382097999997, 2024-09-11T06:49:41.618 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.019036849000000002, 2024-09-11T06:49:41.618 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:41.618 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:41.618 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:41.618 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:41.619 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:41.619 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:41.619 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 115 2024-09-11T06:49:41.619 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.619 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:41.620 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.620 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:41.620 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:49:41.620 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.620 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.620 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.621 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:41.621 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:49:41.621 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.621 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.622 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:41.622 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441330+0000", 2024-09-11T06:49:41.622 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1930000000000001e-06 2024-09-11T06:49:41.622 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.622 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.622 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:41.623 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441544+0000", 2024-09-11T06:49:41.623 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021405799999999999 2024-09-11T06:49:41.623 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.623 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.623 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:41.624 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441572+0000", 2024-09-11T06:49:41.624 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7317e-05 2024-09-11T06:49:41.624 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.624 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.624 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:41.624 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441593+0000", 2024-09-11T06:49:41.625 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1702999999999999e-05 2024-09-11T06:49:41.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.625 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:41.625 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455891+0000", 2024-09-11T06:49:41.626 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01429708 2024-09-11T06:49:41.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.626 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.626 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:41.626 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456046+0000", 2024-09-11T06:49:41.626 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000155682 2024-09-11T06:49:41.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.627 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:41.627 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456242+0000", 2024-09-11T06:49:41.627 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000195692 2024-09-11T06:49:41.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.628 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.628 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:41.628 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459876+0000", 2024-09-11T06:49:41.628 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036336480000000002 2024-09-11T06:49:41.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.628 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.628 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:41.628 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460245+0000", 2024-09-11T06:49:41.628 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036901700000000002 2024-09-11T06:49:41.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.628 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.629 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:41.629 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460316+0000", 2024-09-11T06:49:41.629 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1846000000000003e-05 2024-09-11T06:49:41.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.629 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.629 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:41.629 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460366+0000", 2024-09-11T06:49:41.629 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9613000000000001e-05 2024-09-11T06:49:41.629 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.629 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:41.629 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.630 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.630 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:41.630 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:49:41.630 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 35.097060018000001, 2024-09-11T06:49:41.630 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01904432, 2024-09-11T06:49:41.630 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:41.630 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:41.630 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:41.630 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:41.630 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:41.630 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:41.630 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 116 2024-09-11T06:49:41.630 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.630 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:41.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.631 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:41.631 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:49:41.631 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.631 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:41.631 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:49:41.631 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.632 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:41.632 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441653+0000", 2024-09-11T06:49:41.632 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:49:41.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.632 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:41.632 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441665+0000", 2024-09-11T06:49:41.632 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1653e-05 2024-09-11T06:49:41.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.633 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:41.633 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441684+0000", 2024-09-11T06:49:41.633 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9490000000000001e-05 2024-09-11T06:49:41.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.633 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:41.633 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441699+0000", 2024-09-11T06:49:41.633 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5044e-05 2024-09-11T06:49:41.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.634 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.634 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:41.634 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457297+0000", 2024-09-11T06:49:41.634 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015598241000000001 2024-09-11T06:49:41.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.634 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.634 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:41.634 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457577+0000", 2024-09-11T06:49:41.634 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027930799999999998 2024-09-11T06:49:41.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.635 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.635 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:41.635 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457957+0000", 2024-09-11T06:49:41.635 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00038088 2024-09-11T06:49:41.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.635 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.635 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:41.635 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460432+0000", 2024-09-11T06:49:41.635 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024743349999999998 2024-09-11T06:49:41.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.635 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.635 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:41.635 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460584+0000", 2024-09-11T06:49:41.635 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015232900000000001 2024-09-11T06:49:41.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.636 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.636 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:41.636 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460646+0000", 2024-09-11T06:49:41.636 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1883000000000004e-05 2024-09-11T06:49:41.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.636 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.636 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:41.636 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460696+0000", 2024-09-11T06:49:41.636 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9589999999999998e-05 2024-09-11T06:49:41.637 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.637 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:41.637 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.638 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:41.638 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:49:41.638 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 35.096987951999999, 2024-09-11T06:49:41.638 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.021834473, 2024-09-11T06:49:41.638 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:41.638 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:41.638 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:41.638 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:41.638 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:41.638 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:41.638 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 117 2024-09-11T06:49:41.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.639 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:41.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.639 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:41.639 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:49:41.639 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.639 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:41.639 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:49:41.639 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.639 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:41.639 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441725+0000", 2024-09-11T06:49:41.640 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5069999999999999e-06 2024-09-11T06:49:41.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.640 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.640 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:41.640 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441738+0000", 2024-09-11T06:49:41.640 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2813000000000001e-05 2024-09-11T06:49:41.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.640 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.640 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:41.640 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441759+0000", 2024-09-11T06:49:41.640 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1246999999999998e-05 2024-09-11T06:49:41.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.640 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.640 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:41.641 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441779+0000", 2024-09-11T06:49:41.641 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0392999999999999e-05 2024-09-11T06:49:41.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.641 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:41.641 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458804+0000", 2024-09-11T06:49:41.641 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.017024182999999998 2024-09-11T06:49:41.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.641 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:41.641 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458977+0000", 2024-09-11T06:49:41.641 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000173926 2024-09-11T06:49:41.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.642 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:41.642 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459170+0000", 2024-09-11T06:49:41.642 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000192184 2024-09-11T06:49:41.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.642 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:41.642 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.462300+0000", 2024-09-11T06:49:41.642 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031304969999999999 2024-09-11T06:49:41.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.643 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:41.643 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463408+0000", 2024-09-11T06:49:41.643 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011077839999999999 2024-09-11T06:49:41.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.643 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:41.643 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463497+0000", 2024-09-11T06:49:41.643 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8828999999999999e-05 2024-09-11T06:49:41.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.643 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:41.643 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463558+0000", 2024-09-11T06:49:41.644 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1110000000000003e-05 2024-09-11T06:49:41.644 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.644 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:41.644 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.644 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:41.644 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:49:41.644 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 34.720480264000003, 2024-09-11T06:49:41.644 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012216898, 2024-09-11T06:49:41.644 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:41.644 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:41.644 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:41.644 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:41.645 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:41.645 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:41.645 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 137 2024-09-11T06:49:41.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.645 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:41.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:41.645 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:49:41.645 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:41.645 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:49:41.646 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.646 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:41.646 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818232+0000", 2024-09-11T06:49:41.646 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2839999999999999e-06 2024-09-11T06:49:41.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.646 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:41.646 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818326+0000", 2024-09-11T06:49:41.646 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3531999999999998e-05 2024-09-11T06:49:41.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.646 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:41.646 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818337+0000", 2024-09-11T06:49:41.647 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0655999999999999e-05 2024-09-11T06:49:41.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.647 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:41.647 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818342+0000", 2024-09-11T06:49:41.647 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4600000000000002e-06 2024-09-11T06:49:41.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.647 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:41.647 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823890+0000", 2024-09-11T06:49:41.647 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0055480700000000004 2024-09-11T06:49:41.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.647 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:41.647 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823953+0000", 2024-09-11T06:49:41.648 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2848999999999994e-05 2024-09-11T06:49:41.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.648 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:41.648 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824074+0000", 2024-09-11T06:49:41.648 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121309 2024-09-11T06:49:41.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.648 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:41.648 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828043+0000", 2024-09-11T06:49:41.648 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0039685240000000002 2024-09-11T06:49:41.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.649 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:41.649 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830293+0000", 2024-09-11T06:49:41.649 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0022503179999999998 2024-09-11T06:49:41.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.649 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:41.649 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830366+0000", 2024-09-11T06:49:41.649 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2539999999999993e-05 2024-09-11T06:49:41.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.649 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:41.649 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830448+0000", 2024-09-11T06:49:41.649 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2355999999999996e-05 2024-09-11T06:49:41.649 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.650 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:41.650 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.650 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:41.650 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:41.650 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 34.720346450000001, 2024-09-11T06:49:41.650 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013136297, 2024-09-11T06:49:41.650 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:41.650 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:41.650 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:41.650 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:41.651 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:41.651 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:41.651 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 138 2024-09-11T06:49:41.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.651 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:41.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.651 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:41.651 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:41.651 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.651 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:41.651 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:41.652 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.652 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:41.652 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:41.652 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.89e-07 2024-09-11T06:49:41.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.652 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:41.652 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818369+0000", 2024-09-11T06:49:41.652 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4170000000000001e-06 2024-09-11T06:49:41.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.653 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:41.653 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818376+0000", 2024-09-11T06:49:41.653 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4270000000000002e-06 2024-09-11T06:49:41.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.653 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:41.653 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818383+0000", 2024-09-11T06:49:41.653 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3060000000000003e-06 2024-09-11T06:49:41.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.653 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:41.653 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824783+0000", 2024-09-11T06:49:41.654 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064003020000000001 2024-09-11T06:49:41.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.654 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:41.654 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825013+0000", 2024-09-11T06:49:41.654 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023039499999999999 2024-09-11T06:49:41.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.654 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:41.654 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825484+0000", 2024-09-11T06:49:41.654 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000470562 2024-09-11T06:49:41.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.655 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:41.655 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828633+0000", 2024-09-11T06:49:41.655 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031496689999999999 2024-09-11T06:49:41.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.655 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:41.655 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831413+0000", 2024-09-11T06:49:41.655 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027793219999999999 2024-09-11T06:49:41.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.656 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:41.656 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831450+0000", 2024-09-11T06:49:41.656 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7370999999999998e-05 2024-09-11T06:49:41.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.656 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:41.656 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831501+0000", 2024-09-11T06:49:41.656 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1036999999999999e-05 2024-09-11T06:49:41.656 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.656 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:41.656 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.656 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:41.657 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:49:41.657 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 34.720321970000001, 2024-09-11T06:49:41.657 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013223011, 2024-09-11T06:49:41.657 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:41.657 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:41.657 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:41.657 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:41.657 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:41.657 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:41.657 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 139 2024-09-11T06:49:41.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.657 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:41.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.658 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:41.658 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:49:41.658 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.658 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:41.658 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:49:41.658 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.658 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:41.658 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818390+0000", 2024-09-11T06:49:41.658 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5600000000000001e-07 2024-09-11T06:49:41.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.659 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:41.659 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818394+0000", 2024-09-11T06:49:41.659 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5929999999999999e-06 2024-09-11T06:49:41.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.659 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:41.659 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818399+0000", 2024-09-11T06:49:41.659 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7740000000000004e-06 2024-09-11T06:49:41.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.659 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:41.659 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818412+0000", 2024-09-11T06:49:41.659 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2625e-05 2024-09-11T06:49:41.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.660 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:41.660 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826205+0000", 2024-09-11T06:49:41.660 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0077930730000000002 2024-09-11T06:49:41.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.660 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:41.660 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826281+0000", 2024-09-11T06:49:41.660 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5876000000000003e-05 2024-09-11T06:49:41.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.660 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:41.661 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826384+0000", 2024-09-11T06:49:41.661 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103838 2024-09-11T06:49:41.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.661 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.661 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:41.661 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830642+0000", 2024-09-11T06:49:41.661 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004257083 2024-09-11T06:49:41.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.661 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.661 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:41.661 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831509+0000", 2024-09-11T06:49:41.661 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086730900000000003 2024-09-11T06:49:41.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.661 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.662 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:41.662 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831547+0000", 2024-09-11T06:49:41.662 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7793000000000002e-05 2024-09-11T06:49:41.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.662 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.662 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:41.662 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831612+0000", 2024-09-11T06:49:41.662 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5691000000000005e-05 2024-09-11T06:49:41.662 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.662 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:41.662 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.662 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.663 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:41.663 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:41.663 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 34.720293276, 2024-09-11T06:49:41.663 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016098024999999998, 2024-09-11T06:49:41.663 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:41.663 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:41.663 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:41.663 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:41.663 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:41.663 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:41.663 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 140 2024-09-11T06:49:41.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.663 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:41.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.664 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:41.664 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:41.664 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.664 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.664 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:41.664 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:41.664 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.664 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.664 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:41.664 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:41.665 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5199999999999998e-07 2024-09-11T06:49:41.665 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.665 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.665 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:41.665 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818479+0000", 2024-09-11T06:49:41.665 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0368000000000002e-05 2024-09-11T06:49:41.665 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.665 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.665 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:41.665 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818487+0000", 2024-09-11T06:49:41.665 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2339999999999994e-06 2024-09-11T06:49:41.665 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.665 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.666 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:41.666 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818491+0000", 2024-09-11T06:49:41.666 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4329999999999997e-06 2024-09-11T06:49:41.666 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.666 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.666 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:41.666 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828080+0000", 2024-09-11T06:49:41.666 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0095890449999999992 2024-09-11T06:49:41.666 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.666 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.666 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:41.666 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828130+0000", 2024-09-11T06:49:41.666 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9239000000000001e-05 2024-09-11T06:49:41.666 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.667 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.667 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:41.667 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828208+0000", 2024-09-11T06:49:41.667 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.852e-05 2024-09-11T06:49:41.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.667 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.667 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:41.667 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831625+0000", 2024-09-11T06:49:41.667 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034169410000000002 2024-09-11T06:49:41.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.667 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.667 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:41.667 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834411+0000", 2024-09-11T06:49:41.668 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002786111 2024-09-11T06:49:41.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.668 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:41.668 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834457+0000", 2024-09-11T06:49:41.668 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6041000000000002e-05 2024-09-11T06:49:41.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.668 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:41.668 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834516+0000", 2024-09-11T06:49:41.668 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8740999999999999e-05 2024-09-11T06:49:41.668 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.668 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:41.668 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.669 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.669 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:41.669 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:49:41.669 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 34.720212209000003, 2024-09-11T06:49:41.669 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016588772000000002, 2024-09-11T06:49:41.669 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:41.669 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:41.669 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:41.669 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:41.669 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:41.669 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:41.669 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 141 2024-09-11T06:49:41.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.670 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:41.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.670 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:41.670 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:49:41.670 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.670 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:41.670 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:49:41.670 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.671 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:41.671 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818500+0000", 2024-09-11T06:49:41.671 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0800000000000005e-07 2024-09-11T06:49:41.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.671 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.671 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:41.671 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818505+0000", 2024-09-11T06:49:41.671 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2429999999999996e-06 2024-09-11T06:49:41.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.671 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.671 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:41.671 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818510+0000", 2024-09-11T06:49:41.671 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9339999999999997e-06 2024-09-11T06:49:41.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.672 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.672 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:41.672 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818514+0000", 2024-09-11T06:49:41.672 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7270000000000001e-06 2024-09-11T06:49:41.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.672 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.672 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:41.672 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828711+0000", 2024-09-11T06:49:41.672 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010197031 2024-09-11T06:49:41.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.673 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:41.673 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828853+0000", 2024-09-11T06:49:41.673 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014201200000000001 2024-09-11T06:49:41.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.673 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:41.673 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.829115+0000", 2024-09-11T06:49:41.673 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026257500000000003 2024-09-11T06:49:41.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.673 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:41.673 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831629+0000", 2024-09-11T06:49:41.673 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025138169999999998 2024-09-11T06:49:41.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.674 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.674 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:41.674 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834991+0000", 2024-09-11T06:49:41.674 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033624459999999998 2024-09-11T06:49:41.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.674 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.674 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:41.674 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835032+0000", 2024-09-11T06:49:41.674 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1001000000000003e-05 2024-09-11T06:49:41.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.674 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.674 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:41.674 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835088+0000", 2024-09-11T06:49:41.674 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5478e-05 2024-09-11T06:49:41.675 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.675 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:41.675 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.675 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.675 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:49:41.675 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:49:41.675 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 34.096577306, 2024-09-11T06:49:41.675 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011161912, 2024-09-11T06:49:41.675 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:41.675 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:41.675 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:41.675 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:41.675 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:41.675 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:41.676 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 158 2024-09-11T06:49:41.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.676 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:41.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.676 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:41.676 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:49:41.676 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.676 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:41.676 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:49:41.676 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.677 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:41.677 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442135+0000", 2024-09-11T06:49:41.677 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3e-07 2024-09-11T06:49:41.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.677 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:41.677 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442259+0000", 2024-09-11T06:49:41.677 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000123903 2024-09-11T06:49:41.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.677 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:41.677 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442289+0000", 2024-09-11T06:49:41.677 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0833000000000001e-05 2024-09-11T06:49:41.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.678 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.678 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:41.678 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442301+0000", 2024-09-11T06:49:41.678 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1433e-05 2024-09-11T06:49:41.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.678 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.678 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:41.678 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449796+0000", 2024-09-11T06:49:41.678 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0074954959999999999 2024-09-11T06:49:41.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.678 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.678 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:41.678 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449917+0000", 2024-09-11T06:49:41.678 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121033 2024-09-11T06:49:41.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.679 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:41.679 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.450166+0000", 2024-09-11T06:49:41.679 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00024832400000000002 2024-09-11T06:49:41.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.679 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:41.679 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.452630+0000", 2024-09-11T06:49:41.679 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002464188 2024-09-11T06:49:41.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.679 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:41.679 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453149+0000", 2024-09-11T06:49:41.680 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00051860999999999997 2024-09-11T06:49:41.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.680 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:41.680 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453234+0000", 2024-09-11T06:49:41.680 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.5865000000000005e-05 2024-09-11T06:49:41.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.680 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:41.680 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453296+0000", 2024-09-11T06:49:41.680 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1496999999999998e-05 2024-09-11T06:49:41.680 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.680 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:41.681 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.681 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:49:41.681 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:49:41.681 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 34.096391204, 2024-09-11T06:49:41.681 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011561900999999999, 2024-09-11T06:49:41.681 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:41.681 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:41.681 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:41.681 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:41.681 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:41.681 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:41.682 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 159 2024-09-11T06:49:41.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.682 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:41.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.682 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:41.682 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:49:41.682 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.682 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:41.682 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:49:41.682 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:41.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.682 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:41.683 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442321+0000", 2024-09-11T06:49:41.683 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8400000000000003e-07 2024-09-11T06:49:41.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.683 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.683 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:41.683 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442328+0000", 2024-09-11T06:49:41.683 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5130000000000002e-06 2024-09-11T06:49:41.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.683 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.683 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:41.683 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442341+0000", 2024-09-11T06:49:41.683 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.363e-05 2024-09-11T06:49:41.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.683 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.684 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:41.684 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442349+0000", 2024-09-11T06:49:41.684 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4760000000000001e-06 2024-09-11T06:49:41.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.684 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:41.684 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451257+0000", 2024-09-11T06:49:41.684 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089088569999999992 2024-09-11T06:49:41.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.684 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:41.684 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451389+0000", 2024-09-11T06:49:41.684 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013110899999999999 2024-09-11T06:49:41.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.685 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:41.685 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451608+0000", 2024-09-11T06:49:41.685 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000219588 2024-09-11T06:49:41.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.685 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:41.685 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453517+0000", 2024-09-11T06:49:41.685 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0019088779999999999 2024-09-11T06:49:41.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.685 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:41.685 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453727+0000", 2024-09-11T06:49:41.686 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020955100000000001 2024-09-11T06:49:41.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.686 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:41.686 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453815+0000", 2024-09-11T06:49:41.686 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8858999999999997e-05 2024-09-11T06:49:41.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:41.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:41.686 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:41.686 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453882+0000", 2024-09-11T06:49:41.686 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.6556000000000004e-05 2024-09-11T06:49:41.686 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.686 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:41.687 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.687 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:41.687 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:41.687 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:41.689 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:49:41.689 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:41.686+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:41.790 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:41.786+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:41.826 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:49:41.861 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:41.861 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:41.861 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:49:41.861 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:41.877 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:49:41.919 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:41.918+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:41.978 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:49:41.978 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:49:41.991 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:41.990+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:41.994 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:49:41.994 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:49:42.011 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:49:42.012 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:49:42.032 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:49:42.032 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:49:42.045 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:42.045 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:42.045 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:49:42.046 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:49:42.046 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:42.053 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:49:42.053 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:49:42.060 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:49:42.072 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:49:42.072 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:49:42.102 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:42.102+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:42.193 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:42.190+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:42.225 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:42.225 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:42.239 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:49:42.294 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:42.294+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:42.357 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:42.358 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:42.373 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:49:42.394 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:42.390+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:42.465 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T06:49:42.465 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T06:49:42.465 INFO:tasks.thrashosds.thrasher:Reweighting osd 4 to 0.7026704585515361 2024-09-11T06:49:42.466 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 0.7026704585515361 2024-09-11T06:49:42.496 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:42.494+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:42.538 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:42.539 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:42.552 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:49:42.596 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:42.594+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:42.697 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:42.694+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:42.717 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:42.717 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:42.731 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:49:42.799 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:42.798+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:42.850 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:42.850 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:42.864 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:49:42.911 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:42.910+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:42.987 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:42.987 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:43.001 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:49:43.002 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:42.998+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:43.102 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:43.098+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:43.165 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:43.166 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:43.179 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:49:43.223 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:43.222+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:43.304 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:43.302+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:43.343 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:43.343 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:43.357 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:49:43.397 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.4 to 0.70267 (b3e2) 2024-09-11T06:49:43.405 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:43.402+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:43.426 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T06:49:43.485 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:43.486 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:43.500 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:49:43.547 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:43.546+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:43.607 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:43.606+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:43.673 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:43.674 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:43.688 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:49:43.708 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:43.710+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:43.809 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:43.809 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:43.809 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:43.806+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:43.823 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:49:43.910 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:43.906+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:43.944 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:43.944 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:43.958 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:49:44.010 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:44.010+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:44.090 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:49:44.111 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:44.110+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:44.121 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:44.121 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:44.136 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:49:44.211 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:44.210+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:44.232 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:49:44.232 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:49:44.247 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:49:44.247 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:49:44.263 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:49:44.263 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:49:44.268 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:44.269 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:44.282 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:49:44.283 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:49:44.284 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:49:44.300 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:49:44.300 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:49:44.318 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:49:44.319 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:49:44.326 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:44.326+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:44.410 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is set 2024-09-11T06:49:44.413 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:44.410+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:44.452 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:44.452 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:44.452 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:49:44.452 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:49:44.452 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:44.466 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:49:44.550 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:44.550+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:44.615 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:44.614+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:44.631 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:44.631 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:49:44.631 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:49:44.631 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:49:44.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.632 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:44.632 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:49:44.632 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 38.459362579, 2024-09-11T06:49:44.632 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011420669, 2024-09-11T06:49:44.632 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:44.632 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:44.632 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:44.632 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:44.632 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:44.632 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:44.632 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 37 2024-09-11T06:49:44.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.633 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:44.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.633 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:44.633 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:49:44.633 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.633 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:44.633 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:49:44.633 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.633 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:44.633 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170509+0000", 2024-09-11T06:49:44.634 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.13e-06 2024-09-11T06:49:44.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.634 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.634 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:44.634 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170521+0000", 2024-09-11T06:49:44.634 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1776999999999999e-05 2024-09-11T06:49:44.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.634 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.634 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:44.634 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170538+0000", 2024-09-11T06:49:44.634 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7413000000000001e-05 2024-09-11T06:49:44.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.634 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.634 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:44.635 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170570+0000", 2024-09-11T06:49:44.635 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2313e-05 2024-09-11T06:49:44.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.635 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.635 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:44.635 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179768+0000", 2024-09-11T06:49:44.635 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0091978010000000002 2024-09-11T06:49:44.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.635 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.635 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:44.635 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179951+0000", 2024-09-11T06:49:44.635 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018227900000000001 2024-09-11T06:49:44.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.635 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.636 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:44.636 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.180175+0000", 2024-09-11T06:49:44.636 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022432800000000001 2024-09-11T06:49:44.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.636 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.636 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:44.636 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181112+0000", 2024-09-11T06:49:44.636 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00093712500000000002 2024-09-11T06:49:44.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.636 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.636 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:44.636 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181789+0000", 2024-09-11T06:49:44.636 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00067705100000000002 2024-09-11T06:49:44.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.637 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:44.637 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181871+0000", 2024-09-11T06:49:44.637 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1816000000000004e-05 2024-09-11T06:49:44.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.637 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:44.637 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181929+0000", 2024-09-11T06:49:44.637 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7636000000000002e-05 2024-09-11T06:49:44.637 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.637 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:44.637 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.638 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:44.638 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:49:44.638 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 38.358103618000001, 2024-09-11T06:49:44.638 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283336, 2024-09-11T06:49:44.638 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:44.638 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:44.638 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:44.638 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:44.638 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:44.638 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:44.638 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 87 2024-09-11T06:49:44.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.639 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:44.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.639 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:44.639 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:49:44.639 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.639 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:44.639 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:49:44.639 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.639 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:44.639 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271768+0000", 2024-09-11T06:49:44.639 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7599999999999996e-07 2024-09-11T06:49:44.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.640 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.640 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:44.640 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271860+0000", 2024-09-11T06:49:44.640 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2639999999999994e-05 2024-09-11T06:49:44.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.640 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.640 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:44.640 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271873+0000", 2024-09-11T06:49:44.640 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3139000000000001e-05 2024-09-11T06:49:44.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.640 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.640 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:44.641 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271882+0000", 2024-09-11T06:49:44.641 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2369999999999992e-06 2024-09-11T06:49:44.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.641 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:44.641 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277150+0000", 2024-09-11T06:49:44.641 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052684419999999999 2024-09-11T06:49:44.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.641 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:44.641 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277273+0000", 2024-09-11T06:49:44.641 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012278899999999999 2024-09-11T06:49:44.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.642 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:44.642 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277466+0000", 2024-09-11T06:49:44.642 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193578 2024-09-11T06:49:44.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.642 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:44.642 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.280808+0000", 2024-09-11T06:49:44.642 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003341971 2024-09-11T06:49:44.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.642 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:44.642 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284419+0000", 2024-09-11T06:49:44.643 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003610982 2024-09-11T06:49:44.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.643 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:44.643 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284531+0000", 2024-09-11T06:49:44.643 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011121299999999999 2024-09-11T06:49:44.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.643 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:44.643 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284600+0000", 2024-09-11T06:49:44.643 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9592999999999996e-05 2024-09-11T06:49:44.643 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.643 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:44.643 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.644 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:44.644 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:49:44.644 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 38.357973366000003, 2024-09-11T06:49:44.644 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283314, 2024-09-11T06:49:44.644 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:44.644 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:44.644 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:44.644 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:44.644 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:44.644 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:44.644 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 88 2024-09-11T06:49:44.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.645 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:44.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:44.645 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:49:44.645 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:44.645 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:49:44.645 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:44.646 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271898+0000", 2024-09-11T06:49:44.646 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2999999999999999e-07 2024-09-11T06:49:44.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.646 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:44.646 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271904+0000", 2024-09-11T06:49:44.646 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.767e-06 2024-09-11T06:49:44.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.646 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:44.646 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271913+0000", 2024-09-11T06:49:44.646 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7200000000000001e-06 2024-09-11T06:49:44.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.647 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:44.647 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271920+0000", 2024-09-11T06:49:44.647 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7100000000000001e-06 2024-09-11T06:49:44.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.647 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:44.647 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278684+0000", 2024-09-11T06:49:44.647 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0067642650000000002 2024-09-11T06:49:44.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.647 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:44.647 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278838+0000", 2024-09-11T06:49:44.647 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015371499999999999 2024-09-11T06:49:44.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.648 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:44.648 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.279040+0000", 2024-09-11T06:49:44.648 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020207499999999999 2024-09-11T06:49:44.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.648 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:44.648 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.281448+0000", 2024-09-11T06:49:44.648 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002407686 2024-09-11T06:49:44.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.648 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:44.648 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284615+0000", 2024-09-11T06:49:44.649 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031675499999999999 2024-09-11T06:49:44.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.649 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:44.649 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284682+0000", 2024-09-11T06:49:44.649 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7032000000000005e-05 2024-09-11T06:49:44.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.649 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:44.649 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284730+0000", 2024-09-11T06:49:44.649 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7790000000000002e-05 2024-09-11T06:49:44.649 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.649 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:44.649 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.650 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:44.650 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:49:44.650 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 38.190072143999998, 2024-09-11T06:49:44.650 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011114877, 2024-09-11T06:49:44.650 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:44.650 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:44.650 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:44.650 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:44.650 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:44.650 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:44.650 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 107 2024-09-11T06:49:44.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.651 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:44.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.651 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:44.651 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:49:44.651 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.651 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:44.651 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:49:44.651 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.651 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:44.652 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439799+0000", 2024-09-11T06:49:44.652 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.16e-07 2024-09-11T06:49:44.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.652 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:44.652 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439806+0000", 2024-09-11T06:49:44.652 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3640000000000004e-06 2024-09-11T06:49:44.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.652 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:44.652 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439816+0000", 2024-09-11T06:49:44.652 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0485999999999999e-05 2024-09-11T06:49:44.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.653 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:44.653 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439824+0000", 2024-09-11T06:49:44.653 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.6699999999999994e-06 2024-09-11T06:49:44.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.653 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:44.653 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443328+0000", 2024-09-11T06:49:44.653 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035044329999999999 2024-09-11T06:49:44.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.653 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:44.653 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443461+0000", 2024-09-11T06:49:44.654 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013272199999999999 2024-09-11T06:49:44.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.654 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:44.654 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443678+0000", 2024-09-11T06:49:44.654 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021674500000000001 2024-09-11T06:49:44.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.654 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:44.654 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449635+0000", 2024-09-11T06:49:44.654 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0059572119999999999 2024-09-11T06:49:44.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.654 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:44.655 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450750+0000", 2024-09-11T06:49:44.655 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001115654 2024-09-11T06:49:44.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.655 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:44.655 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450832+0000", 2024-09-11T06:49:44.655 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1188999999999998e-05 2024-09-11T06:49:44.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.655 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:44.655 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450913+0000", 2024-09-11T06:49:44.655 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1485999999999999e-05 2024-09-11T06:49:44.655 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.655 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:44.656 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.656 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:44.656 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:49:44.656 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 38.189481168999997, 2024-09-11T06:49:44.656 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011236556, 2024-09-11T06:49:44.656 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:44.656 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:44.656 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:44.656 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:44.656 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:44.656 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:44.656 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 109 2024-09-11T06:49:44.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.657 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:44.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.657 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:44.657 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:49:44.657 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.657 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:44.657 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:49:44.657 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.657 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:44.658 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440391+0000", 2024-09-11T06:49:44.658 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8360000000000001e-06 2024-09-11T06:49:44.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.658 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:44.658 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440560+0000", 2024-09-11T06:49:44.658 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000169085 2024-09-11T06:49:44.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.658 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:44.658 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440585+0000", 2024-09-11T06:49:44.658 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4530000000000001e-05 2024-09-11T06:49:44.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.659 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:44.659 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440606+0000", 2024-09-11T06:49:44.659 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1277e-05 2024-09-11T06:49:44.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.659 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:44.659 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446479+0000", 2024-09-11T06:49:44.659 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0058734989999999999 2024-09-11T06:49:44.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.659 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:44.659 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446649+0000", 2024-09-11T06:49:44.659 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016948199999999999 2024-09-11T06:49:44.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.660 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:44.660 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446854+0000", 2024-09-11T06:49:44.660 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020491500000000001 2024-09-11T06:49:44.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.660 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:44.660 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451380+0000", 2024-09-11T06:49:44.660 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0045264110000000001 2024-09-11T06:49:44.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.660 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:44.660 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451479+0000", 2024-09-11T06:49:44.661 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.8975999999999996e-05 2024-09-11T06:49:44.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.661 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.661 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:44.661 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451562+0000", 2024-09-11T06:49:44.661 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3213000000000004e-05 2024-09-11T06:49:44.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.661 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.661 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:44.661 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451626+0000", 2024-09-11T06:49:44.661 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3331999999999996e-05 2024-09-11T06:49:44.661 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.661 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:44.662 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.662 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.662 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:44.662 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:49:44.662 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 38.189232040999997, 2024-09-11T06:49:44.662 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011252866, 2024-09-11T06:49:44.662 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:44.662 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:44.662 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:44.662 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:44.662 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:44.662 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:44.662 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 110 2024-09-11T06:49:44.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.663 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:44.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.663 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:44.663 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:49:44.663 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.663 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:44.663 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:49:44.663 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.664 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:44.664 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440640+0000", 2024-09-11T06:49:44.664 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5430000000000001e-06 2024-09-11T06:49:44.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.664 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.664 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:44.664 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440651+0000", 2024-09-11T06:49:44.664 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0672999999999999e-05 2024-09-11T06:49:44.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.664 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.664 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:44.664 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440670+0000", 2024-09-11T06:49:44.664 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9726999999999999e-05 2024-09-11T06:49:44.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.665 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.665 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:44.665 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440691+0000", 2024-09-11T06:49:44.665 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0526e-05 2024-09-11T06:49:44.665 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.665 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.665 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:44.665 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448191+0000", 2024-09-11T06:49:44.665 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0075005990000000002 2024-09-11T06:49:44.665 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.665 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.665 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:44.665 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448390+0000", 2024-09-11T06:49:44.665 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000198815 2024-09-11T06:49:44.666 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.666 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.666 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:44.666 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448735+0000", 2024-09-11T06:49:44.666 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034424999999999999 2024-09-11T06:49:44.666 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.666 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.666 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:44.666 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451742+0000", 2024-09-11T06:49:44.666 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030077440000000001 2024-09-11T06:49:44.666 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.666 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.666 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:44.667 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451761+0000", 2024-09-11T06:49:44.667 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9196999999999999e-05 2024-09-11T06:49:44.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.667 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.667 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:44.667 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451834+0000", 2024-09-11T06:49:44.667 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2071999999999997e-05 2024-09-11T06:49:44.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.667 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.667 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:44.667 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451891+0000", 2024-09-11T06:49:44.667 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7720000000000003e-05 2024-09-11T06:49:44.667 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.667 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:44.668 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.668 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:44.668 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:49:44.668 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 38.189124165999999, 2024-09-11T06:49:44.668 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014944941, 2024-09-11T06:49:44.668 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:44.668 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:44.668 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:44.668 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:44.668 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:44.668 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:44.669 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 111 2024-09-11T06:49:44.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.669 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:44.669 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.669 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:44.669 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:49:44.669 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.669 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.669 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:44.669 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:49:44.669 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.669 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.670 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:44.670 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440748+0000", 2024-09-11T06:49:44.670 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:49:44.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.670 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:44.670 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440760+0000", 2024-09-11T06:49:44.670 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1897e-05 2024-09-11T06:49:44.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.670 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:44.670 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440782+0000", 2024-09-11T06:49:44.670 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2552999999999999e-05 2024-09-11T06:49:44.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.671 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.671 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:44.671 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440793+0000", 2024-09-11T06:49:44.671 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0643e-05 2024-09-11T06:49:44.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.671 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.671 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:44.671 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449755+0000", 2024-09-11T06:49:44.671 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089622620000000004 2024-09-11T06:49:44.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.671 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.671 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:44.672 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449918+0000", 2024-09-11T06:49:44.672 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163086 2024-09-11T06:49:44.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.672 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.672 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:44.672 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450150+0000", 2024-09-11T06:49:44.672 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023204500000000001 2024-09-11T06:49:44.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.672 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.672 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:44.672 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454533+0000", 2024-09-11T06:49:44.672 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0043831349999999998 2024-09-11T06:49:44.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.673 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:44.673 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455511+0000", 2024-09-11T06:49:44.673 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0009776050000000001 2024-09-11T06:49:44.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.673 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:44.673 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455597+0000", 2024-09-11T06:49:44.673 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6446000000000005e-05 2024-09-11T06:49:44.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.673 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:44.673 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455691+0000", 2024-09-11T06:49:44.673 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3832000000000006e-05 2024-09-11T06:49:44.674 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.674 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:44.674 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.674 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.674 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:44.674 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:49:44.674 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 38.189054403, 2024-09-11T06:49:44.674 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018885411000000001, 2024-09-11T06:49:44.674 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:44.674 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:44.674 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:44.674 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:44.675 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:44.675 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:44.675 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 112 2024-09-11T06:49:44.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.675 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:44.675 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.675 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:44.675 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:49:44.675 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.675 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.675 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:44.675 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:49:44.675 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.676 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:44.676 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440817+0000", 2024-09-11T06:49:44.676 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:49:44.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.676 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:44.676 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441003+0000", 2024-09-11T06:49:44.676 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018661200000000001 2024-09-11T06:49:44.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.676 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:44.676 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441027+0000", 2024-09-11T06:49:44.677 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3920000000000001e-05 2024-09-11T06:49:44.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.677 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:44.677 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441062+0000", 2024-09-11T06:49:44.677 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4663000000000003e-05 2024-09-11T06:49:44.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.677 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:44.677 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451981+0000", 2024-09-11T06:49:44.677 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010918926000000001 2024-09-11T06:49:44.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.677 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:44.678 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452111+0000", 2024-09-11T06:49:44.678 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129529 2024-09-11T06:49:44.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.678 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.678 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:44.678 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452311+0000", 2024-09-11T06:49:44.678 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020013799999999999 2024-09-11T06:49:44.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.678 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.678 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:44.678 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455809+0000", 2024-09-11T06:49:44.678 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003498443 2024-09-11T06:49:44.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.678 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.678 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:44.679 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459566+0000", 2024-09-11T06:49:44.679 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003757271 2024-09-11T06:49:44.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.679 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:44.679 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459640+0000", 2024-09-11T06:49:44.679 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3949000000000006e-05 2024-09-11T06:49:44.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.679 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:44.679 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459701+0000", 2024-09-11T06:49:44.679 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1117000000000005e-05 2024-09-11T06:49:44.679 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.680 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:44.680 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.680 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:44.680 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:49:44.680 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 38.188775552000003, 2024-09-11T06:49:44.680 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018760915, 2024-09-11T06:49:44.680 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:44.680 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:44.680 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:44.680 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:44.680 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:44.681 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:44.681 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 113 2024-09-11T06:49:44.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.681 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:44.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.681 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:44.681 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:49:44.681 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.681 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:44.681 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:49:44.681 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.682 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:44.682 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441096+0000", 2024-09-11T06:49:44.682 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4130000000000001e-06 2024-09-11T06:49:44.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.682 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:44.682 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441110+0000", 2024-09-11T06:49:44.682 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3900000000000001e-05 2024-09-11T06:49:44.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.682 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:44.682 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441131+0000", 2024-09-11T06:49:44.683 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0636999999999999e-05 2024-09-11T06:49:44.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.683 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.683 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:44.683 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441147+0000", 2024-09-11T06:49:44.683 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6512999999999999e-05 2024-09-11T06:49:44.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.683 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.683 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:44.683 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453274+0000", 2024-09-11T06:49:44.683 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012126764999999999 2024-09-11T06:49:44.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.683 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.683 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:44.684 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453498+0000", 2024-09-11T06:49:44.684 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000223568 2024-09-11T06:49:44.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.684 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:44.684 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453872+0000", 2024-09-11T06:49:44.684 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037467099999999998 2024-09-11T06:49:44.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.684 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:44.684 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458709+0000", 2024-09-11T06:49:44.684 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0048365680000000003 2024-09-11T06:49:44.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.685 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:44.685 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459716+0000", 2024-09-11T06:49:44.685 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010075240000000001 2024-09-11T06:49:44.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.685 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:44.685 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459779+0000", 2024-09-11T06:49:44.685 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2336999999999996e-05 2024-09-11T06:49:44.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.685 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:44.685 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459856+0000", 2024-09-11T06:49:44.685 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7019000000000002e-05 2024-09-11T06:49:44.686 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.686 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:44.686 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.686 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:44.686 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:44.686 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 38.188698813000002, 2024-09-11T06:49:44.686 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018963114, 2024-09-11T06:49:44.686 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:44.686 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:44.686 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:44.686 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:44.686 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:44.687 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:44.687 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 114 2024-09-11T06:49:44.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.687 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:44.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.687 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:44.687 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:44.687 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.687 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:44.687 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:44.687 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.688 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:44.688 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:44.688 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9299999999999996e-07 2024-09-11T06:49:44.688 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.688 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:44.688 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441180+0000", 2024-09-11T06:49:44.688 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7200000000000006e-06 2024-09-11T06:49:44.688 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.688 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:44.688 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441265+0000", 2024-09-11T06:49:44.689 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4753000000000007e-05 2024-09-11T06:49:44.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.689 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:44.689 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441288+0000", 2024-09-11T06:49:44.689 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2860000000000001e-05 2024-09-11T06:49:44.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.689 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:44.689 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454585+0000", 2024-09-11T06:49:44.689 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013297286 2024-09-11T06:49:44.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.690 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:44.690 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454714+0000", 2024-09-11T06:49:44.690 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000128465 2024-09-11T06:49:44.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.690 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:44.690 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454909+0000", 2024-09-11T06:49:44.690 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019522499999999999 2024-09-11T06:49:44.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.690 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:44.690 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459868+0000", 2024-09-11T06:49:44.690 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0049591239999999997 2024-09-11T06:49:44.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.691 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:44.691 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459988+0000", 2024-09-11T06:49:44.691 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000120532 2024-09-11T06:49:44.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.691 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:44.691 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460071+0000", 2024-09-11T06:49:44.691 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2256000000000007e-05 2024-09-11T06:49:44.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.691 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:44.691 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460135+0000", 2024-09-11T06:49:44.691 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3999999999999997e-05 2024-09-11T06:49:44.692 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.692 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:44.692 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.692 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:44.692 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:49:44.692 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 38.188541184000002, 2024-09-11T06:49:44.692 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.019036849000000002, 2024-09-11T06:49:44.692 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:44.692 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:44.692 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:44.692 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:44.692 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:44.693 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:44.693 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 115 2024-09-11T06:49:44.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.693 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:44.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.693 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:44.693 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:49:44.693 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.693 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:44.693 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:49:44.693 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.694 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:44.694 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441330+0000", 2024-09-11T06:49:44.694 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1930000000000001e-06 2024-09-11T06:49:44.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.694 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:44.694 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441544+0000", 2024-09-11T06:49:44.694 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021405799999999999 2024-09-11T06:49:44.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.694 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:44.694 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441572+0000", 2024-09-11T06:49:44.694 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7317e-05 2024-09-11T06:49:44.695 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.695 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.695 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:44.695 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441593+0000", 2024-09-11T06:49:44.695 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1702999999999999e-05 2024-09-11T06:49:44.695 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.695 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.695 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:44.695 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455891+0000", 2024-09-11T06:49:44.695 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01429708 2024-09-11T06:49:44.695 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.695 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.695 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:44.695 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456046+0000", 2024-09-11T06:49:44.696 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000155682 2024-09-11T06:49:44.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.696 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.696 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:44.696 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456242+0000", 2024-09-11T06:49:44.696 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000195692 2024-09-11T06:49:44.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.696 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.696 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:44.696 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459876+0000", 2024-09-11T06:49:44.696 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036336480000000002 2024-09-11T06:49:44.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.696 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.696 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:44.697 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460245+0000", 2024-09-11T06:49:44.697 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036901700000000002 2024-09-11T06:49:44.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.697 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.697 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:44.697 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460316+0000", 2024-09-11T06:49:44.697 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1846000000000003e-05 2024-09-11T06:49:44.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.697 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.697 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:44.697 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460366+0000", 2024-09-11T06:49:44.697 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9613000000000001e-05 2024-09-11T06:49:44.697 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.697 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:44.698 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.698 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.698 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:44.698 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:49:44.698 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 38.188219103999998, 2024-09-11T06:49:44.698 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01904432, 2024-09-11T06:49:44.698 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:44.698 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:44.698 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:44.698 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:44.698 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:44.698 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:44.698 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 116 2024-09-11T06:49:44.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.699 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:44.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.699 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:44.699 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:49:44.699 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.699 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:44.699 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:49:44.699 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.699 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:44.700 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441653+0000", 2024-09-11T06:49:44.700 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:49:44.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.700 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.700 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:44.700 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441665+0000", 2024-09-11T06:49:44.700 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1653e-05 2024-09-11T06:49:44.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.700 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.700 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:44.700 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441684+0000", 2024-09-11T06:49:44.700 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9490000000000001e-05 2024-09-11T06:49:44.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.700 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.701 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:44.701 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441699+0000", 2024-09-11T06:49:44.701 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5044e-05 2024-09-11T06:49:44.701 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.701 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.701 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:44.701 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457297+0000", 2024-09-11T06:49:44.701 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015598241000000001 2024-09-11T06:49:44.701 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.701 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.701 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:44.701 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457577+0000", 2024-09-11T06:49:44.701 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027930799999999998 2024-09-11T06:49:44.701 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.702 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.702 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:44.702 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457957+0000", 2024-09-11T06:49:44.702 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00038088 2024-09-11T06:49:44.702 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.702 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.702 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:44.702 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460432+0000", 2024-09-11T06:49:44.702 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024743349999999998 2024-09-11T06:49:44.702 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.702 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.702 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:44.702 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460584+0000", 2024-09-11T06:49:44.702 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015232900000000001 2024-09-11T06:49:44.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.703 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.703 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:44.703 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460646+0000", 2024-09-11T06:49:44.703 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1883000000000004e-05 2024-09-11T06:49:44.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.703 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.703 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:44.703 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460696+0000", 2024-09-11T06:49:44.703 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9589999999999998e-05 2024-09-11T06:49:44.703 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.703 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:44.703 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.704 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.704 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:44.704 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:49:44.704 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 38.188147037999997, 2024-09-11T06:49:44.704 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.021834473, 2024-09-11T06:49:44.704 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:44.704 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:44.704 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:44.704 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:44.704 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:44.704 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:44.704 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 117 2024-09-11T06:49:44.704 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.704 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:44.705 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.705 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:44.705 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:49:44.705 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.705 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.705 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:44.705 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:49:44.705 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.705 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.705 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:44.705 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441725+0000", 2024-09-11T06:49:44.705 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5069999999999999e-06 2024-09-11T06:49:44.706 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.706 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.706 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:44.706 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441738+0000", 2024-09-11T06:49:44.706 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2813000000000001e-05 2024-09-11T06:49:44.706 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.706 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.706 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:44.706 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441759+0000", 2024-09-11T06:49:44.706 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1246999999999998e-05 2024-09-11T06:49:44.706 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.706 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.706 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:44.706 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441779+0000", 2024-09-11T06:49:44.707 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0392999999999999e-05 2024-09-11T06:49:44.707 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.707 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.707 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:44.707 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458804+0000", 2024-09-11T06:49:44.707 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.017024182999999998 2024-09-11T06:49:44.707 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.707 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.707 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:44.707 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458977+0000", 2024-09-11T06:49:44.707 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000173926 2024-09-11T06:49:44.707 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.707 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.707 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:44.708 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459170+0000", 2024-09-11T06:49:44.708 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000192184 2024-09-11T06:49:44.708 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.708 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.708 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:44.708 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.462300+0000", 2024-09-11T06:49:44.708 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031304969999999999 2024-09-11T06:49:44.708 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.708 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.708 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:44.708 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463408+0000", 2024-09-11T06:49:44.708 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011077839999999999 2024-09-11T06:49:44.708 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.708 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.709 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:44.709 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463497+0000", 2024-09-11T06:49:44.709 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8828999999999999e-05 2024-09-11T06:49:44.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.709 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.709 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:44.709 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463558+0000", 2024-09-11T06:49:44.709 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1110000000000003e-05 2024-09-11T06:49:44.709 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.709 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:44.709 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.709 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.709 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:44.710 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:49:44.710 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 37.81163935, 2024-09-11T06:49:44.710 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012216898, 2024-09-11T06:49:44.710 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:44.710 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:44.710 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:44.710 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:44.710 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:44.710 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:44.710 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 137 2024-09-11T06:49:44.710 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.710 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:44.710 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.710 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:44.711 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:49:44.711 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.711 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.711 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.711 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:44.711 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:49:44.711 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.711 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.711 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.711 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:44.711 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818232+0000", 2024-09-11T06:49:44.711 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2839999999999999e-06 2024-09-11T06:49:44.711 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.711 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.712 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:44.712 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818326+0000", 2024-09-11T06:49:44.712 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3531999999999998e-05 2024-09-11T06:49:44.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.712 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.712 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:44.712 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818337+0000", 2024-09-11T06:49:44.712 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0655999999999999e-05 2024-09-11T06:49:44.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.712 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.712 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:44.712 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818342+0000", 2024-09-11T06:49:44.712 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4600000000000002e-06 2024-09-11T06:49:44.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.713 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.713 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:44.713 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823890+0000", 2024-09-11T06:49:44.713 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0055480700000000004 2024-09-11T06:49:44.713 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.713 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.713 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:44.713 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823953+0000", 2024-09-11T06:49:44.713 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2848999999999994e-05 2024-09-11T06:49:44.713 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.713 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.713 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:44.713 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824074+0000", 2024-09-11T06:49:44.713 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121309 2024-09-11T06:49:44.714 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.714 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.714 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:44.714 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828043+0000", 2024-09-11T06:49:44.714 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0039685240000000002 2024-09-11T06:49:44.714 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.714 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.714 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:44.714 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830293+0000", 2024-09-11T06:49:44.714 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0022503179999999998 2024-09-11T06:49:44.714 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.714 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.714 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:44.714 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830366+0000", 2024-09-11T06:49:44.715 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2539999999999993e-05 2024-09-11T06:49:44.715 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.715 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.715 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:44.715 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830448+0000", 2024-09-11T06:49:44.715 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2355999999999996e-05 2024-09-11T06:49:44.715 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.715 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:44.715 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.715 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.715 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.715 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:44.715 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:44.715 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 37.811505535999999, 2024-09-11T06:49:44.716 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013136297, 2024-09-11T06:49:44.716 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:44.716 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:44.716 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:44.716 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:44.716 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:44.716 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:44.716 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 138 2024-09-11T06:49:44.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.716 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:44.716 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.716 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:44.716 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:44.716 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.717 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.717 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:44.717 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:44.717 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.717 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.717 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:44.717 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:44.717 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.89e-07 2024-09-11T06:49:44.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.717 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.717 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:44.717 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818369+0000", 2024-09-11T06:49:44.718 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4170000000000001e-06 2024-09-11T06:49:44.718 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.718 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.718 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:44.718 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818376+0000", 2024-09-11T06:49:44.718 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4270000000000002e-06 2024-09-11T06:49:44.718 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.718 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.718 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:44.718 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818383+0000", 2024-09-11T06:49:44.718 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3060000000000003e-06 2024-09-11T06:49:44.718 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.718 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.718 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:44.719 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824783+0000", 2024-09-11T06:49:44.719 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064003020000000001 2024-09-11T06:49:44.719 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.719 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.719 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:44.719 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825013+0000", 2024-09-11T06:49:44.719 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023039499999999999 2024-09-11T06:49:44.719 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.719 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.719 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:44.719 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825484+0000", 2024-09-11T06:49:44.719 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000470562 2024-09-11T06:49:44.719 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.720 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.720 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:44.720 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828633+0000", 2024-09-11T06:49:44.720 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031496689999999999 2024-09-11T06:49:44.720 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.720 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.720 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:44.720 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831413+0000", 2024-09-11T06:49:44.720 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027793219999999999 2024-09-11T06:49:44.720 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.720 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.720 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:44.720 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831450+0000", 2024-09-11T06:49:44.720 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7370999999999998e-05 2024-09-11T06:49:44.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.721 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.721 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:44.721 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831501+0000", 2024-09-11T06:49:44.721 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1036999999999999e-05 2024-09-11T06:49:44.721 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.721 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:44.721 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.721 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.721 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:44.721 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:49:44.721 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 37.811481055999998, 2024-09-11T06:49:44.721 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013223011, 2024-09-11T06:49:44.722 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:44.722 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:44.722 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:44.722 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:44.722 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:44.722 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:44.722 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 139 2024-09-11T06:49:44.722 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.722 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:44.722 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.722 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:44.722 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:49:44.722 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.722 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.722 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.723 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:44.723 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:49:44.723 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.723 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.723 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.723 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:44.723 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818390+0000", 2024-09-11T06:49:44.723 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5600000000000001e-07 2024-09-11T06:49:44.723 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.723 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.723 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:44.723 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818394+0000", 2024-09-11T06:49:44.723 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5929999999999999e-06 2024-09-11T06:49:44.723 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.724 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.724 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:44.724 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818399+0000", 2024-09-11T06:49:44.724 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7740000000000004e-06 2024-09-11T06:49:44.724 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.724 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.724 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:44.724 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818412+0000", 2024-09-11T06:49:44.724 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2625e-05 2024-09-11T06:49:44.724 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.724 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.724 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:44.724 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826205+0000", 2024-09-11T06:49:44.724 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0077930730000000002 2024-09-11T06:49:44.725 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.725 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.725 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:44.725 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826281+0000", 2024-09-11T06:49:44.725 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5876000000000003e-05 2024-09-11T06:49:44.725 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.725 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.725 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:44.725 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826384+0000", 2024-09-11T06:49:44.725 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103838 2024-09-11T06:49:44.725 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.725 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.725 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:44.725 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830642+0000", 2024-09-11T06:49:44.725 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004257083 2024-09-11T06:49:44.726 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.726 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.726 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:44.726 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831509+0000", 2024-09-11T06:49:44.726 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086730900000000003 2024-09-11T06:49:44.726 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.726 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.726 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:44.726 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831547+0000", 2024-09-11T06:49:44.726 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7793000000000002e-05 2024-09-11T06:49:44.726 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.726 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.726 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:44.726 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831612+0000", 2024-09-11T06:49:44.727 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5691000000000005e-05 2024-09-11T06:49:44.727 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.727 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:44.727 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.727 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.727 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.727 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:44.727 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:44.727 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 37.811452361999997, 2024-09-11T06:49:44.727 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016098024999999998, 2024-09-11T06:49:44.727 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:44.727 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:44.727 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:44.727 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:44.728 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:44.728 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:44.728 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 140 2024-09-11T06:49:44.728 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.728 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:44.728 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.728 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:44.728 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:44.728 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.728 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.728 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.728 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:44.728 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:44.728 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.729 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.729 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.729 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:44.729 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:44.729 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5199999999999998e-07 2024-09-11T06:49:44.729 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.729 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.729 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:44.729 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818479+0000", 2024-09-11T06:49:44.729 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0368000000000002e-05 2024-09-11T06:49:44.729 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.729 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.729 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:44.729 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818487+0000", 2024-09-11T06:49:44.730 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2339999999999994e-06 2024-09-11T06:49:44.730 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.730 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.730 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:44.730 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818491+0000", 2024-09-11T06:49:44.730 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4329999999999997e-06 2024-09-11T06:49:44.730 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.730 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.730 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:44.730 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828080+0000", 2024-09-11T06:49:44.730 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0095890449999999992 2024-09-11T06:49:44.730 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.730 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.730 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:44.731 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828130+0000", 2024-09-11T06:49:44.731 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9239000000000001e-05 2024-09-11T06:49:44.731 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.731 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.731 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:44.731 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828208+0000", 2024-09-11T06:49:44.731 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.852e-05 2024-09-11T06:49:44.731 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.731 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.731 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:44.731 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831625+0000", 2024-09-11T06:49:44.731 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034169410000000002 2024-09-11T06:49:44.731 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.731 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.732 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:44.732 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834411+0000", 2024-09-11T06:49:44.732 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002786111 2024-09-11T06:49:44.732 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.732 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.732 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:44.732 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834457+0000", 2024-09-11T06:49:44.732 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6041000000000002e-05 2024-09-11T06:49:44.732 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.732 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.732 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:44.732 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834516+0000", 2024-09-11T06:49:44.732 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8740999999999999e-05 2024-09-11T06:49:44.733 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.733 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:44.733 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.733 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.733 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.733 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:44.733 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:49:44.733 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 37.811371295000001, 2024-09-11T06:49:44.733 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016588772000000002, 2024-09-11T06:49:44.733 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:44.733 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:44.733 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:44.733 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:44.733 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:44.734 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:44.734 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 141 2024-09-11T06:49:44.734 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.734 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:44.734 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.734 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:44.734 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:49:44.734 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.734 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.734 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.734 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:44.734 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:49:44.734 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.735 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.735 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.735 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:44.735 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818500+0000", 2024-09-11T06:49:44.735 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0800000000000005e-07 2024-09-11T06:49:44.735 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.735 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.735 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:44.735 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818505+0000", 2024-09-11T06:49:44.735 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2429999999999996e-06 2024-09-11T06:49:44.735 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.735 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.735 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:44.735 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818510+0000", 2024-09-11T06:49:44.736 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9339999999999997e-06 2024-09-11T06:49:44.736 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.736 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.736 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:44.736 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818514+0000", 2024-09-11T06:49:44.736 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7270000000000001e-06 2024-09-11T06:49:44.736 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.736 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.736 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:44.736 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828711+0000", 2024-09-11T06:49:44.736 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010197031 2024-09-11T06:49:44.736 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.736 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.736 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:44.737 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828853+0000", 2024-09-11T06:49:44.737 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014201200000000001 2024-09-11T06:49:44.737 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.737 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.737 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:44.737 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.829115+0000", 2024-09-11T06:49:44.737 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026257500000000003 2024-09-11T06:49:44.737 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.737 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.737 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:44.737 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831629+0000", 2024-09-11T06:49:44.737 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025138169999999998 2024-09-11T06:49:44.737 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.738 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.738 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:44.738 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834991+0000", 2024-09-11T06:49:44.738 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033624459999999998 2024-09-11T06:49:44.738 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.738 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.738 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:44.738 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835032+0000", 2024-09-11T06:49:44.738 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1001000000000003e-05 2024-09-11T06:49:44.738 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.738 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.738 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:44.738 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835088+0000", 2024-09-11T06:49:44.738 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5478e-05 2024-09-11T06:49:44.739 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.739 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:44.739 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.739 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.739 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:49:44.739 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:49:44.739 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 37.187736391999998, 2024-09-11T06:49:44.739 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011161912, 2024-09-11T06:49:44.739 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:44.739 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:44.739 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:44.739 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:44.740 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:44.740 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:44.740 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 158 2024-09-11T06:49:44.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.740 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:44.740 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.740 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:44.740 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:49:44.740 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.740 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.740 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:44.740 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:49:44.740 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.741 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.741 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.741 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:44.741 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442135+0000", 2024-09-11T06:49:44.741 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3e-07 2024-09-11T06:49:44.741 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.741 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.741 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:44.741 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442259+0000", 2024-09-11T06:49:44.741 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000123903 2024-09-11T06:49:44.741 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.741 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.741 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:44.741 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442289+0000", 2024-09-11T06:49:44.742 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0833000000000001e-05 2024-09-11T06:49:44.742 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.742 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.742 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:44.742 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442301+0000", 2024-09-11T06:49:44.742 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1433e-05 2024-09-11T06:49:44.742 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.742 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.742 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:44.742 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449796+0000", 2024-09-11T06:49:44.742 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0074954959999999999 2024-09-11T06:49:44.742 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.742 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.743 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:44.743 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449917+0000", 2024-09-11T06:49:44.743 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121033 2024-09-11T06:49:44.743 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.743 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.743 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:44.743 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.450166+0000", 2024-09-11T06:49:44.743 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00024832400000000002 2024-09-11T06:49:44.743 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.743 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.743 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:44.743 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.452630+0000", 2024-09-11T06:49:44.743 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002464188 2024-09-11T06:49:44.743 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.744 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.744 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:44.744 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453149+0000", 2024-09-11T06:49:44.744 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00051860999999999997 2024-09-11T06:49:44.744 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.744 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.744 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:44.744 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453234+0000", 2024-09-11T06:49:44.744 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.5865000000000005e-05 2024-09-11T06:49:44.744 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.744 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.744 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:44.744 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453296+0000", 2024-09-11T06:49:44.744 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1496999999999998e-05 2024-09-11T06:49:44.745 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.745 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:44.745 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.745 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.745 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.745 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:49:44.745 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:49:44.745 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 37.187550289999997, 2024-09-11T06:49:44.745 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011561900999999999, 2024-09-11T06:49:44.745 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:44.745 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:44.745 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:44.745 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:44.746 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:44.746 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:44.746 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 159 2024-09-11T06:49:44.746 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.746 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:44.746 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.746 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:44.746 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:49:44.746 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.746 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.746 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.746 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:44.746 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:49:44.746 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:44.747 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.747 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.747 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:44.747 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442321+0000", 2024-09-11T06:49:44.747 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8400000000000003e-07 2024-09-11T06:49:44.747 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.747 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.747 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:44.747 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442328+0000", 2024-09-11T06:49:44.747 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5130000000000002e-06 2024-09-11T06:49:44.747 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.747 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.747 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:44.747 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442341+0000", 2024-09-11T06:49:44.748 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.363e-05 2024-09-11T06:49:44.748 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.748 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.748 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:44.748 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442349+0000", 2024-09-11T06:49:44.748 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4760000000000001e-06 2024-09-11T06:49:44.748 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.748 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.748 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:44.748 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451257+0000", 2024-09-11T06:49:44.748 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089088569999999992 2024-09-11T06:49:44.748 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.748 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.749 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:44.749 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451389+0000", 2024-09-11T06:49:44.749 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013110899999999999 2024-09-11T06:49:44.749 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.749 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.749 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:44.749 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451608+0000", 2024-09-11T06:49:44.749 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000219588 2024-09-11T06:49:44.749 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.749 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.749 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:44.749 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453517+0000", 2024-09-11T06:49:44.749 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0019088779999999999 2024-09-11T06:49:44.749 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.750 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.750 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:44.750 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453727+0000", 2024-09-11T06:49:44.750 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020955100000000001 2024-09-11T06:49:44.750 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.750 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.750 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:44.750 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453815+0000", 2024-09-11T06:49:44.750 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8858999999999997e-05 2024-09-11T06:49:44.750 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:44.750 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:44.750 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:44.750 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453882+0000", 2024-09-11T06:49:44.750 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.6556000000000004e-05 2024-09-11T06:49:44.751 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.751 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:44.751 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.751 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:44.751 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:44.751 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:44.752 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:49:44.753 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:44.750+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:44.853 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:44.850+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:44.923 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:44.923 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:44.924 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:49:44.924 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:44.938 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:49:44.978 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:44.978+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:45.055 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:45.054+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:45.098 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:45.098 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:45.098 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:49:45.098 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:49:45.098 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:45.113 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:49:45.157 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:45.154+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:45.233 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:45.233 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:49:45.233 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:49:45.233 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [] 2024-09-11T06:49:45.233 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:45.247 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:49:45.294 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:45.294+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:45.359 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:45.358+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:45.364 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:45.364 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:45.364 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:49:45.364 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:45.378 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:49:45.460 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:45.458+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:45.545 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:45.545 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:45.545 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:49:45.545 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:49:45.545 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:45.559 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:49:45.561 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:45.558+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:45.662 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:45.662+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:45.678 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:45.678 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:49:45.678 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:49:45.679 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:49:45.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.679 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:49:45.679 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:49:45.679 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 34.449461587000002, 2024-09-11T06:49:45.679 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010553425999999999, 2024-09-11T06:49:45.679 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:45.679 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:45.679 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:45.679 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:45.679 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:49:45.679 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:49:45.680 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:49:45.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.680 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:45.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.680 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:45.680 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:49:45.680 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:45.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.680 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:45.680 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:49:45.680 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:45.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.681 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:45.681 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230274+0000", 2024-09-11T06:49:45.681 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.533e-06 2024-09-11T06:49:45.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.681 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:45.681 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230285+0000", 2024-09-11T06:49:45.681 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0540000000000001e-05 2024-09-11T06:49:45.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.681 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:45.681 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230309+0000", 2024-09-11T06:49:45.682 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4223000000000002e-05 2024-09-11T06:49:45.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.682 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:45.682 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230327+0000", 2024-09-11T06:49:45.682 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:49:45.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.682 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:45.682 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230478+0000", 2024-09-11T06:49:45.682 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000151459 2024-09-11T06:49:45.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.683 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:49:45.683 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230490+0000", 2024-09-11T06:49:45.683 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.147e-05 2024-09-11T06:49:45.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.683 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.683 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:45.683 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231803+0000", 2024-09-11T06:49:45.683 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001313158 2024-09-11T06:49:45.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.683 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.683 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:49:45.683 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231817+0000", 2024-09-11T06:49:45.683 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4197e-05 2024-09-11T06:49:45.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.684 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:45.684 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.235945+0000", 2024-09-11T06:49:45.684 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0041282480000000002 2024-09-11T06:49:45.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.684 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:45.684 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236312+0000", 2024-09-11T06:49:45.684 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036652700000000001 2024-09-11T06:49:45.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.684 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:49:45.684 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236689+0000", 2024-09-11T06:49:45.685 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000377117 2024-09-11T06:49:45.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.685 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:45.685 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238460+0000", 2024-09-11T06:49:45.685 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001770918 2024-09-11T06:49:45.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.685 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:45.685 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240627+0000", 2024-09-11T06:49:45.685 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021670779999999998 2024-09-11T06:49:45.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.685 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:45.686 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240751+0000", 2024-09-11T06:49:45.686 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012384900000000001 2024-09-11T06:49:45.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.686 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:45.686 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240821+0000", 2024-09-11T06:49:45.686 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0516e-05 2024-09-11T06:49:45.686 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:45.686 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:45.686 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:45.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.686 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop_reply(client.4230.0:1 2.3 e16/15 ondisk, result = 0)", 2024-09-11T06:49:45.687 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.238282+0000", 2024-09-11T06:49:45.687 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 34.441447261999997, 2024-09-11T06:49:45.687 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018970799999999999, 2024-09-11T06:49:45.687 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:45.687 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:45.687 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:49:45.687 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:45.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.687 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:45.687 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238282+0000", 2024-09-11T06:49:45.687 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:45.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.688 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:45.688 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238282+0000", 2024-09-11T06:49:45.688 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:45.688 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.688 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:45.688 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238284+0000", 2024-09-11T06:49:45.688 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:49:45.688 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.688 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:45.688 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238292+0000", 2024-09-11T06:49:45.688 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2390000000000002e-06 2024-09-11T06:49:45.688 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.689 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:45.689 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238314+0000", 2024-09-11T06:49:45.689 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1776999999999998e-05 2024-09-11T06:49:45.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.689 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:45.689 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238339+0000", 2024-09-11T06:49:45.689 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5119000000000001e-05 2024-09-11T06:49:45.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.689 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:45.689 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238438+0000", 2024-09-11T06:49:45.689 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9553e-05 2024-09-11T06:49:45.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.690 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:45.690 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238451+0000", 2024-09-11T06:49:45.690 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2286000000000001e-05 2024-09-11T06:49:45.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.690 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:45.690 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238472+0000", 2024-09-11T06:49:45.690 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1166999999999999e-05 2024-09-11T06:49:45.690 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:45.690 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:45.690 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:45.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.690 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:49:45.690 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.244627+0000", 2024-09-11T06:49:45.691 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 34.435102563999997, 2024-09-11T06:49:45.691 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00046141999999999998, 2024-09-11T06:49:45.691 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:45.691 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:45.691 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:49:45.691 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:45.691 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:49:45.691 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:49:45.691 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 2 2024-09-11T06:49:45.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.691 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:45.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.691 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:45.691 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244627+0000", 2024-09-11T06:49:45.691 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:45.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.692 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:45.692 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244627+0000", 2024-09-11T06:49:45.692 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:45.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.692 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:45.692 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244629+0000", 2024-09-11T06:49:45.692 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.95e-06 2024-09-11T06:49:45.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.692 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:45.692 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244640+0000", 2024-09-11T06:49:45.692 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1494e-05 2024-09-11T06:49:45.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.693 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:45.693 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244664+0000", 2024-09-11T06:49:45.693 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4006e-05 2024-09-11T06:49:45.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.693 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:45.693 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244684+0000", 2024-09-11T06:49:45.693 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.961e-05 2024-09-11T06:49:45.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.693 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:45.693 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244862+0000", 2024-09-11T06:49:45.693 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017863500000000001 2024-09-11T06:49:45.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.694 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:45.694 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245088+0000", 2024-09-11T06:49:45.694 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000225725 2024-09-11T06:49:45.694 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:45.694 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:45.694 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:45.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.694 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:49:45.694 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.245492+0000", 2024-09-11T06:49:45.694 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 34.434237621000001, 2024-09-11T06:49:45.694 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00042092600000000001, 2024-09-11T06:49:45.694 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:45.695 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:45.695 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:49:45.695 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:45.695 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:49:45.695 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:49:45.695 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 3 2024-09-11T06:49:45.695 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.695 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:45.695 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.695 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:45.695 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245492+0000", 2024-09-11T06:49:45.695 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:45.695 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.695 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.695 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:45.695 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245492+0000", 2024-09-11T06:49:45.696 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:45.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.696 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.696 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:45.696 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245493+0000", 2024-09-11T06:49:45.696 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.657e-06 2024-09-11T06:49:45.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.696 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.696 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:45.696 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245501+0000", 2024-09-11T06:49:45.696 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2490000000000004e-06 2024-09-11T06:49:45.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.696 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.696 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:45.696 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245523+0000", 2024-09-11T06:49:45.697 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2016999999999999e-05 2024-09-11T06:49:45.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.697 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.697 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:45.697 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245548+0000", 2024-09-11T06:49:45.697 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5060000000000001e-05 2024-09-11T06:49:45.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.697 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.697 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:45.697 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245720+0000", 2024-09-11T06:49:45.697 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017190200000000001 2024-09-11T06:49:45.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.697 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.697 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:45.697 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245913+0000", 2024-09-11T06:49:45.698 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193041 2024-09-11T06:49:45.698 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:45.698 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:45.698 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:45.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.698 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.698 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-11T06:49:45.698 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:49:45.698 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 33.447290744, 2024-09-11T06:49:45.698 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069904770000000001, 2024-09-11T06:49:45.698 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:45.698 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:45.698 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:45.698 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:45.698 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:49:45.699 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:49:45.699 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 5 2024-09-11T06:49:45.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.699 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:45.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.699 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:45.699 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:49:45.699 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:45.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.699 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:45.699 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:49:45.699 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:45.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.700 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:45.700 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232442+0000", 2024-09-11T06:49:45.700 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.044e-06 2024-09-11T06:49:45.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.700 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.700 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:45.700 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232460+0000", 2024-09-11T06:49:45.700 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7863000000000002e-05 2024-09-11T06:49:45.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.700 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.700 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:45.700 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232492+0000", 2024-09-11T06:49:45.700 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2812999999999999e-05 2024-09-11T06:49:45.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.700 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.700 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:45.701 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232510+0000", 2024-09-11T06:49:45.701 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7552999999999999e-05 2024-09-11T06:49:45.701 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.701 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.701 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:45.701 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232640+0000", 2024-09-11T06:49:45.701 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013029900000000001 2024-09-11T06:49:45.701 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.701 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.701 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:49:45.701 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232650+0000", 2024-09-11T06:49:45.701 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0023999999999999e-05 2024-09-11T06:49:45.701 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.701 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.702 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:45.702 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235641+0000", 2024-09-11T06:49:45.702 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00299089 2024-09-11T06:49:45.702 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.702 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.702 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:45.702 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235980+0000", 2024-09-11T06:49:45.702 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000338907 2024-09-11T06:49:45.702 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.702 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.702 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:49:45.702 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.236351+0000", 2024-09-11T06:49:45.702 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037087699999999998 2024-09-11T06:49:45.702 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.702 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.702 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:45.703 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238074+0000", 2024-09-11T06:49:45.703 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001722859 2024-09-11T06:49:45.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.703 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.703 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:45.703 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239160+0000", 2024-09-11T06:49:45.703 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001086074 2024-09-11T06:49:45.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.703 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.703 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:45.703 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239299+0000", 2024-09-11T06:49:45.703 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013933500000000001 2024-09-11T06:49:45.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.703 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.704 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:45.704 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239429+0000", 2024-09-11T06:49:45.704 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129939 2024-09-11T06:49:45.704 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:45.704 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:45.704 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:45.704 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.704 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.704 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop_reply(client.4230.0:5 2.2 e16/15 ondisk, result = 0)", 2024-09-11T06:49:45.704 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:12.238911+0000", 2024-09-11T06:49:45.704 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 33.440818456000002, 2024-09-11T06:49:45.704 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00053624900000000004, 2024-09-11T06:49:45.704 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:45.704 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:45.705 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:49:45.705 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:45.705 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.705 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:45.705 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238911+0000", 2024-09-11T06:49:45.705 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:45.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.705 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.705 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:45.705 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238911+0000", 2024-09-11T06:49:45.705 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:45.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.705 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.705 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:45.705 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238913+0000", 2024-09-11T06:49:45.706 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2069999999999998e-06 2024-09-11T06:49:45.706 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.706 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.706 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:45.706 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238923+0000", 2024-09-11T06:49:45.706 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0360000000000001e-05 2024-09-11T06:49:45.706 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.706 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.706 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:45.706 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238953+0000", 2024-09-11T06:49:45.706 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9258999999999999e-05 2024-09-11T06:49:45.706 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.706 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.706 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:45.707 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238967+0000", 2024-09-11T06:49:45.707 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3797e-05 2024-09-11T06:49:45.707 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.707 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.707 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:45.707 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239117+0000", 2024-09-11T06:49:45.707 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000150765 2024-09-11T06:49:45.707 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.707 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.707 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:45.707 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239143+0000", 2024-09-11T06:49:45.707 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5514e-05 2024-09-11T06:49:45.707 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.707 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.707 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:45.708 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239447+0000", 2024-09-11T06:49:45.708 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00030434700000000001 2024-09-11T06:49:45.708 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:45.708 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:45.708 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:45.708 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.708 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.708 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-11T06:49:45.708 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:49:45.708 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 32.433202792000003, 2024-09-11T06:49:45.708 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069253250000000004, 2024-09-11T06:49:45.708 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:45.708 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:45.708 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:45.709 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:45.709 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:49:45.709 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:49:45.709 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 7 2024-09-11T06:49:45.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.709 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:45.709 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.709 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:45.709 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:49:45.709 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:45.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.709 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.709 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:45.709 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:49:45.709 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:45.710 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.710 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.710 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:45.710 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246530+0000", 2024-09-11T06:49:45.710 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9469999999999999e-06 2024-09-11T06:49:45.710 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.710 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.710 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:45.710 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246545+0000", 2024-09-11T06:49:45.710 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5973000000000001e-05 2024-09-11T06:49:45.710 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.710 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.710 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:45.710 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246578+0000", 2024-09-11T06:49:45.710 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2759999999999998e-05 2024-09-11T06:49:45.710 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.711 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.711 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:45.711 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246595+0000", 2024-09-11T06:49:45.711 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6939000000000002e-05 2024-09-11T06:49:45.711 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.711 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.711 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:45.711 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246770+0000", 2024-09-11T06:49:45.711 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000174723 2024-09-11T06:49:45.711 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.711 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.711 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:49:45.711 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246789+0000", 2024-09-11T06:49:45.711 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9159000000000001e-05 2024-09-11T06:49:45.711 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.712 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.712 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:45.712 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250337+0000", 2024-09-11T06:49:45.712 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035480759999999998 2024-09-11T06:49:45.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.712 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.712 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:45.712 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250549+0000", 2024-09-11T06:49:45.712 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021192199999999999 2024-09-11T06:49:45.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.712 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.712 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:49:45.712 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250750+0000", 2024-09-11T06:49:45.712 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020104199999999999 2024-09-11T06:49:45.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.713 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.713 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:45.713 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.252455+0000", 2024-09-11T06:49:45.713 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001705249 2024-09-11T06:49:45.713 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.713 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.713 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:45.713 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253305+0000", 2024-09-11T06:49:45.713 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00084941200000000002 2024-09-11T06:49:45.713 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.713 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.713 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:45.713 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253388+0000", 2024-09-11T06:49:45.713 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3642999999999999e-05 2024-09-11T06:49:45.713 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.714 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.714 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:45.714 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253452+0000", 2024-09-11T06:49:45.714 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3479999999999995e-05 2024-09-11T06:49:45.714 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:45.714 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:45.714 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:45.714 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.714 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.714 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop_reply(client.4230.0:7 2.2 e18/15 ondisk, result = 0)", 2024-09-11T06:49:45.714 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:13.253128+0000", 2024-09-11T06:49:45.714 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 32.426601218000002, 2024-09-11T06:49:45.714 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034023700000000002, 2024-09-11T06:49:45.714 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:45.714 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:45.714 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:49:45.715 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:45.715 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.715 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:45.715 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253128+0000", 2024-09-11T06:49:45.715 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:45.715 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.715 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.715 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:45.715 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253128+0000", 2024-09-11T06:49:45.715 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:45.715 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.715 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.715 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:45.715 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253135+0000", 2024-09-11T06:49:45.716 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.657e-06 2024-09-11T06:49:45.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.716 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.716 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:45.716 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253144+0000", 2024-09-11T06:49:45.716 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0869999999999996e-06 2024-09-11T06:49:45.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.716 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.716 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:45.716 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253167+0000", 2024-09-11T06:49:45.716 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3252e-05 2024-09-11T06:49:45.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.716 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.716 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:45.716 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253186+0000", 2024-09-11T06:49:45.716 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9239999999999999e-05 2024-09-11T06:49:45.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.717 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.717 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:45.717 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253282+0000", 2024-09-11T06:49:45.717 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.6093000000000003e-05 2024-09-11T06:49:45.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.717 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.717 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:45.717 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253294+0000", 2024-09-11T06:49:45.717 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1986e-05 2024-09-11T06:49:45.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:45.717 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:45.717 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:45.717 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253468+0000", 2024-09-11T06:49:45.718 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017392199999999999 2024-09-11T06:49:45.718 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:45.718 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:45.718 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:45.718 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:45.718 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:45.718 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:45.719 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:49:45.763 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:45.762+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:45.845 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:49:45.845 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:45.858 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:49:45.903 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:45.902+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:45.965 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:45.962+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:45.980 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:45.980 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:45.980 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:49:45.980 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:49:45.980 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:45.994 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:49:46.065 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:46.066+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:46.158 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:46.158 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:49:46.158 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:49:46.158 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:49:46.158 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.158 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:54 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'32, mlcod=13'29)", 2024-09-11T06:49:46.158 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:49:46.159 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 39.946170068000001, 2024-09-11T06:49:46.159 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034271670000000001, 2024-09-11T06:49:46.159 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:46.159 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:46.159 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:46.159 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:46.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.159 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:46.159 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:49:46.159 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.159 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:46.159 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:49:46.160 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.160 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.160 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:46.160 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210361+0000", 2024-09-11T06:49:46.160 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1260000000000001e-06 2024-09-11T06:49:46.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.160 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.160 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:46.160 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210370+0000", 2024-09-11T06:49:46.160 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8200000000000003e-06 2024-09-11T06:49:46.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.160 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.160 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:46.161 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210390+0000", 2024-09-11T06:49:46.161 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0126e-05 2024-09-11T06:49:46.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.161 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.161 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:46.161 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210404+0000", 2024-09-11T06:49:46.161 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4589999999999999e-05 2024-09-11T06:49:46.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.161 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.161 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:46.161 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210538+0000", 2024-09-11T06:49:46.161 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000133719 2024-09-11T06:49:46.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.162 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.162 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:46.162 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210593+0000", 2024-09-11T06:49:46.162 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5016999999999999e-05 2024-09-11T06:49:46.162 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.162 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.162 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:46.162 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213695+0000", 2024-09-11T06:49:46.162 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003101866 2024-09-11T06:49:46.162 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.162 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.162 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:46.162 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213787+0000", 2024-09-11T06:49:46.162 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1903000000000004e-05 2024-09-11T06:49:46.163 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.163 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:46.163 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.163 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.163 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.163 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:55 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'33, mlcod=13'29)", 2024-09-11T06:49:46.163 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:49:46.163 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 39.945150830000003, 2024-09-11T06:49:46.163 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.005086091, 2024-09-11T06:49:46.163 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:46.163 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:46.163 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:46.163 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:46.163 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.164 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:46.164 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:49:46.164 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.164 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.164 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.164 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:46.164 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:49:46.164 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.164 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.164 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.164 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:46.164 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211380+0000", 2024-09-11T06:49:46.164 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.097e-06 2024-09-11T06:49:46.164 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.165 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.165 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:46.165 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211537+0000", 2024-09-11T06:49:46.165 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000157369 2024-09-11T06:49:46.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.165 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.165 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:46.165 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211559+0000", 2024-09-11T06:49:46.165 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000001e-05 2024-09-11T06:49:46.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.165 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.165 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:46.165 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211573+0000", 2024-09-11T06:49:46.165 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4946000000000001e-05 2024-09-11T06:49:46.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.166 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.166 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:46.166 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211676+0000", 2024-09-11T06:49:46.166 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0001026 2024-09-11T06:49:46.166 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.166 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.166 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:46.166 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211709+0000", 2024-09-11T06:49:46.166 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3071999999999998e-05 2024-09-11T06:49:46.166 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.166 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.166 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:46.166 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216379+0000", 2024-09-11T06:49:46.166 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0046693780000000001 2024-09-11T06:49:46.167 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.167 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.167 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:46.167 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216465+0000", 2024-09-11T06:49:46.167 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6469000000000002e-05 2024-09-11T06:49:46.167 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.167 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:46.167 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.167 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.167 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.167 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:56 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'34, mlcod=13'29)", 2024-09-11T06:49:46.167 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:49:46.167 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 39.943716971000001, 2024-09-11T06:49:46.167 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0060310629999999997, 2024-09-11T06:49:46.168 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:46.168 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:46.168 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:46.168 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:46.168 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.168 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:46.168 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:49:46.168 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.168 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.168 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:46.168 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:49:46.168 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.169 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.169 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:46.169 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212814+0000", 2024-09-11T06:49:46.169 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0869999999999999e-06 2024-09-11T06:49:46.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.169 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.169 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:46.169 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212822+0000", 2024-09-11T06:49:46.169 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0870000000000007e-06 2024-09-11T06:49:46.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.169 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.169 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:46.169 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212840+0000", 2024-09-11T06:49:46.170 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:49:46.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.170 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.170 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:46.170 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212854+0000", 2024-09-11T06:49:46.170 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4003e-05 2024-09-11T06:49:46.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.170 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.170 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:46.170 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213026+0000", 2024-09-11T06:49:46.170 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017207499999999999 2024-09-11T06:49:46.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.170 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.171 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:46.171 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213063+0000", 2024-09-11T06:49:46.171 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7187000000000002e-05 2024-09-11T06:49:46.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.171 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.171 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:46.171 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218762+0000", 2024-09-11T06:49:46.171 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0056993290000000004 2024-09-11T06:49:46.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.171 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.171 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:46.171 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218844+0000", 2024-09-11T06:49:46.171 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1701999999999998e-05 2024-09-11T06:49:46.171 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.171 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:46.172 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.172 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.172 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.172 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:60 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'38, mlcod=13'36)", 2024-09-11T06:49:46.172 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:49:46.172 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 39.929775622000001, 2024-09-11T06:49:46.172 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039254270000000004, 2024-09-11T06:49:46.172 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:46.172 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:46.172 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:46.172 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:46.172 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.172 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:46.172 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:49:46.173 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.173 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.173 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.173 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:46.173 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:49:46.173 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.173 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.173 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.173 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:46.173 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226756+0000", 2024-09-11T06:49:46.173 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-11T06:49:46.173 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.173 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.173 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:46.173 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226768+0000", 2024-09-11T06:49:46.174 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1919999999999999e-05 2024-09-11T06:49:46.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.174 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.174 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:46.174 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226797+0000", 2024-09-11T06:49:46.174 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9573e-05 2024-09-11T06:49:46.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.174 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.174 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:46.174 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226818+0000", 2024-09-11T06:49:46.174 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0432999999999999e-05 2024-09-11T06:49:46.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.174 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.174 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:46.174 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226909+0000", 2024-09-11T06:49:46.174 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0920000000000004e-05 2024-09-11T06:49:46.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.175 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.175 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:46.175 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226936+0000", 2024-09-11T06:49:46.175 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7472999999999999e-05 2024-09-11T06:49:46.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.175 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.175 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:46.175 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230596+0000", 2024-09-11T06:49:46.175 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0036602520000000001 2024-09-11T06:49:46.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.175 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.175 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:46.176 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230680+0000", 2024-09-11T06:49:46.176 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3106e-05 2024-09-11T06:49:46.176 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.176 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:46.176 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.176 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.176 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:107 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'69, mlcod=13'64)", 2024-09-11T06:49:46.176 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:49:46.176 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 39.713328165, 2024-09-11T06:49:46.176 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040728830000000002, 2024-09-11T06:49:46.176 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:46.176 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:46.176 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:46.176 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:46.177 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.177 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:46.177 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:49:46.177 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.177 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.177 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:46.177 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:49:46.177 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.177 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.177 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:46.177 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443203+0000", 2024-09-11T06:49:46.177 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1480000000000001e-06 2024-09-11T06:49:46.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.178 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:46.178 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443222+0000", 2024-09-11T06:49:46.178 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9040000000000001e-05 2024-09-11T06:49:46.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.178 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:46.178 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443246+0000", 2024-09-11T06:49:46.178 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4606000000000001e-05 2024-09-11T06:49:46.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.178 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:46.178 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443277+0000", 2024-09-11T06:49:46.178 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0207999999999999e-05 2024-09-11T06:49:46.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.179 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.179 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:46.179 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443340+0000", 2024-09-11T06:49:46.179 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3602000000000005e-05 2024-09-11T06:49:46.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.179 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.179 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:46.179 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443363+0000", 2024-09-11T06:49:46.179 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2331e-05 2024-09-11T06:49:46.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.179 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.179 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:46.179 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447216+0000", 2024-09-11T06:49:46.180 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0038539690000000001 2024-09-11T06:49:46.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.180 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.180 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:46.180 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447274+0000", 2024-09-11T06:49:46.180 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7979000000000002e-05 2024-09-11T06:49:46.180 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.180 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:46.180 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.180 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.180 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:111 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'73, mlcod=13'67)", 2024-09-11T06:49:46.180 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:49:46.180 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 39.706816535000002, 2024-09-11T06:49:46.181 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034372360000000002, 2024-09-11T06:49:46.181 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:46.181 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:46.181 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:46.181 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:46.181 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.181 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:46.181 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:49:46.181 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.181 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.181 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.181 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:46.181 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:49:46.182 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.182 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:46.182 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449714+0000", 2024-09-11T06:49:46.182 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5000000000000001e-07 2024-09-11T06:49:46.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.182 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:46.182 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449721+0000", 2024-09-11T06:49:46.182 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8530000000000003e-06 2024-09-11T06:49:46.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.182 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:46.182 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449736+0000", 2024-09-11T06:49:46.183 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5114e-05 2024-09-11T06:49:46.183 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.183 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.183 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:46.183 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449743+0000", 2024-09-11T06:49:46.183 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0210000000000002e-06 2024-09-11T06:49:46.183 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.183 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.183 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:46.183 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449804+0000", 2024-09-11T06:49:46.183 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0479999999999997e-05 2024-09-11T06:49:46.183 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.183 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.183 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:46.184 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449826+0000", 2024-09-11T06:49:46.184 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1922000000000001e-05 2024-09-11T06:49:46.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.184 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.184 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:46.184 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453104+0000", 2024-09-11T06:49:46.184 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003278652 2024-09-11T06:49:46.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.184 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.184 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:46.184 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453150+0000", 2024-09-11T06:49:46.184 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6244e-05 2024-09-11T06:49:46.184 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.184 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:46.185 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.185 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.185 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:114 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'76, mlcod=13'71)", 2024-09-11T06:49:46.185 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:49:46.185 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 39.702138464000001, 2024-09-11T06:49:46.185 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0042576660000000002, 2024-09-11T06:49:46.185 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:46.185 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:46.185 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:46.185 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:46.185 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.185 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:46.185 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:49:46.186 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.186 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:46.186 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:49:46.186 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.186 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:46.186 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454392+0000", 2024-09-11T06:49:46.186 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5999999999999991e-07 2024-09-11T06:49:46.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.187 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:46.187 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454399+0000", 2024-09-11T06:49:46.187 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.878e-06 2024-09-11T06:49:46.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.187 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.187 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:46.187 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454415+0000", 2024-09-11T06:49:46.187 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5659999999999999e-05 2024-09-11T06:49:46.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.187 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.187 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:46.187 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454422+0000", 2024-09-11T06:49:46.187 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:49:46.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.187 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.188 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:46.188 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454474+0000", 2024-09-11T06:49:46.188 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1860999999999997e-05 2024-09-11T06:49:46.188 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.188 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.188 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:46.188 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454497+0000", 2024-09-11T06:49:46.188 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3686e-05 2024-09-11T06:49:46.188 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.188 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.188 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:46.188 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458552+0000", 2024-09-11T06:49:46.188 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040544550000000002 2024-09-11T06:49:46.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.189 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.189 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:46.189 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458649+0000", 2024-09-11T06:49:46.189 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7005999999999999e-05 2024-09-11T06:49:46.189 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.189 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:46.189 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.189 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.189 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:115 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'77, mlcod=13'72)", 2024-09-11T06:49:46.189 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:46.189 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 39.700797852000001, 2024-09-11T06:49:46.189 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003355584, 2024-09-11T06:49:46.190 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:46.190 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:46.190 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:46.190 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:46.190 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.190 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:46.190 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:46.190 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.190 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.190 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:46.190 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:46.190 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.191 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:46.191 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:46.191 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5999999999999999e-07 2024-09-11T06:49:46.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.191 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:46.191 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455803+0000", 2024-09-11T06:49:46.191 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0310999999999998e-05 2024-09-11T06:49:46.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.191 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:46.191 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455815+0000", 2024-09-11T06:49:46.192 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2787e-05 2024-09-11T06:49:46.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.192 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:46.192 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455823+0000", 2024-09-11T06:49:46.192 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5959999999999998e-06 2024-09-11T06:49:46.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.192 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:46.192 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455875+0000", 2024-09-11T06:49:46.192 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2054e-05 2024-09-11T06:49:46.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.192 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:46.193 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455896+0000", 2024-09-11T06:49:46.193 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0554999999999999e-05 2024-09-11T06:49:46.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.193 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.193 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:46.193 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459000+0000", 2024-09-11T06:49:46.193 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003104842 2024-09-11T06:49:46.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.193 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.193 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:46.193 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459087+0000", 2024-09-11T06:49:46.193 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6978999999999995e-05 2024-09-11T06:49:46.193 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.194 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:46.194 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.194 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.194 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:120 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'82, mlcod=13'78)", 2024-09-11T06:49:46.194 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:49:46.194 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 39.690120567000001, 2024-09-11T06:49:46.194 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0064612549999999999, 2024-09-11T06:49:46.194 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:46.194 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:46.194 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:46.194 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:46.194 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.194 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:46.195 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:49:46.195 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.195 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.195 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.195 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:46.195 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:49:46.195 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.195 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.195 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.195 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:46.195 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466410+0000", 2024-09-11T06:49:46.195 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0389999999999999e-06 2024-09-11T06:49:46.195 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.195 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.195 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:46.195 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466496+0000", 2024-09-11T06:49:46.196 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5519000000000006e-05 2024-09-11T06:49:46.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.196 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.196 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:46.196 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466510+0000", 2024-09-11T06:49:46.196 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3842999999999999e-05 2024-09-11T06:49:46.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.196 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.196 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:46.196 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466517+0000", 2024-09-11T06:49:46.196 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5229999999999998e-06 2024-09-11T06:49:46.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.196 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.197 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:46.197 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466595+0000", 2024-09-11T06:49:46.197 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8019e-05 2024-09-11T06:49:46.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.197 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:46.197 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466612+0000", 2024-09-11T06:49:46.197 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7013000000000001e-05 2024-09-11T06:49:46.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.197 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:46.197 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472787+0000", 2024-09-11T06:49:46.197 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0061750269999999996 2024-09-11T06:49:46.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:46.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472870+0000", 2024-09-11T06:49:46.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3271999999999998e-05 2024-09-11T06:49:46.198 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.198 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:46.198 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.198 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:121 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'83, mlcod=13'78)", 2024-09-11T06:49:46.198 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:49:46.198 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 39.688895770999999, 2024-09-11T06:49:46.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055613069999999997, 2024-09-11T06:49:46.198 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:46.198 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:46.198 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:46.199 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:46.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.199 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:46.199 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:49:46.199 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.199 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:46.199 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:49:46.199 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.199 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:46.199 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467635+0000", 2024-09-11T06:49:46.200 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6899999999999996e-07 2024-09-11T06:49:46.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.200 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.200 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:46.200 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467727+0000", 2024-09-11T06:49:46.200 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1800999999999997e-05 2024-09-11T06:49:46.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.200 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.200 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:46.200 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467740+0000", 2024-09-11T06:49:46.200 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3747999999999999e-05 2024-09-11T06:49:46.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.200 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.201 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:46.201 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467753+0000", 2024-09-11T06:49:46.201 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2807e-05 2024-09-11T06:49:46.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.201 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.201 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:46.201 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467812+0000", 2024-09-11T06:49:46.201 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8381999999999998e-05 2024-09-11T06:49:46.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.201 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.201 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:46.201 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467828+0000", 2024-09-11T06:49:46.201 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6728e-05 2024-09-11T06:49:46.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.202 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.202 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:46.202 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473112+0000", 2024-09-11T06:49:46.202 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052833619999999998 2024-09-11T06:49:46.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.202 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.202 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:46.202 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473195+0000", 2024-09-11T06:49:46.202 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3609999999999994e-05 2024-09-11T06:49:46.202 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.202 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:46.202 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.202 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.202 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:131 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'89, mlcod=13'85)", 2024-09-11T06:49:46.203 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:49:46.203 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 39.337490056, 2024-09-11T06:49:46.203 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052492260000000001, 2024-09-11T06:49:46.203 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:46.203 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:46.203 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:46.203 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:46.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.203 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:46.203 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:49:46.203 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.203 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:46.203 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:49:46.203 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.204 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.204 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:46.204 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819041+0000", 2024-09-11T06:49:46.204 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4360000000000001e-06 2024-09-11T06:49:46.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.204 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.204 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:46.204 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819100+0000", 2024-09-11T06:49:46.204 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9225999999999998e-05 2024-09-11T06:49:46.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.204 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.204 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:46.204 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819128+0000", 2024-09-11T06:49:46.204 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.779e-05 2024-09-11T06:49:46.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.205 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:46.205 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819142+0000", 2024-09-11T06:49:46.205 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3573e-05 2024-09-11T06:49:46.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.205 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:46.205 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819263+0000", 2024-09-11T06:49:46.205 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000121086 2024-09-11T06:49:46.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.205 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:46.205 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819292+0000", 2024-09-11T06:49:46.205 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9363e-05 2024-09-11T06:49:46.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.206 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.206 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:46.206 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824185+0000", 2024-09-11T06:49:46.206 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0048924620000000002 2024-09-11T06:49:46.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.206 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.206 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:46.206 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824289+0000", 2024-09-11T06:49:46.206 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010429000000000001 2024-09-11T06:49:46.206 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.206 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:46.206 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.206 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.206 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:132 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'90, mlcod=13'85)", 2024-09-11T06:49:46.206 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:49:46.207 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 39.336715366, 2024-09-11T06:49:46.207 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.004532342, 2024-09-11T06:49:46.207 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:46.207 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:46.207 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:46.207 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:46.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.207 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:46.207 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:49:46.207 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.207 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:46.208 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:49:46.208 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.208 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.208 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:46.208 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819816+0000", 2024-09-11T06:49:46.208 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.61e-06 2024-09-11T06:49:46.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.208 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.208 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:46.208 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819827+0000", 2024-09-11T06:49:46.208 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1e-05 2024-09-11T06:49:46.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.208 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819851+0000", 2024-09-11T06:49:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4267e-05 2024-09-11T06:49:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819864+0000", 2024-09-11T06:49:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2767e-05 2024-09-11T06:49:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820071+0000", 2024-09-11T06:49:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000207061 2024-09-11T06:49:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.210 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.210 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:46.210 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820103+0000", 2024-09-11T06:49:46.210 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2326999999999997e-05 2024-09-11T06:49:46.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.210 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.210 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:46.210 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824302+0000", 2024-09-11T06:49:46.210 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0041986000000000002 2024-09-11T06:49:46.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.210 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.210 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:46.210 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824347+0000", 2024-09-11T06:49:46.210 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4709999999999997e-05 2024-09-11T06:49:46.210 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.212 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:46.212 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.212 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:133 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'91, mlcod=13'86)", 2024-09-11T06:49:46.212 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:49:46.213 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 39.336083707, 2024-09-11T06:49:46.213 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039421459999999997, 2024-09-11T06:49:46.213 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:46.213 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:46.213 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:46.213 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:46.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.213 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:46.213 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:49:46.213 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.213 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:46.214 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:49:46.214 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.214 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.214 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:46.214 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820448+0000", 2024-09-11T06:49:46.214 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.84e-06 2024-09-11T06:49:46.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.214 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.214 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:46.214 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820463+0000", 2024-09-11T06:49:46.214 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5356e-05 2024-09-11T06:49:46.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.214 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.215 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:46.215 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820499+0000", 2024-09-11T06:49:46.215 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.625e-05 2024-09-11T06:49:46.215 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.215 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.215 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:46.215 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820546+0000", 2024-09-11T06:49:46.215 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7017000000000001e-05 2024-09-11T06:49:46.215 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.215 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.215 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:46.215 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820882+0000", 2024-09-11T06:49:46.215 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033512 2024-09-11T06:49:46.215 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.216 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:46.216 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820941+0000", 2024-09-11T06:49:46.216 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8922999999999998e-05 2024-09-11T06:49:46.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.216 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:46.216 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824351+0000", 2024-09-11T06:49:46.216 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034101940000000001 2024-09-11T06:49:46.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.216 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:46.216 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824388+0000", 2024-09-11T06:49:46.216 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7446e-05 2024-09-11T06:49:46.217 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.217 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:46.217 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.217 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.217 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.217 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:134 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'92, mlcod=13'86)", 2024-09-11T06:49:46.217 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:49:46.217 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 39.335424588999999, 2024-09-11T06:49:46.217 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033248179999999998, 2024-09-11T06:49:46.217 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:46.217 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:46.217 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:46.217 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:46.217 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.218 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:46.218 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:49:46.218 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.218 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:46.218 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:49:46.218 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.218 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:46.218 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821107+0000", 2024-09-11T06:49:46.218 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7430000000000001e-06 2024-09-11T06:49:46.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.219 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.219 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:46.219 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821205+0000", 2024-09-11T06:49:46.219 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7666999999999998e-05 2024-09-11T06:49:46.219 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.219 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.219 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:46.219 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821234+0000", 2024-09-11T06:49:46.219 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9756000000000001e-05 2024-09-11T06:49:46.219 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.219 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.219 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:46.219 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821254+0000", 2024-09-11T06:49:46.219 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9446e-05 2024-09-11T06:49:46.219 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.220 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.220 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:46.220 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821846+0000", 2024-09-11T06:49:46.220 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00059187599999999997 2024-09-11T06:49:46.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.220 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.220 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:46.220 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821894+0000", 2024-09-11T06:49:46.220 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8766e-05 2024-09-11T06:49:46.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.220 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.220 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:46.220 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824392+0000", 2024-09-11T06:49:46.220 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0024977509999999999 2024-09-11T06:49:46.221 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.221 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.221 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:46.221 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824430+0000", 2024-09-11T06:49:46.221 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7812999999999999e-05 2024-09-11T06:49:46.221 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.221 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:46.221 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.221 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.221 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.221 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:137 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'95, mlcod=13'86)", 2024-09-11T06:49:46.221 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:49:46.221 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 39.332962448000004, 2024-09-11T06:49:46.221 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003084299, 2024-09-11T06:49:46.222 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:46.222 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:46.222 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:46.222 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:46.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.222 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:46.222 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:49:46.222 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.222 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:46.222 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:49:46.222 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.223 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:46.223 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823568+0000", 2024-09-11T06:49:46.223 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1030000000000001e-06 2024-09-11T06:49:46.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.223 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.223 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:46.223 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823662+0000", 2024-09-11T06:49:46.223 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3529999999999994e-05 2024-09-11T06:49:46.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.223 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.223 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:46.223 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823686+0000", 2024-09-11T06:49:46.223 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4358999999999998e-05 2024-09-11T06:49:46.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.223 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.223 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:46.224 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823701+0000", 2024-09-11T06:49:46.224 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4847e-05 2024-09-11T06:49:46.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.224 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.224 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:46.224 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824518+0000", 2024-09-11T06:49:46.224 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00081716700000000005 2024-09-11T06:49:46.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.224 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.224 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:46.224 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824546+0000", 2024-09-11T06:49:46.224 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7860000000000001e-05 2024-09-11T06:49:46.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.224 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.225 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:46.225 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826558+0000", 2024-09-11T06:49:46.225 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0020120009999999998 2024-09-11T06:49:46.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.225 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:46.225 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826652+0000", 2024-09-11T06:49:46.225 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3431999999999996e-05 2024-09-11T06:49:46.225 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.225 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:46.225 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.225 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:139 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'97, mlcod=13'86)", 2024-09-11T06:49:46.225 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:49:46.226 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 39.330673728999997, 2024-09-11T06:49:46.226 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003732221, 2024-09-11T06:49:46.226 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:46.226 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:46.226 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:46.226 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:46.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.226 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:46.226 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:49:46.226 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.226 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:46.226 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:49:46.226 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.227 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.227 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:46.227 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825858+0000", 2024-09-11T06:49:46.227 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5829999999999999e-06 2024-09-11T06:49:46.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.227 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.227 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:46.227 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825874+0000", 2024-09-11T06:49:46.227 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5967000000000001e-05 2024-09-11T06:49:46.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.227 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.227 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:46.227 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825909+0000", 2024-09-11T06:49:46.227 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5692999999999999e-05 2024-09-11T06:49:46.228 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.228 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.228 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:46.228 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825923+0000", 2024-09-11T06:49:46.228 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4080000000000001e-05 2024-09-11T06:49:46.228 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.228 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.228 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:46.228 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826010+0000", 2024-09-11T06:49:46.228 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6248999999999994e-05 2024-09-11T06:49:46.228 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.228 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.228 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:46.228 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826038+0000", 2024-09-11T06:49:46.228 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8010000000000001e-05 2024-09-11T06:49:46.229 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.229 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.229 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:46.229 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829499+0000", 2024-09-11T06:49:46.229 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003461783 2024-09-11T06:49:46.229 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.229 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.229 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:46.229 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829588+0000", 2024-09-11T06:49:46.229 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8856000000000004e-05 2024-09-11T06:49:46.229 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.229 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:46.229 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.230 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:140 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'98, mlcod=13'93)", 2024-09-11T06:49:46.230 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:49:46.230 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 39.328823, 2024-09-11T06:49:46.230 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0057995929999999996, 2024-09-11T06:49:46.230 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:46.230 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:46.230 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:46.230 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:46.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.230 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:46.230 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:49:46.230 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.231 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.231 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:46.231 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:49:46.231 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.231 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.231 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:46.231 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827708+0000", 2024-09-11T06:49:46.231 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.547e-06 2024-09-11T06:49:46.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.231 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.231 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:46.231 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827765+0000", 2024-09-11T06:49:46.231 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6266e-05 2024-09-11T06:49:46.232 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.232 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.232 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:46.232 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827791+0000", 2024-09-11T06:49:46.232 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6098999999999999e-05 2024-09-11T06:49:46.232 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.232 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.232 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:46.232 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827806+0000", 2024-09-11T06:49:46.232 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5537e-05 2024-09-11T06:49:46.232 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.232 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.232 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:46.232 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827928+0000", 2024-09-11T06:49:46.232 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000122012 2024-09-11T06:49:46.233 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.233 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.233 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:46.233 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827961+0000", 2024-09-11T06:49:46.233 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2663999999999998e-05 2024-09-11T06:49:46.233 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.233 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.233 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:46.233 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833409+0000", 2024-09-11T06:49:46.233 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054479350000000001 2024-09-11T06:49:46.233 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.233 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.233 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:46.233 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833506+0000", 2024-09-11T06:49:46.233 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7533000000000003e-05 2024-09-11T06:49:46.234 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.234 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:46.234 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.234 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:141 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'99, mlcod=13'93)", 2024-09-11T06:49:46.234 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:49:46.234 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 39.327929617000002, 2024-09-11T06:49:46.234 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055394709999999998, 2024-09-11T06:49:46.234 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:46.234 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:46.234 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:46.234 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:46.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.235 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:46.235 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:49:46.235 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.235 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:46.235 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:49:46.235 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.235 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:46.235 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828602+0000", 2024-09-11T06:49:46.235 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.46e-06 2024-09-11T06:49:46.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.236 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.236 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:46.236 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828618+0000", 2024-09-11T06:49:46.236 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6592999999999999e-05 2024-09-11T06:49:46.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.236 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.236 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:46.236 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828646+0000", 2024-09-11T06:49:46.236 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7552999999999999e-05 2024-09-11T06:49:46.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.236 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.236 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:46.236 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828663+0000", 2024-09-11T06:49:46.236 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.719e-05 2024-09-11T06:49:46.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.237 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.237 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:46.237 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829023+0000", 2024-09-11T06:49:46.237 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036054799999999998 2024-09-11T06:49:46.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.237 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.237 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:46.237 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829062+0000", 2024-09-11T06:49:46.237 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8865999999999997e-05 2024-09-11T06:49:46.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.237 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.237 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:46.237 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834078+0000", 2024-09-11T06:49:46.237 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050157480000000004 2024-09-11T06:49:46.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.238 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.238 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:46.238 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834140+0000", 2024-09-11T06:49:46.238 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1513000000000005e-05 2024-09-11T06:49:46.238 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.238 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:46.238 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.238 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.238 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:144 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'102, mlcod=13'98)", 2024-09-11T06:49:46.238 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:49:46.238 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 39.318769004000004, 2024-09-11T06:49:46.238 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003210348, 2024-09-11T06:49:46.238 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:46.239 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:46.239 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:46.239 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:46.239 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.239 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:46.239 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:49:46.239 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.239 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.239 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:46.239 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:49:46.239 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.240 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.240 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:46.240 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837768+0000", 2024-09-11T06:49:46.240 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9330000000000004e-06 2024-09-11T06:49:46.240 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.240 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.240 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:46.240 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837858+0000", 2024-09-11T06:49:46.240 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0185999999999993e-05 2024-09-11T06:49:46.240 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.240 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.240 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:46.240 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837890+0000", 2024-09-11T06:49:46.240 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2437000000000001e-05 2024-09-11T06:49:46.241 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.241 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.241 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:46.241 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837905+0000", 2024-09-11T06:49:46.241 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:49:46.241 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.241 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.241 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:46.241 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838019+0000", 2024-09-11T06:49:46.241 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000114336 2024-09-11T06:49:46.241 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.241 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.241 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:46.241 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838052+0000", 2024-09-11T06:49:46.241 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3139000000000002e-05 2024-09-11T06:49:46.242 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.242 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.242 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:46.242 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840914+0000", 2024-09-11T06:49:46.242 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028612780000000001 2024-09-11T06:49:46.242 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.242 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.242 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:46.242 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840971+0000", 2024-09-11T06:49:46.242 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7402999999999999e-05 2024-09-11T06:49:46.242 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.242 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:46.242 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.243 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.243 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.243 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:145 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'103, mlcod=13'98)", 2024-09-11T06:49:46.243 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:49:46.243 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 39.317977120000002, 2024-09-11T06:49:46.243 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054455409999999999, 2024-09-11T06:49:46.243 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:46.243 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:46.243 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:46.243 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:46.243 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.243 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:46.243 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:49:46.243 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.244 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.244 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.244 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:46.244 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:49:46.244 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:46.244 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.244 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.244 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:46.244 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838554+0000", 2024-09-11T06:49:46.244 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3430000000000001e-06 2024-09-11T06:49:46.244 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.244 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.244 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:46.244 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838640+0000", 2024-09-11T06:49:46.245 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5569999999999996e-05 2024-09-11T06:49:46.245 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.245 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.245 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:46.245 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838660+0000", 2024-09-11T06:49:46.245 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0576000000000001e-05 2024-09-11T06:49:46.245 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.245 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.245 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:46.245 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838674+0000", 2024-09-11T06:49:46.245 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3529999999999999e-05 2024-09-11T06:49:46.245 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.245 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.245 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:46.246 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838838+0000", 2024-09-11T06:49:46.246 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016468799999999999 2024-09-11T06:49:46.246 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.246 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.246 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:46.246 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838885+0000", 2024-09-11T06:49:46.246 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6884e-05 2024-09-11T06:49:46.246 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.246 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.246 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:46.246 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843901+0000", 2024-09-11T06:49:46.246 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050162310000000003 2024-09-11T06:49:46.246 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:46.246 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:46.247 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:46.247 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843998+0000", 2024-09-11T06:49:46.247 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.6718999999999993e-05 2024-09-11T06:49:46.247 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.247 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:46.247 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.247 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:46.247 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:46.247 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:46.249 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:49:46.249 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:46.250+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:46.334 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:49:46.374 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:46.374 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:46.374 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:49:46.374 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:46.375 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:46.370+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:46.389 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:49:46.430 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T06:49:46.450 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:46.446+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:46.489 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:49:46.489 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:49:46.507 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:49:46.507 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:49:46.523 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:49:46.524 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:49:46.544 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:49:46.544 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:49:46.551 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:46.546+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:46.568 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:46.568 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:46.568 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:49:46.568 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:49:46.581 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:49:46.588 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:49:46.589 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:49:46.652 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:46.650+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:46.697 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:46.697 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:46.710 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:49:46.753 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:46.750+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:46.854 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:46.850+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:46.856 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:46.856 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:46.869 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:49:46.954 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:46.954+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:47.037 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:47.037 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:47.051 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:49:47.095 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:47.094+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:47.156 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:47.154+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:47.170 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:47.170 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:47.184 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:49:47.257 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:47.253+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:47.316 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:47.316 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:47.330 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:49:47.358 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:47.353+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:47.447 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:47.447 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:47.459 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:47.457+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:47.461 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:49:47.522 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is set 2024-09-11T06:49:47.579 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:47.579 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:47.587 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:47.585+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:47.594 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:49:47.660 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:47.657+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:47.715 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:47.715 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:47.729 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:49:47.778 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:47.778+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:47.850 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:47.850 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:47.862 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:47.861+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:47.864 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:49:47.962 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:47.962+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:48.034 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:48.034 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:48.048 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:49:48.063 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:48.061+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:48.165 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:48.165+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:48.209 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:48.210 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:48.223 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:49:48.274 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:48.274+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:48.348 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:48.348 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:48.363 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:49:48.366 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:48.361+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:48.420 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T06:49:48.420 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T06:49:48.420 INFO:tasks.thrashosds.thrasher:Reweighting osd 0 to 0.14695583911316873 2024-09-11T06:49:48.421 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 0.14695583911316873 2024-09-11T06:49:48.469 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:48.466+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:48.530 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:48.530 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:48.544 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:49:48.586 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:48.586+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:48.606 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:49:48.670 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:48.670 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:48.671 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:48.670+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:48.683 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:49:48.740 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:49:48.741 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:49:48.757 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:49:48.757 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:49:48.772 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:48.770+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:48.774 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:49:48.774 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:49:48.791 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:48.792 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:48.792 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:49:48.792 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:49:48.792 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:48.794 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:49:48.794 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:49:48.805 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:49:48.816 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:49:48.816 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:49:48.836 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:49:48.836 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:49:48.873 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:48.870+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:48.917 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:48.917 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:49:48.917 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:49:48.917 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:49:48.917 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.917 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:54 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'32, mlcod=13'29)", 2024-09-11T06:49:48.918 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:49:48.918 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 42.706627329, 2024-09-11T06:49:48.918 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034271670000000001, 2024-09-11T06:49:48.918 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:48.918 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:48.918 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:48.918 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:48.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.918 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:48.918 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:49:48.918 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.919 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:48.919 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:49:48.919 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.919 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.919 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.919 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:48.919 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210361+0000", 2024-09-11T06:49:48.919 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1260000000000001e-06 2024-09-11T06:49:48.919 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.919 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.919 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:48.919 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210370+0000", 2024-09-11T06:49:48.919 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8200000000000003e-06 2024-09-11T06:49:48.919 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.920 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.920 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:48.920 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210390+0000", 2024-09-11T06:49:48.920 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0126e-05 2024-09-11T06:49:48.920 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.920 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.920 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:48.920 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210404+0000", 2024-09-11T06:49:48.920 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4589999999999999e-05 2024-09-11T06:49:48.920 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.920 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.920 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:48.920 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210538+0000", 2024-09-11T06:49:48.921 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000133719 2024-09-11T06:49:48.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.921 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.921 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:48.921 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210593+0000", 2024-09-11T06:49:48.921 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5016999999999999e-05 2024-09-11T06:49:48.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.921 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.921 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:48.921 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213695+0000", 2024-09-11T06:49:48.921 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003101866 2024-09-11T06:49:48.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.921 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.921 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:48.922 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213787+0000", 2024-09-11T06:49:48.922 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1903000000000004e-05 2024-09-11T06:49:48.922 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.922 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:48.922 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.922 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.922 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:55 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'33, mlcod=13'29)", 2024-09-11T06:49:48.922 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:49:48.922 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 42.705608091000002, 2024-09-11T06:49:48.922 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.005086091, 2024-09-11T06:49:48.922 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:48.922 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:48.922 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:48.923 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:48.923 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.923 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:48.923 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:49:48.923 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.923 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.923 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.923 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:48.923 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:49:48.923 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.923 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.923 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.923 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:48.924 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211380+0000", 2024-09-11T06:49:48.924 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.097e-06 2024-09-11T06:49:48.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.924 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.924 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:48.924 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211537+0000", 2024-09-11T06:49:48.924 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000157369 2024-09-11T06:49:48.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.924 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.924 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:48.924 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211559+0000", 2024-09-11T06:49:48.924 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000001e-05 2024-09-11T06:49:48.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.924 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.925 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:48.925 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211573+0000", 2024-09-11T06:49:48.925 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4946000000000001e-05 2024-09-11T06:49:48.925 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.925 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.925 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:48.925 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211676+0000", 2024-09-11T06:49:48.925 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0001026 2024-09-11T06:49:48.925 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.925 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.925 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:48.925 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211709+0000", 2024-09-11T06:49:48.925 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3071999999999998e-05 2024-09-11T06:49:48.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.926 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.926 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:48.926 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216379+0000", 2024-09-11T06:49:48.926 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0046693780000000001 2024-09-11T06:49:48.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.926 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.926 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:48.926 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216465+0000", 2024-09-11T06:49:48.926 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6469000000000002e-05 2024-09-11T06:49:48.926 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.926 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:48.926 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.927 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.927 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:56 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'34, mlcod=13'29)", 2024-09-11T06:49:48.927 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:49:48.927 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 42.704174232, 2024-09-11T06:49:48.927 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0060310629999999997, 2024-09-11T06:49:48.927 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:48.927 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:48.927 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:48.927 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:48.927 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.927 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:48.927 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:49:48.927 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.928 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:48.928 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:49:48.928 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.928 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:48.928 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212814+0000", 2024-09-11T06:49:48.928 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0869999999999999e-06 2024-09-11T06:49:48.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.928 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:48.928 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212822+0000", 2024-09-11T06:49:48.928 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0870000000000007e-06 2024-09-11T06:49:48.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.929 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:48.929 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212840+0000", 2024-09-11T06:49:48.929 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:49:48.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.929 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:48.929 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212854+0000", 2024-09-11T06:49:48.929 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4003e-05 2024-09-11T06:49:48.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.929 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:48.930 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213026+0000", 2024-09-11T06:49:48.930 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017207499999999999 2024-09-11T06:49:48.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.930 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:48.930 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213063+0000", 2024-09-11T06:49:48.930 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7187000000000002e-05 2024-09-11T06:49:48.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.930 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:48.930 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218762+0000", 2024-09-11T06:49:48.930 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0056993290000000004 2024-09-11T06:49:48.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.931 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:48.931 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218844+0000", 2024-09-11T06:49:48.931 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1701999999999998e-05 2024-09-11T06:49:48.931 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.931 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:48.931 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.931 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.931 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.931 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:60 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'38, mlcod=13'36)", 2024-09-11T06:49:48.931 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:49:48.931 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 42.690232883, 2024-09-11T06:49:48.931 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039254270000000004, 2024-09-11T06:49:48.931 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:48.932 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:48.932 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:48.932 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:48.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.932 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:48.932 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:49:48.932 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.932 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:48.932 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:49:48.932 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.933 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:48.933 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226756+0000", 2024-09-11T06:49:48.933 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-11T06:49:48.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.933 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:48.933 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226768+0000", 2024-09-11T06:49:48.933 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1919999999999999e-05 2024-09-11T06:49:48.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.933 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:48.933 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226797+0000", 2024-09-11T06:49:48.933 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9573e-05 2024-09-11T06:49:48.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.934 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:48.934 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226818+0000", 2024-09-11T06:49:48.934 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0432999999999999e-05 2024-09-11T06:49:48.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.934 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:48.934 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226909+0000", 2024-09-11T06:49:48.934 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0920000000000004e-05 2024-09-11T06:49:48.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.934 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:48.935 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226936+0000", 2024-09-11T06:49:48.935 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7472999999999999e-05 2024-09-11T06:49:48.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.935 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.935 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:48.935 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230596+0000", 2024-09-11T06:49:48.935 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0036602520000000001 2024-09-11T06:49:48.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.935 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.935 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:48.935 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230680+0000", 2024-09-11T06:49:48.935 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3106e-05 2024-09-11T06:49:48.935 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.936 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:48.936 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.936 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.936 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:107 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'69, mlcod=13'64)", 2024-09-11T06:49:48.936 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:49:48.936 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 42.473785425999999, 2024-09-11T06:49:48.936 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040728830000000002, 2024-09-11T06:49:48.936 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:48.936 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:48.936 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:48.936 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:48.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.937 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:48.937 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:49:48.937 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.937 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.937 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:48.937 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:49:48.937 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.937 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.937 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:48.937 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443203+0000", 2024-09-11T06:49:48.937 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1480000000000001e-06 2024-09-11T06:49:48.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.938 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:48.938 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443222+0000", 2024-09-11T06:49:48.938 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9040000000000001e-05 2024-09-11T06:49:48.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.938 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:48.938 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443246+0000", 2024-09-11T06:49:48.938 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4606000000000001e-05 2024-09-11T06:49:48.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.938 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:48.939 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443277+0000", 2024-09-11T06:49:48.939 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0207999999999999e-05 2024-09-11T06:49:48.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.939 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.939 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:48.939 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443340+0000", 2024-09-11T06:49:48.939 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3602000000000005e-05 2024-09-11T06:49:48.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.939 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.939 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:48.939 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443363+0000", 2024-09-11T06:49:48.939 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2331e-05 2024-09-11T06:49:48.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.940 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:48.940 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447216+0000", 2024-09-11T06:49:48.940 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0038539690000000001 2024-09-11T06:49:48.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.940 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:48.940 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447274+0000", 2024-09-11T06:49:48.940 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7979000000000002e-05 2024-09-11T06:49:48.940 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.940 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:48.940 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.941 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.941 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:111 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'73, mlcod=13'67)", 2024-09-11T06:49:48.941 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:49:48.941 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 42.467273796000001, 2024-09-11T06:49:48.941 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034372360000000002, 2024-09-11T06:49:48.941 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:48.941 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:48.941 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:48.941 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:48.941 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.941 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:48.941 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:49:48.941 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.942 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.942 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.942 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:48.942 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:49:48.942 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.942 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.942 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.942 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:48.942 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449714+0000", 2024-09-11T06:49:48.942 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5000000000000001e-07 2024-09-11T06:49:48.942 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.942 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.942 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:48.942 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449721+0000", 2024-09-11T06:49:48.943 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8530000000000003e-06 2024-09-11T06:49:48.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.943 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.943 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:48.943 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449736+0000", 2024-09-11T06:49:48.943 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5114e-05 2024-09-11T06:49:48.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.943 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.943 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:48.943 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449743+0000", 2024-09-11T06:49:48.943 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0210000000000002e-06 2024-09-11T06:49:48.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.943 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.943 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:48.944 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449804+0000", 2024-09-11T06:49:48.944 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0479999999999997e-05 2024-09-11T06:49:48.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.944 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:48.944 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449826+0000", 2024-09-11T06:49:48.944 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1922000000000001e-05 2024-09-11T06:49:48.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.944 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:48.944 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453104+0000", 2024-09-11T06:49:48.944 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003278652 2024-09-11T06:49:48.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.945 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:48.945 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453150+0000", 2024-09-11T06:49:48.945 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6244e-05 2024-09-11T06:49:48.945 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.945 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:48.945 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.945 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.945 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.945 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:114 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'76, mlcod=13'71)", 2024-09-11T06:49:48.945 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:49:48.945 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 42.462595725, 2024-09-11T06:49:48.945 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0042576660000000002, 2024-09-11T06:49:48.945 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:48.945 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:48.946 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:48.946 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:48.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.946 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:48.946 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:49:48.946 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.946 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:48.946 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:49:48.946 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.946 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:48.947 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454392+0000", 2024-09-11T06:49:48.947 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5999999999999991e-07 2024-09-11T06:49:48.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.947 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.947 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:48.947 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454399+0000", 2024-09-11T06:49:48.947 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.878e-06 2024-09-11T06:49:48.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.947 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.947 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:48.947 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454415+0000", 2024-09-11T06:49:48.947 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5659999999999999e-05 2024-09-11T06:49:48.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.947 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.948 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:48.948 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454422+0000", 2024-09-11T06:49:48.948 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:49:48.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.948 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:48.948 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454474+0000", 2024-09-11T06:49:48.948 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1860999999999997e-05 2024-09-11T06:49:48.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.948 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:48.948 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454497+0000", 2024-09-11T06:49:48.948 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3686e-05 2024-09-11T06:49:48.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.949 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:48.949 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458552+0000", 2024-09-11T06:49:48.949 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040544550000000002 2024-09-11T06:49:48.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.949 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:48.949 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458649+0000", 2024-09-11T06:49:48.949 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7005999999999999e-05 2024-09-11T06:49:48.949 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.949 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:48.949 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.950 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:115 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'77, mlcod=13'72)", 2024-09-11T06:49:48.950 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:48.950 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 42.461255113, 2024-09-11T06:49:48.950 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003355584, 2024-09-11T06:49:48.950 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:48.950 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:48.950 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:48.950 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:48.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.950 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:48.950 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:48.950 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.951 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:48.951 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:48.951 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.951 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:48.951 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:48.951 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5999999999999999e-07 2024-09-11T06:49:48.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.951 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:48.951 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455803+0000", 2024-09-11T06:49:48.951 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0310999999999998e-05 2024-09-11T06:49:48.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.952 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.952 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:48.952 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455815+0000", 2024-09-11T06:49:48.952 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2787e-05 2024-09-11T06:49:48.952 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.952 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.952 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:48.952 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455823+0000", 2024-09-11T06:49:48.952 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5959999999999998e-06 2024-09-11T06:49:48.952 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.952 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.952 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:48.952 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455875+0000", 2024-09-11T06:49:48.953 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2054e-05 2024-09-11T06:49:48.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.953 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:48.953 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455896+0000", 2024-09-11T06:49:48.953 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0554999999999999e-05 2024-09-11T06:49:48.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.953 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:48.953 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459000+0000", 2024-09-11T06:49:48.953 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003104842 2024-09-11T06:49:48.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.954 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:48.954 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459087+0000", 2024-09-11T06:49:48.954 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6978999999999995e-05 2024-09-11T06:49:48.954 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.954 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:48.954 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.954 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.954 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:120 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'82, mlcod=13'78)", 2024-09-11T06:49:48.954 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:49:48.954 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 42.450577828, 2024-09-11T06:49:48.954 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0064612549999999999, 2024-09-11T06:49:48.954 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:48.954 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:48.955 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:48.955 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:48.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.955 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:48.955 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:49:48.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.955 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:48.955 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:49:48.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.956 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:48.956 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466410+0000", 2024-09-11T06:49:48.956 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0389999999999999e-06 2024-09-11T06:49:48.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.956 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:48.956 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466496+0000", 2024-09-11T06:49:48.956 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5519000000000006e-05 2024-09-11T06:49:48.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.956 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:48.956 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466510+0000", 2024-09-11T06:49:48.956 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3842999999999999e-05 2024-09-11T06:49:48.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.957 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:48.957 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466517+0000", 2024-09-11T06:49:48.957 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5229999999999998e-06 2024-09-11T06:49:48.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.957 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:48.957 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466595+0000", 2024-09-11T06:49:48.957 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8019e-05 2024-09-11T06:49:48.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.957 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:48.957 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466612+0000", 2024-09-11T06:49:48.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7013000000000001e-05 2024-09-11T06:49:48.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.958 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:48.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472787+0000", 2024-09-11T06:49:48.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0061750269999999996 2024-09-11T06:49:48.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.958 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:48.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472870+0000", 2024-09-11T06:49:48.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3271999999999998e-05 2024-09-11T06:49:48.958 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.959 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:48.959 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.959 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.959 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:121 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'83, mlcod=13'78)", 2024-09-11T06:49:48.959 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:49:48.959 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 42.449353031999998, 2024-09-11T06:49:48.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055613069999999997, 2024-09-11T06:49:48.959 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:48.959 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:48.960 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:48.960 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:48.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.960 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:48.960 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:49:48.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.960 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:48.960 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:49:48.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.961 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.961 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:48.961 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467635+0000", 2024-09-11T06:49:48.961 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6899999999999996e-07 2024-09-11T06:49:48.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.961 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.961 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:48.961 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467727+0000", 2024-09-11T06:49:48.961 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1800999999999997e-05 2024-09-11T06:49:48.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.961 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.961 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:48.961 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467740+0000", 2024-09-11T06:49:48.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3747999999999999e-05 2024-09-11T06:49:48.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:48.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467753+0000", 2024-09-11T06:49:48.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2807e-05 2024-09-11T06:49:48.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:48.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467812+0000", 2024-09-11T06:49:48.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8381999999999998e-05 2024-09-11T06:49:48.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.963 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:48.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467828+0000", 2024-09-11T06:49:48.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6728e-05 2024-09-11T06:49:48.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.963 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:48.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473112+0000", 2024-09-11T06:49:48.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052833619999999998 2024-09-11T06:49:48.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.963 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:48.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473195+0000", 2024-09-11T06:49:48.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3609999999999994e-05 2024-09-11T06:49:48.964 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.964 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:48.964 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.964 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.964 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.964 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:131 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'89, mlcod=13'85)", 2024-09-11T06:49:48.964 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:49:48.964 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 42.097947316999999, 2024-09-11T06:49:48.964 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052492260000000001, 2024-09-11T06:49:48.964 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:48.964 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:48.964 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:48.964 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:48.965 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.965 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:48.965 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:49:48.965 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.965 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.965 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:48.965 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:49:48.965 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.965 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.965 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:48.965 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819041+0000", 2024-09-11T06:49:48.965 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4360000000000001e-06 2024-09-11T06:49:48.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.966 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:48.966 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819100+0000", 2024-09-11T06:49:48.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9225999999999998e-05 2024-09-11T06:49:48.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.966 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:48.966 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819128+0000", 2024-09-11T06:49:48.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.779e-05 2024-09-11T06:49:48.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.966 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:48.967 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819142+0000", 2024-09-11T06:49:48.967 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3573e-05 2024-09-11T06:49:48.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.967 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:48.967 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819263+0000", 2024-09-11T06:49:48.967 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000121086 2024-09-11T06:49:48.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.967 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:48.967 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819292+0000", 2024-09-11T06:49:48.967 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9363e-05 2024-09-11T06:49:48.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.968 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.968 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:48.968 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824185+0000", 2024-09-11T06:49:48.968 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0048924620000000002 2024-09-11T06:49:48.968 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.968 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.968 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:48.968 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824289+0000", 2024-09-11T06:49:48.968 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010429000000000001 2024-09-11T06:49:48.968 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.968 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:48.968 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.968 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.969 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:132 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'90, mlcod=13'85)", 2024-09-11T06:49:48.969 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:49:48.969 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 42.097172626999999, 2024-09-11T06:49:48.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.004532342, 2024-09-11T06:49:48.969 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:48.969 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:48.969 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:48.969 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:48.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.969 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:48.969 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:49:48.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.970 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:48.970 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:49:48.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.970 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:48.970 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819816+0000", 2024-09-11T06:49:48.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.61e-06 2024-09-11T06:49:48.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:48.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819827+0000", 2024-09-11T06:49:48.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1e-05 2024-09-11T06:49:48.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:48.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819851+0000", 2024-09-11T06:49:48.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4267e-05 2024-09-11T06:49:48.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:48.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819864+0000", 2024-09-11T06:49:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2767e-05 2024-09-11T06:49:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820071+0000", 2024-09-11T06:49:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000207061 2024-09-11T06:49:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820103+0000", 2024-09-11T06:49:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2326999999999997e-05 2024-09-11T06:49:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:48.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824302+0000", 2024-09-11T06:49:48.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0041986000000000002 2024-09-11T06:49:48.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:48.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824347+0000", 2024-09-11T06:49:48.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4709999999999997e-05 2024-09-11T06:49:48.973 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.973 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:48.973 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.973 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:133 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'91, mlcod=13'86)", 2024-09-11T06:49:48.973 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:49:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 42.096540967999999, 2024-09-11T06:49:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039421459999999997, 2024-09-11T06:49:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:49:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:49:48.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:48.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820448+0000", 2024-09-11T06:49:48.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.84e-06 2024-09-11T06:49:48.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:48.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820463+0000", 2024-09-11T06:49:48.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5356e-05 2024-09-11T06:49:48.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.976 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:48.976 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820499+0000", 2024-09-11T06:49:48.976 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.625e-05 2024-09-11T06:49:48.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.976 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.976 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:48.976 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820546+0000", 2024-09-11T06:49:48.976 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7017000000000001e-05 2024-09-11T06:49:48.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.976 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.976 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:48.976 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820882+0000", 2024-09-11T06:49:48.976 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033512 2024-09-11T06:49:48.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:48.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820941+0000", 2024-09-11T06:49:48.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8922999999999998e-05 2024-09-11T06:49:48.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:48.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824351+0000", 2024-09-11T06:49:48.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034101940000000001 2024-09-11T06:49:48.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824388+0000", 2024-09-11T06:49:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7446e-05 2024-09-11T06:49:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:134 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'92, mlcod=13'86)", 2024-09-11T06:49:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:49:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 42.095881849999998, 2024-09-11T06:49:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033248179999999998, 2024-09-11T06:49:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:49:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:49:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:48.980 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821107+0000", 2024-09-11T06:49:48.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7430000000000001e-06 2024-09-11T06:49:48.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.980 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.980 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:48.980 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821205+0000", 2024-09-11T06:49:48.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7666999999999998e-05 2024-09-11T06:49:48.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.980 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.980 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:48.980 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821234+0000", 2024-09-11T06:49:48.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9756000000000001e-05 2024-09-11T06:49:48.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.981 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:48.981 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821254+0000", 2024-09-11T06:49:48.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9446e-05 2024-09-11T06:49:48.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.981 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:48.981 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821846+0000", 2024-09-11T06:49:48.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00059187599999999997 2024-09-11T06:49:48.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.982 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:48.982 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821894+0000", 2024-09-11T06:49:48.982 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8766e-05 2024-09-11T06:49:48.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.982 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:48.982 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824392+0000", 2024-09-11T06:49:48.982 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0024977509999999999 2024-09-11T06:49:48.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.982 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:48.982 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824430+0000", 2024-09-11T06:49:48.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7812999999999999e-05 2024-09-11T06:49:48.983 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.983 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:48.983 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.983 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:137 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'95, mlcod=13'86)", 2024-09-11T06:49:48.983 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:49:48.983 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 42.093419709000003, 2024-09-11T06:49:48.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003084299, 2024-09-11T06:49:48.983 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:48.983 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:48.983 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:48.983 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:48.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.984 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:48.984 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:49:48.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.984 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:48.984 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:49:48.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.984 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:48.984 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823568+0000", 2024-09-11T06:49:48.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1030000000000001e-06 2024-09-11T06:49:48.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:48.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823662+0000", 2024-09-11T06:49:48.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3529999999999994e-05 2024-09-11T06:49:48.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:48.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823686+0000", 2024-09-11T06:49:48.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4358999999999998e-05 2024-09-11T06:49:48.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:48.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823701+0000", 2024-09-11T06:49:48.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4847e-05 2024-09-11T06:49:48.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:48.986 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824518+0000", 2024-09-11T06:49:48.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00081716700000000005 2024-09-11T06:49:48.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:48.986 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824546+0000", 2024-09-11T06:49:48.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7860000000000001e-05 2024-09-11T06:49:48.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:48.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826558+0000", 2024-09-11T06:49:48.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0020120009999999998 2024-09-11T06:49:48.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.987 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:48.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826652+0000", 2024-09-11T06:49:48.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3431999999999996e-05 2024-09-11T06:49:48.987 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.987 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:48.987 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.987 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:139 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'97, mlcod=13'86)", 2024-09-11T06:49:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:49:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 42.091130990000003, 2024-09-11T06:49:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003732221, 2024-09-11T06:49:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:49:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:48.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:49:48.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:48.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825858+0000", 2024-09-11T06:49:48.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5829999999999999e-06 2024-09-11T06:49:48.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:48.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825874+0000", 2024-09-11T06:49:48.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5967000000000001e-05 2024-09-11T06:49:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825909+0000", 2024-09-11T06:49:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5692999999999999e-05 2024-09-11T06:49:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825923+0000", 2024-09-11T06:49:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4080000000000001e-05 2024-09-11T06:49:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826010+0000", 2024-09-11T06:49:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6248999999999994e-05 2024-09-11T06:49:48.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.991 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:48.991 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826038+0000", 2024-09-11T06:49:48.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8010000000000001e-05 2024-09-11T06:49:48.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.991 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:48.991 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829499+0000", 2024-09-11T06:49:48.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003461783 2024-09-11T06:49:48.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.991 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829588+0000", 2024-09-11T06:49:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8856000000000004e-05 2024-09-11T06:49:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:140 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'98, mlcod=13'93)", 2024-09-11T06:49:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:49:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 42.089280260999999, 2024-09-11T06:49:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0057995929999999996, 2024-09-11T06:49:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:48.993 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:48.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.993 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:48.993 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:49:48.993 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.993 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:48.993 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:49:48.993 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.993 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:48.993 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827708+0000", 2024-09-11T06:49:48.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.547e-06 2024-09-11T06:49:48.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.994 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:48.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827765+0000", 2024-09-11T06:49:48.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6266e-05 2024-09-11T06:49:48.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.994 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:48.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827791+0000", 2024-09-11T06:49:48.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6098999999999999e-05 2024-09-11T06:49:48.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827806+0000", 2024-09-11T06:49:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5537e-05 2024-09-11T06:49:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827928+0000", 2024-09-11T06:49:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000122012 2024-09-11T06:49:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827961+0000", 2024-09-11T06:49:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2663999999999998e-05 2024-09-11T06:49:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:48.996 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833409+0000", 2024-09-11T06:49:48.996 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054479350000000001 2024-09-11T06:49:48.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:48.996 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833506+0000", 2024-09-11T06:49:48.996 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7533000000000003e-05 2024-09-11T06:49:48.996 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.996 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:48.996 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:48.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.997 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:141 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'99, mlcod=13'93)", 2024-09-11T06:49:48.997 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:49:48.997 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 42.088386878000001, 2024-09-11T06:49:48.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055394709999999998, 2024-09-11T06:49:48.997 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:48.997 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:48.997 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:48.997 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:48.997 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.997 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:48.997 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:49:48.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:48.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:49:48.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:48.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:48.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828602+0000", 2024-09-11T06:49:48.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.46e-06 2024-09-11T06:49:48.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:48.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828618+0000", 2024-09-11T06:49:48.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6592999999999999e-05 2024-09-11T06:49:48.999 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.999 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:48.999 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828646+0000", 2024-09-11T06:49:48.999 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7552999999999999e-05 2024-09-11T06:49:48.999 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.999 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:48.999 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828663+0000", 2024-09-11T06:49:48.999 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.719e-05 2024-09-11T06:49:48.999 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:48.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:48.999 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:48.999 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829023+0000", 2024-09-11T06:49:49.000 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036054799999999998 2024-09-11T06:49:49.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.000 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:49.000 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829062+0000", 2024-09-11T06:49:49.000 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8865999999999997e-05 2024-09-11T06:49:49.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.000 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:49.000 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834078+0000", 2024-09-11T06:49:49.000 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050157480000000004 2024-09-11T06:49:49.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.000 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:49.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834140+0000", 2024-09-11T06:49:49.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1513000000000005e-05 2024-09-11T06:49:49.001 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:49.001 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:49.001 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:49.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.001 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:144 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'102, mlcod=13'98)", 2024-09-11T06:49:49.001 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:49:49.001 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 42.079226265000003, 2024-09-11T06:49:49.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003210348, 2024-09-11T06:49:49.001 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:49.001 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:49.001 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:49.001 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:49.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:49.002 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:49:49.002 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:49.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:49.002 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:49:49.002 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:49.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:49.002 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837768+0000", 2024-09-11T06:49:49.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9330000000000004e-06 2024-09-11T06:49:49.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:49.003 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837858+0000", 2024-09-11T06:49:49.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0185999999999993e-05 2024-09-11T06:49:49.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:49.003 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837890+0000", 2024-09-11T06:49:49.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2437000000000001e-05 2024-09-11T06:49:49.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:49.004 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837905+0000", 2024-09-11T06:49:49.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:49:49.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.004 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:49.004 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838019+0000", 2024-09-11T06:49:49.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000114336 2024-09-11T06:49:49.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.004 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:49.004 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838052+0000", 2024-09-11T06:49:49.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3139000000000002e-05 2024-09-11T06:49:49.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:49.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840914+0000", 2024-09-11T06:49:49.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028612780000000001 2024-09-11T06:49:49.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:49.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840971+0000", 2024-09-11T06:49:49.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7402999999999999e-05 2024-09-11T06:49:49.005 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:49.005 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:49.005 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:49.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.006 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:145 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'103, mlcod=13'98)", 2024-09-11T06:49:49.006 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:49:49.006 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 42.078434381000001, 2024-09-11T06:49:49.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054455409999999999, 2024-09-11T06:49:49.006 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:49.006 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:49.006 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:49.006 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:49.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.006 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:49.006 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:49:49.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:49.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:49:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838554+0000", 2024-09-11T06:49:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3430000000000001e-06 2024-09-11T06:49:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838640+0000", 2024-09-11T06:49:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5569999999999996e-05 2024-09-11T06:49:49.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.008 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:49.008 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838660+0000", 2024-09-11T06:49:49.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0576000000000001e-05 2024-09-11T06:49:49.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.008 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:49.008 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838674+0000", 2024-09-11T06:49:49.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3529999999999999e-05 2024-09-11T06:49:49.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.008 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:49.009 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838838+0000", 2024-09-11T06:49:49.009 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016468799999999999 2024-09-11T06:49:49.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.009 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:49.009 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838885+0000", 2024-09-11T06:49:49.009 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6884e-05 2024-09-11T06:49:49.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.009 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:49.009 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843901+0000", 2024-09-11T06:49:49.009 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050162310000000003 2024-09-11T06:49:49.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:49.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843998+0000", 2024-09-11T06:49:49.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.6718999999999993e-05 2024-09-11T06:49:49.010 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:49.010 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:49.010 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:49.010 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:49.010 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:49.010 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:49.012 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:49:49.012 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:49.014+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:49.112 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:49.109+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:49.129 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:49.130 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:49.130 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:49:49.130 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:49.146 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:49:49.232 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:49.229+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:49.305 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:49.306 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:49.306 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:49:49.306 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:49:49.306 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:49.315 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:49.314+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:49.318 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:49:49.416 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:49.418+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:49.473 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:49.474 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:49:49.474 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:49:49.474 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:49:49.474 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.474 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:5 2.2 e17/15 2:5cea7035:::rbd_info:head v 17'1, mlcod=0'0)", 2024-09-11T06:49:49.474 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:49:49.474 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.238553242000002, 2024-09-11T06:49:49.474 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0019446719999999999, 2024-09-11T06:49:49.474 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:49.474 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:49.474 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:49.475 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:49.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.475 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:49.475 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:49:49.475 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:49.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.475 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:49.475 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:49:49.475 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:49.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.475 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:49.475 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236085+0000", 2024-09-11T06:49:49.476 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6000000000000003e-07 2024-09-11T06:49:49.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.476 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.476 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:49.476 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236108+0000", 2024-09-11T06:49:49.476 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3113e-05 2024-09-11T06:49:49.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.476 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.476 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:49.476 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236133+0000", 2024-09-11T06:49:49.476 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5723000000000001e-05 2024-09-11T06:49:49.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.477 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:49.477 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236154+0000", 2024-09-11T06:49:49.477 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0333999999999999e-05 2024-09-11T06:49:49.477 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.477 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:49.477 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236297+0000", 2024-09-11T06:49:49.477 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014278500000000001 2024-09-11T06:49:49.477 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.477 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:49.477 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236383+0000", 2024-09-11T06:49:49.478 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5927999999999995e-05 2024-09-11T06:49:49.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.478 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.478 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:49.478 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.237950+0000", 2024-09-11T06:49:49.478 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015673690000000001 2024-09-11T06:49:49.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.478 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.478 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:49.478 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.238029+0000", 2024-09-11T06:49:49.478 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8659999999999996e-05 2024-09-11T06:49:49.478 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:49.479 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:49.479 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:49.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.479 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.479 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:7 2.2 e18/15 2:5cea7035:::rbd_info:head v 18'2, mlcod=0'0)", 2024-09-11T06:49:49.479 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:49:49.479 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 36.224228035000003, 2024-09-11T06:49:49.479 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001907192, 2024-09-11T06:49:49.479 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:49.479 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:49.479 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:49.480 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:49.480 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.480 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:49.480 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:49:49.480 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:49.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.480 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.481 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:49.481 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:49:49.481 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:49.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.481 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.481 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:49.481 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250410+0000", 2024-09-11T06:49:49.481 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7700000000000004e-07 2024-09-11T06:49:49.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.481 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250445+0000", 2024-09-11T06:49:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4749000000000002e-05 2024-09-11T06:49:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250478+0000", 2024-09-11T06:49:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3269999999999998e-05 2024-09-11T06:49:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250501+0000", 2024-09-11T06:49:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3632999999999999e-05 2024-09-11T06:49:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.483 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.483 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:49.483 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250633+0000", 2024-09-11T06:49:49.483 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013143200000000001 2024-09-11T06:49:49.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.483 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.483 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:49.483 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250700+0000", 2024-09-11T06:49:49.483 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7002999999999995e-05 2024-09-11T06:49:49.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.483 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.484 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:49.484 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.252172+0000", 2024-09-11T06:49:49.484 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014719550000000001 2024-09-11T06:49:49.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.484 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.484 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:49.484 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.252316+0000", 2024-09-11T06:49:49.484 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144473 2024-09-11T06:49:49.484 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:49.484 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:49.484 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:49.484 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:49.484 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:49.485 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:49.486 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:49:49.517 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:49.514+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:49.520 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.0 to 0.146956 (259e) 2024-09-11T06:49:49.547 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T06:49:49.605 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:49.605 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:49.605 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:49:49.605 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:49.618 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:49.618+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:49.619 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:49:49.718 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:49.718+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:49.783 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:49.783 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:49.783 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:49:49.783 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:49:49.783 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:49.798 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:49:49.844 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:49.841+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:49.906 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:49.907 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:49:49.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:49:49.907 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:49:49.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.907 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:1 2.3 e16/15 2:c4c92e5a:::rbd_trash:head v 16'1, mlcod=0'0)", 2024-09-11T06:49:49.907 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:49:49.907 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 38.671334539, 2024-09-11T06:49:49.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001234761, 2024-09-11T06:49:49.907 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:49.907 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:49.907 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:49.907 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:49.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:49.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:49:49.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:49.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:49.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:49:49.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:49.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:49.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:49:49.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6000000000000003e-07 2024-09-11T06:49:49.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.909 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:49.909 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236296+0000", 2024-09-11T06:49:49.909 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6393999999999999e-05 2024-09-11T06:49:49.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.909 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:49.909 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236317+0000", 2024-09-11T06:49:49.909 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1243e-05 2024-09-11T06:49:49.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.909 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:49.909 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236333+0000", 2024-09-11T06:49:49.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.558e-05 2024-09-11T06:49:49.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.910 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:49.910 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236435+0000", 2024-09-11T06:49:49.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000102803 2024-09-11T06:49:49.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.910 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:49.910 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236474+0000", 2024-09-11T06:49:49.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8383999999999997e-05 2024-09-11T06:49:49.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.910 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:49.911 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.237432+0000", 2024-09-11T06:49:49.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00095849300000000004 2024-09-11T06:49:49.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:49.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:49.911 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:49.911 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.237513+0000", 2024-09-11T06:49:49.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1204000000000004e-05 2024-09-11T06:49:49.911 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:49.911 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:49.911 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:49.911 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:49.911 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:49.911 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:49.919 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:49:49.964 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:49.961+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:50.022 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:50.018+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:50.071 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:50.071 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:50.071 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:49:50.071 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:50.085 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:49:50.130 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:50.130+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:50.223 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:50.221+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:50.248 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:50.248 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:50.248 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:49:50.248 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:49:50.248 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:50.262 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:49:50.324 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:50.322+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:50.383 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:50.384 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:49:50.384 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:49:50.384 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:49:50.384 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.384 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:50.384 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:49:50.384 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 44.213275256999999, 2024-09-11T06:49:50.384 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011420669, 2024-09-11T06:49:50.384 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:50.384 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:50.384 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:50.384 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:50.385 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:50.385 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:50.385 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 37 2024-09-11T06:49:50.385 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.385 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:50.385 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.385 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:50.385 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:49:50.385 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.385 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.385 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.385 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:50.386 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:49:50.386 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.386 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.386 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.386 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:50.386 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170509+0000", 2024-09-11T06:49:50.386 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.13e-06 2024-09-11T06:49:50.386 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.386 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.386 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:50.386 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170521+0000", 2024-09-11T06:49:50.387 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1776999999999999e-05 2024-09-11T06:49:50.387 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.387 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.387 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:50.387 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170538+0000", 2024-09-11T06:49:50.387 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7413000000000001e-05 2024-09-11T06:49:50.387 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.387 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.387 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:50.387 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170570+0000", 2024-09-11T06:49:50.388 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2313e-05 2024-09-11T06:49:50.388 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.388 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.388 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:50.388 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179768+0000", 2024-09-11T06:49:50.388 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0091978010000000002 2024-09-11T06:49:50.388 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.389 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.389 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:50.389 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179951+0000", 2024-09-11T06:49:50.389 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018227900000000001 2024-09-11T06:49:50.389 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.389 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.389 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:50.389 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.180175+0000", 2024-09-11T06:49:50.389 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022432800000000001 2024-09-11T06:49:50.389 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.389 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.389 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:50.390 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181112+0000", 2024-09-11T06:49:50.390 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00093712500000000002 2024-09-11T06:49:50.390 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.390 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.390 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:50.390 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181789+0000", 2024-09-11T06:49:50.390 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00067705100000000002 2024-09-11T06:49:50.390 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.390 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.390 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:50.390 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181871+0000", 2024-09-11T06:49:50.390 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1816000000000004e-05 2024-09-11T06:49:50.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.391 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.391 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:50.391 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181929+0000", 2024-09-11T06:49:50.391 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7636000000000002e-05 2024-09-11T06:49:50.391 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.391 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:50.391 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.391 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.391 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:50.391 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:49:50.391 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 44.112016296, 2024-09-11T06:49:50.391 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283336, 2024-09-11T06:49:50.392 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:50.392 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:50.392 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:50.392 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:50.392 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:50.392 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:50.392 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 87 2024-09-11T06:49:50.392 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.392 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:50.392 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.392 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:50.393 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:49:50.393 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.393 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.393 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.393 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:50.393 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:49:50.393 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.393 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.393 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.393 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:50.393 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271768+0000", 2024-09-11T06:49:50.394 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7599999999999996e-07 2024-09-11T06:49:50.394 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.394 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.394 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:50.394 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271860+0000", 2024-09-11T06:49:50.394 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2639999999999994e-05 2024-09-11T06:49:50.394 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.394 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.394 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:50.394 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271873+0000", 2024-09-11T06:49:50.394 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3139000000000001e-05 2024-09-11T06:49:50.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.395 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.395 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:50.395 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271882+0000", 2024-09-11T06:49:50.395 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2369999999999992e-06 2024-09-11T06:49:50.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.395 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.395 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:50.395 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277150+0000", 2024-09-11T06:49:50.395 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052684419999999999 2024-09-11T06:49:50.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.395 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.395 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:50.395 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277273+0000", 2024-09-11T06:49:50.396 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012278899999999999 2024-09-11T06:49:50.396 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.396 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:50.396 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277466+0000", 2024-09-11T06:49:50.396 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193578 2024-09-11T06:49:50.396 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.396 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:50.397 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.280808+0000", 2024-09-11T06:49:50.397 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003341971 2024-09-11T06:49:50.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.397 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.397 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:50.397 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284419+0000", 2024-09-11T06:49:50.397 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003610982 2024-09-11T06:49:50.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.397 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.397 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:50.398 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284531+0000", 2024-09-11T06:49:50.398 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011121299999999999 2024-09-11T06:49:50.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.398 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.398 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:50.398 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284600+0000", 2024-09-11T06:49:50.398 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9592999999999996e-05 2024-09-11T06:49:50.398 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.398 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:50.398 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.398 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.398 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:50.398 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:49:50.399 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 44.111886044000002, 2024-09-11T06:49:50.399 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283314, 2024-09-11T06:49:50.399 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:50.399 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:50.399 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:50.399 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:50.399 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:50.399 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:50.399 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 88 2024-09-11T06:49:50.400 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.400 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:50.400 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.400 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:50.400 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:49:50.400 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.400 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.400 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.400 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:50.400 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:49:50.400 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.401 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:50.401 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271898+0000", 2024-09-11T06:49:50.401 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2999999999999999e-07 2024-09-11T06:49:50.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.401 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:50.401 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271904+0000", 2024-09-11T06:49:50.401 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.767e-06 2024-09-11T06:49:50.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.401 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:50.401 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271913+0000", 2024-09-11T06:49:50.402 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7200000000000001e-06 2024-09-11T06:49:50.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.402 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:50.402 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271920+0000", 2024-09-11T06:49:50.402 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7100000000000001e-06 2024-09-11T06:49:50.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.402 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:50.402 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278684+0000", 2024-09-11T06:49:50.402 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0067642650000000002 2024-09-11T06:49:50.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.402 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:50.403 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278838+0000", 2024-09-11T06:49:50.403 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015371499999999999 2024-09-11T06:49:50.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.403 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.403 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:50.403 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.279040+0000", 2024-09-11T06:49:50.403 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020207499999999999 2024-09-11T06:49:50.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.403 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.403 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:50.403 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.281448+0000", 2024-09-11T06:49:50.404 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002407686 2024-09-11T06:49:50.404 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.404 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.404 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:50.404 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284615+0000", 2024-09-11T06:49:50.404 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031675499999999999 2024-09-11T06:49:50.404 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.404 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.405 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:50.405 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284682+0000", 2024-09-11T06:49:50.405 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7032000000000005e-05 2024-09-11T06:49:50.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.405 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.405 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:50.405 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284730+0000", 2024-09-11T06:49:50.405 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7790000000000002e-05 2024-09-11T06:49:50.405 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.405 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:50.405 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.405 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.406 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:50.406 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:49:50.406 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 43.943984821999997, 2024-09-11T06:49:50.406 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011114877, 2024-09-11T06:49:50.406 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:50.406 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:50.406 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:50.406 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:50.406 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:50.406 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:50.406 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 107 2024-09-11T06:49:50.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.407 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:50.407 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.407 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:50.407 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:49:50.407 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.407 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.407 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:50.407 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:49:50.407 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.407 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.407 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:50.408 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439799+0000", 2024-09-11T06:49:50.408 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.16e-07 2024-09-11T06:49:50.408 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.408 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.408 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:50.408 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439806+0000", 2024-09-11T06:49:50.408 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3640000000000004e-06 2024-09-11T06:49:50.408 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.408 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.408 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:50.408 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439816+0000", 2024-09-11T06:49:50.408 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0485999999999999e-05 2024-09-11T06:49:50.408 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.409 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.409 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:50.409 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439824+0000", 2024-09-11T06:49:50.409 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.6699999999999994e-06 2024-09-11T06:49:50.409 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.409 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.409 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:50.409 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443328+0000", 2024-09-11T06:49:50.409 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035044329999999999 2024-09-11T06:49:50.409 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.409 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.409 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:50.409 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443461+0000", 2024-09-11T06:49:50.409 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013272199999999999 2024-09-11T06:49:50.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.410 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.410 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:50.410 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443678+0000", 2024-09-11T06:49:50.410 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021674500000000001 2024-09-11T06:49:50.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.410 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.410 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:50.410 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449635+0000", 2024-09-11T06:49:50.410 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0059572119999999999 2024-09-11T06:49:50.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.411 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.411 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:50.411 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450750+0000", 2024-09-11T06:49:50.411 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001115654 2024-09-11T06:49:50.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.411 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.411 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:50.411 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450832+0000", 2024-09-11T06:49:50.411 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1188999999999998e-05 2024-09-11T06:49:50.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.411 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.411 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:50.411 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450913+0000", 2024-09-11T06:49:50.412 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1485999999999999e-05 2024-09-11T06:49:50.412 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.412 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:50.412 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.412 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.412 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:50.412 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:49:50.412 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 43.943393847000003, 2024-09-11T06:49:50.412 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011236556, 2024-09-11T06:49:50.412 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:50.412 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:50.412 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:50.412 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:50.413 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:50.413 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:50.413 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 109 2024-09-11T06:49:50.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.413 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:50.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.413 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:50.413 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:49:50.413 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.413 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:50.414 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:49:50.414 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.414 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.414 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:50.414 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440391+0000", 2024-09-11T06:49:50.414 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8360000000000001e-06 2024-09-11T06:49:50.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.414 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.414 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:50.414 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440560+0000", 2024-09-11T06:49:50.414 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000169085 2024-09-11T06:49:50.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.414 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.415 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:50.415 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440585+0000", 2024-09-11T06:49:50.415 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4530000000000001e-05 2024-09-11T06:49:50.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.415 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.415 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:50.415 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440606+0000", 2024-09-11T06:49:50.415 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1277e-05 2024-09-11T06:49:50.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.415 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.415 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:50.415 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446479+0000", 2024-09-11T06:49:50.416 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0058734989999999999 2024-09-11T06:49:50.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.416 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.416 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:50.416 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446649+0000", 2024-09-11T06:49:50.416 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016948199999999999 2024-09-11T06:49:50.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.416 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.416 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:50.417 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446854+0000", 2024-09-11T06:49:50.417 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020491500000000001 2024-09-11T06:49:50.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.417 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.417 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:50.417 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451380+0000", 2024-09-11T06:49:50.417 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0045264110000000001 2024-09-11T06:49:50.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.417 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.417 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:50.417 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451479+0000", 2024-09-11T06:49:50.417 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.8975999999999996e-05 2024-09-11T06:49:50.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.418 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.418 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:50.418 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451562+0000", 2024-09-11T06:49:50.418 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3213000000000004e-05 2024-09-11T06:49:50.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.418 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.418 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:50.418 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451626+0000", 2024-09-11T06:49:50.418 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3331999999999996e-05 2024-09-11T06:49:50.418 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.418 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:50.418 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.419 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.419 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:50.419 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:49:50.419 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 43.943144719000003, 2024-09-11T06:49:50.419 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011252866, 2024-09-11T06:49:50.419 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:50.419 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:50.419 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:50.419 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:50.419 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:50.419 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:50.419 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 110 2024-09-11T06:49:50.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.419 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:50.420 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.420 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:50.420 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:49:50.420 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.420 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.420 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:50.420 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:49:50.420 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.420 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.420 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:50.421 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440640+0000", 2024-09-11T06:49:50.421 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5430000000000001e-06 2024-09-11T06:49:50.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.421 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.421 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:50.421 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440651+0000", 2024-09-11T06:49:50.421 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0672999999999999e-05 2024-09-11T06:49:50.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.421 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.421 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:50.421 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440670+0000", 2024-09-11T06:49:50.421 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9726999999999999e-05 2024-09-11T06:49:50.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.422 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.422 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:50.422 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440691+0000", 2024-09-11T06:49:50.422 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0526e-05 2024-09-11T06:49:50.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.422 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.422 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:50.422 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448191+0000", 2024-09-11T06:49:50.422 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0075005990000000002 2024-09-11T06:49:50.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.422 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.422 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:50.422 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448390+0000", 2024-09-11T06:49:50.423 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000198815 2024-09-11T06:49:50.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.423 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.423 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:50.423 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448735+0000", 2024-09-11T06:49:50.423 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034424999999999999 2024-09-11T06:49:50.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.423 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.423 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:50.423 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451742+0000", 2024-09-11T06:49:50.423 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030077440000000001 2024-09-11T06:49:50.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.423 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.423 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:50.424 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451761+0000", 2024-09-11T06:49:50.424 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9196999999999999e-05 2024-09-11T06:49:50.424 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.424 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.424 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:50.424 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451834+0000", 2024-09-11T06:49:50.424 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2071999999999997e-05 2024-09-11T06:49:50.424 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.424 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.424 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:50.424 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451891+0000", 2024-09-11T06:49:50.424 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7720000000000003e-05 2024-09-11T06:49:50.424 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.425 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:50.425 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.425 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.425 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:50.425 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:49:50.425 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 43.943036843999998, 2024-09-11T06:49:50.425 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014944941, 2024-09-11T06:49:50.425 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:50.426 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:50.426 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:50.426 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:50.426 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:50.426 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:50.426 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 111 2024-09-11T06:49:50.426 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.426 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:50.426 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.426 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:50.426 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:49:50.426 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.426 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.426 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.426 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:50.427 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:49:50.427 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.427 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.427 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:50.427 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440748+0000", 2024-09-11T06:49:50.427 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:49:50.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.427 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.427 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:50.427 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440760+0000", 2024-09-11T06:49:50.427 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1897e-05 2024-09-11T06:49:50.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.428 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.428 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:50.428 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440782+0000", 2024-09-11T06:49:50.428 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2552999999999999e-05 2024-09-11T06:49:50.428 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.428 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.428 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:50.428 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440793+0000", 2024-09-11T06:49:50.428 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0643e-05 2024-09-11T06:49:50.428 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.428 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.428 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:50.428 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449755+0000", 2024-09-11T06:49:50.428 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089622620000000004 2024-09-11T06:49:50.428 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.429 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.429 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:50.429 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449918+0000", 2024-09-11T06:49:50.429 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163086 2024-09-11T06:49:50.429 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.429 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.429 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:50.429 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450150+0000", 2024-09-11T06:49:50.429 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023204500000000001 2024-09-11T06:49:50.429 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.429 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.429 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:50.429 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454533+0000", 2024-09-11T06:49:50.430 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0043831349999999998 2024-09-11T06:49:50.430 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.430 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.430 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:50.430 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455511+0000", 2024-09-11T06:49:50.430 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0009776050000000001 2024-09-11T06:49:50.430 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.430 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.430 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:50.430 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455597+0000", 2024-09-11T06:49:50.430 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6446000000000005e-05 2024-09-11T06:49:50.430 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.430 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.430 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:50.431 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455691+0000", 2024-09-11T06:49:50.431 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3832000000000006e-05 2024-09-11T06:49:50.431 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.431 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:50.431 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.431 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.431 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.431 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:50.431 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:49:50.431 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 43.942967080999999, 2024-09-11T06:49:50.431 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018885411000000001, 2024-09-11T06:49:50.431 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:50.431 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:50.431 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:50.431 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:50.432 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:50.432 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:50.432 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 112 2024-09-11T06:49:50.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.432 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:50.432 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.432 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:50.432 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:49:50.432 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.432 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.432 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:50.432 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:49:50.433 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.433 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.433 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:50.433 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440817+0000", 2024-09-11T06:49:50.433 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:49:50.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.433 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.433 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:50.433 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441003+0000", 2024-09-11T06:49:50.433 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018661200000000001 2024-09-11T06:49:50.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.433 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.433 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:50.434 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441027+0000", 2024-09-11T06:49:50.434 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3920000000000001e-05 2024-09-11T06:49:50.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.434 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.434 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:50.434 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441062+0000", 2024-09-11T06:49:50.434 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4663000000000003e-05 2024-09-11T06:49:50.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.434 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.434 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:50.434 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451981+0000", 2024-09-11T06:49:50.434 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010918926000000001 2024-09-11T06:49:50.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.434 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.434 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:50.434 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452111+0000", 2024-09-11T06:49:50.435 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129529 2024-09-11T06:49:50.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.435 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.435 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:50.435 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452311+0000", 2024-09-11T06:49:50.435 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020013799999999999 2024-09-11T06:49:50.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.435 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.435 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:50.435 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455809+0000", 2024-09-11T06:49:50.435 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003498443 2024-09-11T06:49:50.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.436 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.436 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:50.436 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459566+0000", 2024-09-11T06:49:50.436 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003757271 2024-09-11T06:49:50.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.436 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.436 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:50.436 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459640+0000", 2024-09-11T06:49:50.436 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3949000000000006e-05 2024-09-11T06:49:50.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.437 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.437 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:50.437 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459701+0000", 2024-09-11T06:49:50.437 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1117000000000005e-05 2024-09-11T06:49:50.437 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.437 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:50.437 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.437 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.437 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.437 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:50.437 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:49:50.437 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 43.942688230000002, 2024-09-11T06:49:50.437 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018760915, 2024-09-11T06:49:50.438 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:50.438 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:50.438 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:50.438 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:50.438 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:50.438 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:50.438 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 113 2024-09-11T06:49:50.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.438 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:50.438 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.438 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:50.438 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:49:50.438 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.439 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.439 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:50.439 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:49:50.439 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.439 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.439 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:50.439 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441096+0000", 2024-09-11T06:49:50.439 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4130000000000001e-06 2024-09-11T06:49:50.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.439 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.439 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:50.439 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441110+0000", 2024-09-11T06:49:50.439 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3900000000000001e-05 2024-09-11T06:49:50.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.440 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.440 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:50.440 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441131+0000", 2024-09-11T06:49:50.440 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0636999999999999e-05 2024-09-11T06:49:50.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.440 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.440 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:50.440 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441147+0000", 2024-09-11T06:49:50.440 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6512999999999999e-05 2024-09-11T06:49:50.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.440 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.440 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:50.440 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453274+0000", 2024-09-11T06:49:50.441 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012126764999999999 2024-09-11T06:49:50.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.441 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.441 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:50.441 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453498+0000", 2024-09-11T06:49:50.441 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000223568 2024-09-11T06:49:50.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.441 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.441 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:50.441 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453872+0000", 2024-09-11T06:49:50.441 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037467099999999998 2024-09-11T06:49:50.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.442 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.442 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:50.442 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458709+0000", 2024-09-11T06:49:50.442 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0048365680000000003 2024-09-11T06:49:50.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.442 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.442 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:50.442 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459716+0000", 2024-09-11T06:49:50.442 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010075240000000001 2024-09-11T06:49:50.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.442 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.442 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:50.442 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459779+0000", 2024-09-11T06:49:50.442 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2336999999999996e-05 2024-09-11T06:49:50.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.443 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:50.443 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459856+0000", 2024-09-11T06:49:50.443 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7019000000000002e-05 2024-09-11T06:49:50.443 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.443 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:50.443 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.443 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:50.443 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:50.443 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 43.942611491000001, 2024-09-11T06:49:50.443 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018963114, 2024-09-11T06:49:50.443 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:50.443 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:50.444 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:50.444 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:50.444 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:50.444 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:50.444 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 114 2024-09-11T06:49:50.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.444 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:50.444 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.444 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:50.444 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:50.444 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.444 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.444 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:50.444 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:50.445 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.445 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:50.445 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:50.445 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9299999999999996e-07 2024-09-11T06:49:50.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.445 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:50.445 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441180+0000", 2024-09-11T06:49:50.445 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7200000000000006e-06 2024-09-11T06:49:50.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.445 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:50.446 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441265+0000", 2024-09-11T06:49:50.446 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4753000000000007e-05 2024-09-11T06:49:50.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.446 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.446 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:50.446 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441288+0000", 2024-09-11T06:49:50.446 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2860000000000001e-05 2024-09-11T06:49:50.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.446 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.446 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:50.446 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454585+0000", 2024-09-11T06:49:50.447 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013297286 2024-09-11T06:49:50.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.447 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:50.447 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454714+0000", 2024-09-11T06:49:50.447 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000128465 2024-09-11T06:49:50.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.447 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:50.447 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454909+0000", 2024-09-11T06:49:50.447 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019522499999999999 2024-09-11T06:49:50.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.447 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:50.448 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459868+0000", 2024-09-11T06:49:50.448 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0049591239999999997 2024-09-11T06:49:50.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.448 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:50.448 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459988+0000", 2024-09-11T06:49:50.448 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000120532 2024-09-11T06:49:50.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.448 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:50.448 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460071+0000", 2024-09-11T06:49:50.448 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2256000000000007e-05 2024-09-11T06:49:50.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.449 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:50.449 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460135+0000", 2024-09-11T06:49:50.449 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3999999999999997e-05 2024-09-11T06:49:50.449 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.449 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:50.449 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.449 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:50.449 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:49:50.449 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 43.942453862000001, 2024-09-11T06:49:50.449 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.019036849000000002, 2024-09-11T06:49:50.450 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:50.450 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:50.450 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:50.450 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:50.450 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:50.450 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:50.450 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 115 2024-09-11T06:49:50.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.450 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:50.450 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.450 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:50.451 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:49:50.451 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.451 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.451 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:50.451 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:49:50.451 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.451 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.451 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:50.451 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441330+0000", 2024-09-11T06:49:50.452 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1930000000000001e-06 2024-09-11T06:49:50.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.452 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:50.452 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441544+0000", 2024-09-11T06:49:50.452 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021405799999999999 2024-09-11T06:49:50.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.452 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:50.452 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441572+0000", 2024-09-11T06:49:50.452 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7317e-05 2024-09-11T06:49:50.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.452 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:50.453 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441593+0000", 2024-09-11T06:49:50.453 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1702999999999999e-05 2024-09-11T06:49:50.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.453 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:50.453 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455891+0000", 2024-09-11T06:49:50.453 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01429708 2024-09-11T06:49:50.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.453 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:50.453 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456046+0000", 2024-09-11T06:49:50.453 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000155682 2024-09-11T06:49:50.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.454 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:50.454 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456242+0000", 2024-09-11T06:49:50.454 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000195692 2024-09-11T06:49:50.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.454 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:50.454 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459876+0000", 2024-09-11T06:49:50.454 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036336480000000002 2024-09-11T06:49:50.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.455 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:50.455 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460245+0000", 2024-09-11T06:49:50.455 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036901700000000002 2024-09-11T06:49:50.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.455 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:50.455 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460316+0000", 2024-09-11T06:49:50.455 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1846000000000003e-05 2024-09-11T06:49:50.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.455 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:50.455 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460366+0000", 2024-09-11T06:49:50.456 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9613000000000001e-05 2024-09-11T06:49:50.456 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.456 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:50.456 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.456 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:50.456 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:49:50.456 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 43.942131781999997, 2024-09-11T06:49:50.456 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01904432, 2024-09-11T06:49:50.456 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:50.456 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:50.456 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:50.457 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:50.457 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:50.457 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:50.457 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 116 2024-09-11T06:49:50.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.457 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:50.457 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.458 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:50.458 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:49:50.458 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.458 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.458 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:50.458 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:49:50.458 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.458 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.458 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:50.458 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441653+0000", 2024-09-11T06:49:50.458 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:49:50.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.459 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:50.459 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441665+0000", 2024-09-11T06:49:50.459 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1653e-05 2024-09-11T06:49:50.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.459 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:50.459 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441684+0000", 2024-09-11T06:49:50.459 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9490000000000001e-05 2024-09-11T06:49:50.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.460 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.460 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:50.460 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441699+0000", 2024-09-11T06:49:50.460 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5044e-05 2024-09-11T06:49:50.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.460 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.460 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:50.460 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457297+0000", 2024-09-11T06:49:50.460 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015598241000000001 2024-09-11T06:49:50.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.460 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.460 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:50.460 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457577+0000", 2024-09-11T06:49:50.460 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027930799999999998 2024-09-11T06:49:50.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.461 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.461 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:50.461 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457957+0000", 2024-09-11T06:49:50.461 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00038088 2024-09-11T06:49:50.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.461 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.461 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:50.461 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460432+0000", 2024-09-11T06:49:50.462 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024743349999999998 2024-09-11T06:49:50.462 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.462 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.462 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:50.462 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460584+0000", 2024-09-11T06:49:50.462 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015232900000000001 2024-09-11T06:49:50.462 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.462 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.462 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:50.462 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460646+0000", 2024-09-11T06:49:50.462 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1883000000000004e-05 2024-09-11T06:49:50.463 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.463 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.463 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:50.463 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460696+0000", 2024-09-11T06:49:50.463 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9589999999999998e-05 2024-09-11T06:49:50.463 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.463 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:50.463 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.463 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.463 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.464 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:50.464 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:49:50.464 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 43.942059716000003, 2024-09-11T06:49:50.464 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.021834473, 2024-09-11T06:49:50.464 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:50.464 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:50.464 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:50.464 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:50.464 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:50.464 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:50.464 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 117 2024-09-11T06:49:50.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.465 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:50.465 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.465 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:50.465 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:49:50.465 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.465 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.465 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:50.465 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:49:50.465 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.465 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.466 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:50.466 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441725+0000", 2024-09-11T06:49:50.466 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5069999999999999e-06 2024-09-11T06:49:50.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.466 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.466 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:50.466 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441738+0000", 2024-09-11T06:49:50.466 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2813000000000001e-05 2024-09-11T06:49:50.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.466 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.466 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:50.466 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441759+0000", 2024-09-11T06:49:50.466 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1246999999999998e-05 2024-09-11T06:49:50.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.467 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.467 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:50.467 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441779+0000", 2024-09-11T06:49:50.467 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0392999999999999e-05 2024-09-11T06:49:50.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.467 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.468 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:50.468 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458804+0000", 2024-09-11T06:49:50.468 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.017024182999999998 2024-09-11T06:49:50.468 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.468 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.468 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:50.468 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458977+0000", 2024-09-11T06:49:50.468 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000173926 2024-09-11T06:49:50.468 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.468 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.468 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:50.468 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459170+0000", 2024-09-11T06:49:50.468 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000192184 2024-09-11T06:49:50.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.469 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.469 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:50.469 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.462300+0000", 2024-09-11T06:49:50.469 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031304969999999999 2024-09-11T06:49:50.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.469 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.469 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:50.469 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463408+0000", 2024-09-11T06:49:50.469 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011077839999999999 2024-09-11T06:49:50.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.469 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.470 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:50.470 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463497+0000", 2024-09-11T06:49:50.470 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8828999999999999e-05 2024-09-11T06:49:50.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.470 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.470 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:50.470 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463558+0000", 2024-09-11T06:49:50.470 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1110000000000003e-05 2024-09-11T06:49:50.470 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.470 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:50.470 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.470 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.470 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:50.471 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:49:50.471 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 43.565552027999999, 2024-09-11T06:49:50.471 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012216898, 2024-09-11T06:49:50.471 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:50.471 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:50.471 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:50.471 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:50.471 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:50.471 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:50.471 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 137 2024-09-11T06:49:50.471 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.471 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:50.471 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.472 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:50.472 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:49:50.472 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.472 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.472 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:50.472 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:49:50.472 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.472 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.472 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:50.472 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818232+0000", 2024-09-11T06:49:50.472 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2839999999999999e-06 2024-09-11T06:49:50.473 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.473 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.473 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:50.473 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818326+0000", 2024-09-11T06:49:50.473 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3531999999999998e-05 2024-09-11T06:49:50.473 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.473 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.473 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:50.473 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818337+0000", 2024-09-11T06:49:50.473 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0655999999999999e-05 2024-09-11T06:49:50.473 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.473 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.474 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:50.474 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818342+0000", 2024-09-11T06:49:50.474 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4600000000000002e-06 2024-09-11T06:49:50.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.474 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.474 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:50.474 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823890+0000", 2024-09-11T06:49:50.474 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0055480700000000004 2024-09-11T06:49:50.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.474 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.474 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:50.474 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823953+0000", 2024-09-11T06:49:50.474 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2848999999999994e-05 2024-09-11T06:49:50.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.475 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:50.475 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824074+0000", 2024-09-11T06:49:50.475 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121309 2024-09-11T06:49:50.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.475 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:50.475 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828043+0000", 2024-09-11T06:49:50.475 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0039685240000000002 2024-09-11T06:49:50.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.476 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:50.476 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830293+0000", 2024-09-11T06:49:50.476 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0022503179999999998 2024-09-11T06:49:50.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.476 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:50.476 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830366+0000", 2024-09-11T06:49:50.476 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2539999999999993e-05 2024-09-11T06:49:50.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.476 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:50.476 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830448+0000", 2024-09-11T06:49:50.476 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2355999999999996e-05 2024-09-11T06:49:50.477 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.477 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:50.477 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.477 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:50.477 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:50.477 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 43.565418213999997, 2024-09-11T06:49:50.477 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013136297, 2024-09-11T06:49:50.477 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:50.477 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:50.477 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:50.477 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:50.478 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:50.478 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:50.478 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 138 2024-09-11T06:49:50.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.478 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:50.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.478 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:50.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:50.478 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.478 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:50.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:50.479 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.479 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.479 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:50.479 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:50.479 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.89e-07 2024-09-11T06:49:50.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.479 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.479 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:50.479 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818369+0000", 2024-09-11T06:49:50.479 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4170000000000001e-06 2024-09-11T06:49:50.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.480 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:50.480 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818376+0000", 2024-09-11T06:49:50.480 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4270000000000002e-06 2024-09-11T06:49:50.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.480 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:50.480 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818383+0000", 2024-09-11T06:49:50.480 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3060000000000003e-06 2024-09-11T06:49:50.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.481 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:50.481 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824783+0000", 2024-09-11T06:49:50.481 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064003020000000001 2024-09-11T06:49:50.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.481 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:50.481 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825013+0000", 2024-09-11T06:49:50.481 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023039499999999999 2024-09-11T06:49:50.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.481 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:50.481 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825484+0000", 2024-09-11T06:49:50.482 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000470562 2024-09-11T06:49:50.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.482 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:50.482 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828633+0000", 2024-09-11T06:49:50.482 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031496689999999999 2024-09-11T06:49:50.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.482 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:50.482 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831413+0000", 2024-09-11T06:49:50.482 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027793219999999999 2024-09-11T06:49:50.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.483 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.483 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:50.483 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831450+0000", 2024-09-11T06:49:50.483 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7370999999999998e-05 2024-09-11T06:49:50.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.483 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.483 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:50.483 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831501+0000", 2024-09-11T06:49:50.483 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1036999999999999e-05 2024-09-11T06:49:50.483 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.483 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:50.484 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.484 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.484 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:50.484 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:49:50.484 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 43.565393733999997, 2024-09-11T06:49:50.484 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013223011, 2024-09-11T06:49:50.484 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:50.484 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:50.484 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:50.484 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:50.484 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:50.484 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:50.485 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 139 2024-09-11T06:49:50.485 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.485 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:50.485 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.485 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:50.485 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:49:50.485 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.485 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.485 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.485 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:50.485 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:49:50.485 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.486 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.486 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.486 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:50.486 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818390+0000", 2024-09-11T06:49:50.486 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5600000000000001e-07 2024-09-11T06:49:50.486 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.486 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.486 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:50.486 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818394+0000", 2024-09-11T06:49:50.486 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5929999999999999e-06 2024-09-11T06:49:50.487 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.487 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.487 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:50.487 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818399+0000", 2024-09-11T06:49:50.487 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7740000000000004e-06 2024-09-11T06:49:50.487 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.487 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.487 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:50.487 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818412+0000", 2024-09-11T06:49:50.487 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2625e-05 2024-09-11T06:49:50.487 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.487 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.488 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:50.488 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826205+0000", 2024-09-11T06:49:50.488 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0077930730000000002 2024-09-11T06:49:50.488 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.488 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.488 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:50.488 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826281+0000", 2024-09-11T06:49:50.488 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5876000000000003e-05 2024-09-11T06:49:50.488 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.488 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.489 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:50.489 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826384+0000", 2024-09-11T06:49:50.489 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103838 2024-09-11T06:49:50.489 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.489 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.489 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:50.489 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830642+0000", 2024-09-11T06:49:50.489 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004257083 2024-09-11T06:49:50.489 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.489 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.489 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:50.489 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831509+0000", 2024-09-11T06:49:50.489 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086730900000000003 2024-09-11T06:49:50.489 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.489 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.490 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:50.490 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831547+0000", 2024-09-11T06:49:50.490 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7793000000000002e-05 2024-09-11T06:49:50.490 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.490 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.490 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:50.490 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831612+0000", 2024-09-11T06:49:50.490 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5691000000000005e-05 2024-09-11T06:49:50.490 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.490 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:50.490 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.490 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.491 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.491 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:50.491 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:50.491 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 43.565365040000003, 2024-09-11T06:49:50.491 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016098024999999998, 2024-09-11T06:49:50.491 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:50.491 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:50.492 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:50.492 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:50.492 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:50.492 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:50.492 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 140 2024-09-11T06:49:50.492 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.492 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:50.492 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.492 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:50.492 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:50.492 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.492 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.492 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.493 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:50.493 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:50.493 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.493 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.493 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:50.493 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:50.493 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5199999999999998e-07 2024-09-11T06:49:50.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.493 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.494 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:50.494 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818479+0000", 2024-09-11T06:49:50.494 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0368000000000002e-05 2024-09-11T06:49:50.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.494 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.494 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:50.494 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818487+0000", 2024-09-11T06:49:50.494 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2339999999999994e-06 2024-09-11T06:49:50.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.494 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.494 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:50.494 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818491+0000", 2024-09-11T06:49:50.494 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4329999999999997e-06 2024-09-11T06:49:50.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.495 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.495 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:50.495 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828080+0000", 2024-09-11T06:49:50.495 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0095890449999999992 2024-09-11T06:49:50.495 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.495 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.495 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:50.495 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828130+0000", 2024-09-11T06:49:50.495 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9239000000000001e-05 2024-09-11T06:49:50.495 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.495 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.495 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:50.495 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828208+0000", 2024-09-11T06:49:50.495 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.852e-05 2024-09-11T06:49:50.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.496 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.496 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:50.496 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831625+0000", 2024-09-11T06:49:50.496 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034169410000000002 2024-09-11T06:49:50.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.496 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.496 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:50.496 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834411+0000", 2024-09-11T06:49:50.496 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002786111 2024-09-11T06:49:50.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.497 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.497 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:50.497 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834457+0000", 2024-09-11T06:49:50.497 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6041000000000002e-05 2024-09-11T06:49:50.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.497 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.497 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:50.497 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834516+0000", 2024-09-11T06:49:50.497 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8740999999999999e-05 2024-09-11T06:49:50.497 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.497 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:50.497 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.498 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.498 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:50.498 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:49:50.498 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 43.565283973, 2024-09-11T06:49:50.498 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016588772000000002, 2024-09-11T06:49:50.498 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:50.498 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:50.498 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:50.498 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:50.499 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:50.499 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:50.499 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 141 2024-09-11T06:49:50.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.499 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:50.499 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.499 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:50.499 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:49:50.499 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.500 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.500 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:50.500 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:49:50.500 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.500 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.500 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:50.500 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818500+0000", 2024-09-11T06:49:50.500 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0800000000000005e-07 2024-09-11T06:49:50.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.501 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.501 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:50.501 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818505+0000", 2024-09-11T06:49:50.501 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2429999999999996e-06 2024-09-11T06:49:50.501 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.501 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.501 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:50.501 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818510+0000", 2024-09-11T06:49:50.501 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9339999999999997e-06 2024-09-11T06:49:50.501 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.501 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.501 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:50.502 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818514+0000", 2024-09-11T06:49:50.502 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7270000000000001e-06 2024-09-11T06:49:50.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.502 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.502 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:50.502 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828711+0000", 2024-09-11T06:49:50.502 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010197031 2024-09-11T06:49:50.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.502 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.502 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:50.502 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828853+0000", 2024-09-11T06:49:50.502 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014201200000000001 2024-09-11T06:49:50.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.503 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:50.503 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.829115+0000", 2024-09-11T06:49:50.503 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026257500000000003 2024-09-11T06:49:50.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.503 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:50.503 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831629+0000", 2024-09-11T06:49:50.503 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025138169999999998 2024-09-11T06:49:50.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.503 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:50.503 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834991+0000", 2024-09-11T06:49:50.503 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033624459999999998 2024-09-11T06:49:50.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.504 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.504 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:50.504 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835032+0000", 2024-09-11T06:49:50.504 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1001000000000003e-05 2024-09-11T06:49:50.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.504 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.504 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:50.504 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835088+0000", 2024-09-11T06:49:50.504 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5478e-05 2024-09-11T06:49:50.505 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.505 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:50.505 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.505 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:49:50.505 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:49:50.505 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 42.941649069999997, 2024-09-11T06:49:50.505 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011161912, 2024-09-11T06:49:50.505 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:50.505 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:50.505 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:50.505 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:50.506 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:50.506 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:50.506 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 158 2024-09-11T06:49:50.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.506 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:50.506 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.506 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:50.506 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:49:50.506 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.506 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.506 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:50.506 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:49:50.506 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.507 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.507 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.507 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:50.507 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442135+0000", 2024-09-11T06:49:50.507 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3e-07 2024-09-11T06:49:50.507 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.507 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.507 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:50.507 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442259+0000", 2024-09-11T06:49:50.507 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000123903 2024-09-11T06:49:50.507 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.507 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.508 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:50.508 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442289+0000", 2024-09-11T06:49:50.508 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0833000000000001e-05 2024-09-11T06:49:50.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.508 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.508 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:50.508 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442301+0000", 2024-09-11T06:49:50.508 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1433e-05 2024-09-11T06:49:50.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.508 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.508 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:50.508 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449796+0000", 2024-09-11T06:49:50.509 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0074954959999999999 2024-09-11T06:49:50.509 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.509 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.509 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:50.509 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449917+0000", 2024-09-11T06:49:50.509 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121033 2024-09-11T06:49:50.509 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.509 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.509 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:50.509 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.450166+0000", 2024-09-11T06:49:50.509 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00024832400000000002 2024-09-11T06:49:50.509 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.510 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:50.510 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.452630+0000", 2024-09-11T06:49:50.510 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002464188 2024-09-11T06:49:50.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.510 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:50.510 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453149+0000", 2024-09-11T06:49:50.510 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00051860999999999997 2024-09-11T06:49:50.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.510 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:50.510 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453234+0000", 2024-09-11T06:49:50.510 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.5865000000000005e-05 2024-09-11T06:49:50.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.511 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.511 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:50.511 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453296+0000", 2024-09-11T06:49:50.511 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1496999999999998e-05 2024-09-11T06:49:50.511 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.511 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:50.511 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.511 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.511 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:49:50.512 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:49:50.512 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 42.941462968000003, 2024-09-11T06:49:50.512 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011561900999999999, 2024-09-11T06:49:50.512 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:50.512 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:50.512 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:50.512 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:50.512 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:50.512 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:50.512 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 159 2024-09-11T06:49:50.512 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.512 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:50.512 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.513 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:50.513 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:49:50.513 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.513 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.513 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.513 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:50.513 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:49:50.513 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:50.513 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.513 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.513 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:50.513 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442321+0000", 2024-09-11T06:49:50.514 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8400000000000003e-07 2024-09-11T06:49:50.514 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.514 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.514 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:50.514 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442328+0000", 2024-09-11T06:49:50.514 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5130000000000002e-06 2024-09-11T06:49:50.514 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.514 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.514 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:50.514 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442341+0000", 2024-09-11T06:49:50.514 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.363e-05 2024-09-11T06:49:50.514 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.514 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.514 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:50.515 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442349+0000", 2024-09-11T06:49:50.515 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4760000000000001e-06 2024-09-11T06:49:50.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.515 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.515 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:50.515 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451257+0000", 2024-09-11T06:49:50.515 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089088569999999992 2024-09-11T06:49:50.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.515 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.515 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:50.515 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451389+0000", 2024-09-11T06:49:50.515 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013110899999999999 2024-09-11T06:49:50.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.515 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.515 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:50.515 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451608+0000", 2024-09-11T06:49:50.516 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000219588 2024-09-11T06:49:50.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.516 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.516 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:50.516 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453517+0000", 2024-09-11T06:49:50.516 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0019088779999999999 2024-09-11T06:49:50.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.516 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.516 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:50.516 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453727+0000", 2024-09-11T06:49:50.516 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020955100000000001 2024-09-11T06:49:50.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.516 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.517 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:50.517 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453815+0000", 2024-09-11T06:49:50.517 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8858999999999997e-05 2024-09-11T06:49:50.517 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:50.517 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:50.517 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:50.517 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453882+0000", 2024-09-11T06:49:50.517 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.6556000000000004e-05 2024-09-11T06:49:50.517 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.517 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:50.518 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.518 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:50.518 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:50.518 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:50.519 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:49:50.520 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:50.518+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:50.543 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is unset 2024-09-11T06:49:50.620 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:50.618+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:50.675 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:50.675 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:50.675 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:49:50.676 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:50.689 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:49:50.721 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:50.717+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:50.822 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:50.818+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:50.846 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:50.847 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:50.847 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:49:50.847 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:49:50.847 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:50.853 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:49:50.860 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:49:50.922 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:50.917+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:51.006 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:49:51.006 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:49:51.018 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:51.018 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:49:51.018 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:49:51.019 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [] 2024-09-11T06:49:51.019 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:51.023 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:49:51.023 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:49:51.023 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:51.022+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:51.032 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:49:51.039 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:49:51.039 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:49:51.057 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:49:51.058 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:49:51.077 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:49:51.078 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:49:51.096 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:49:51.096 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:49:51.156 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:51.153+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:51.193 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:51.194 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:51.208 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:49:51.225 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:51.221+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:51.326 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:51.326+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:51.328 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:51.328 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:51.344 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:49:51.430 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:51.430+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:51.503 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:51.503 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:51.516 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:49:51.558 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:51.558+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:51.629 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:51.625+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:51.651 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:51.651 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:51.665 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:49:51.756 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:51.753+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:51.831 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:51.825+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:51.836 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:51.836 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:51.850 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:49:51.932 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:51.930+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:52.016 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:52.017 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:52.031 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:49:52.032 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:52.029+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:52.133 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:52.130+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:52.151 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:52.151 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:52.166 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:49:52.234 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:52.229+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:52.289 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:52.289 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:52.303 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:49:52.336 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:52.334+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:52.437 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:52.433+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:52.469 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:52.469 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:52.483 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:49:52.568 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T06:49:52.572 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:52.569+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:52.639 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:52.638+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:52.647 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:52.647 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:52.662 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:49:52.740 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:52.737+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:52.780 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:52.781 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:52.794 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:49:52.840 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:52.837+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:52.911 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:52.911 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:52.924 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is unset 2024-09-11T06:49:52.925 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:49:52.970 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:52.969+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:53.041 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:53.037+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:53.056 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:53.056 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:53.070 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:49:53.114 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:49:53.160 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:53.157+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:53.239 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:53.240 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:53.243 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:53.241+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:53.254 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:49:53.264 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:49:53.265 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:49:53.281 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:49:53.281 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:49:53.298 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:49:53.298 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:49:53.318 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:49:53.318 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:49:53.336 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:49:53.336 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:49:53.343 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:53.341+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:53.353 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:49:53.354 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:49:53.419 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:53.419 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:49:53.419 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:49:53.419 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:49:53.420 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.420 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:53.420 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:49:53.420 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 47.249134107000003, 2024-09-11T06:49:53.420 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011420669, 2024-09-11T06:49:53.420 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:53.420 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:53.420 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:53.420 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:53.420 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:53.420 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:53.420 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 37 2024-09-11T06:49:53.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.421 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:53.421 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.421 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:53.421 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:49:53.421 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.421 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.421 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:53.421 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:49:53.421 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.421 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.422 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:53.422 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170509+0000", 2024-09-11T06:49:53.422 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.13e-06 2024-09-11T06:49:53.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.422 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.422 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:53.422 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170521+0000", 2024-09-11T06:49:53.422 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1776999999999999e-05 2024-09-11T06:49:53.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.422 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.422 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:53.422 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170538+0000", 2024-09-11T06:49:53.423 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7413000000000001e-05 2024-09-11T06:49:53.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.423 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.423 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:53.423 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170570+0000", 2024-09-11T06:49:53.423 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2313e-05 2024-09-11T06:49:53.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.423 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.423 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:53.423 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179768+0000", 2024-09-11T06:49:53.423 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0091978010000000002 2024-09-11T06:49:53.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.423 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.424 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:53.424 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179951+0000", 2024-09-11T06:49:53.424 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018227900000000001 2024-09-11T06:49:53.424 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.424 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.424 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:53.424 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.180175+0000", 2024-09-11T06:49:53.424 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022432800000000001 2024-09-11T06:49:53.424 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.424 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.424 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:53.424 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181112+0000", 2024-09-11T06:49:53.424 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00093712500000000002 2024-09-11T06:49:53.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.425 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.425 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:53.425 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181789+0000", 2024-09-11T06:49:53.425 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00067705100000000002 2024-09-11T06:49:53.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.425 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.425 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:53.425 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181871+0000", 2024-09-11T06:49:53.425 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1816000000000004e-05 2024-09-11T06:49:53.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.425 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.425 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:53.426 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181929+0000", 2024-09-11T06:49:53.426 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7636000000000002e-05 2024-09-11T06:49:53.426 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.426 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:53.426 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.426 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.426 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.426 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:53.426 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:49:53.426 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 47.147875145999997, 2024-09-11T06:49:53.426 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283336, 2024-09-11T06:49:53.426 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:53.426 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:53.426 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:53.427 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:53.427 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:53.427 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:53.427 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 87 2024-09-11T06:49:53.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.427 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:53.427 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.427 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:53.427 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:49:53.428 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.428 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.428 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.428 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:53.428 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:49:53.428 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.428 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.428 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.428 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:53.428 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271768+0000", 2024-09-11T06:49:53.428 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7599999999999996e-07 2024-09-11T06:49:53.428 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.428 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.429 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:53.429 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271860+0000", 2024-09-11T06:49:53.429 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2639999999999994e-05 2024-09-11T06:49:53.429 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.429 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.429 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:53.429 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271873+0000", 2024-09-11T06:49:53.429 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3139000000000001e-05 2024-09-11T06:49:53.429 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.429 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.429 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:53.429 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271882+0000", 2024-09-11T06:49:53.429 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2369999999999992e-06 2024-09-11T06:49:53.430 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.430 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.430 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:53.430 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277150+0000", 2024-09-11T06:49:53.430 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052684419999999999 2024-09-11T06:49:53.430 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.430 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.430 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:53.430 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277273+0000", 2024-09-11T06:49:53.430 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012278899999999999 2024-09-11T06:49:53.430 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.430 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.430 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:53.430 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277466+0000", 2024-09-11T06:49:53.431 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193578 2024-09-11T06:49:53.431 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.431 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.431 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:53.431 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.280808+0000", 2024-09-11T06:49:53.431 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003341971 2024-09-11T06:49:53.431 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.431 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.431 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:53.431 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284419+0000", 2024-09-11T06:49:53.431 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003610982 2024-09-11T06:49:53.431 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.431 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.431 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:53.432 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284531+0000", 2024-09-11T06:49:53.432 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011121299999999999 2024-09-11T06:49:53.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.432 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.432 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:53.432 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284600+0000", 2024-09-11T06:49:53.432 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9592999999999996e-05 2024-09-11T06:49:53.432 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.432 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:53.432 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.432 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.432 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:53.433 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:49:53.433 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 47.147744893999999, 2024-09-11T06:49:53.433 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283314, 2024-09-11T06:49:53.433 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:53.433 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:53.433 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:53.433 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:53.433 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:53.433 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:53.433 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 88 2024-09-11T06:49:53.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.433 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:53.433 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.433 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:53.434 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:49:53.434 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.434 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.434 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:53.434 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:49:53.434 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.434 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.434 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:53.434 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271898+0000", 2024-09-11T06:49:53.434 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2999999999999999e-07 2024-09-11T06:49:53.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.435 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.435 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:53.435 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271904+0000", 2024-09-11T06:49:53.435 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.767e-06 2024-09-11T06:49:53.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.435 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.435 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:53.435 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271913+0000", 2024-09-11T06:49:53.435 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7200000000000001e-06 2024-09-11T06:49:53.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.435 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.435 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:53.435 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271920+0000", 2024-09-11T06:49:53.435 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7100000000000001e-06 2024-09-11T06:49:53.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.436 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.436 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:53.436 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278684+0000", 2024-09-11T06:49:53.436 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0067642650000000002 2024-09-11T06:49:53.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.436 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.436 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:53.436 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278838+0000", 2024-09-11T06:49:53.436 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015371499999999999 2024-09-11T06:49:53.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.436 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.436 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:53.437 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.279040+0000", 2024-09-11T06:49:53.437 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020207499999999999 2024-09-11T06:49:53.437 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.437 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.437 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:53.437 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.281448+0000", 2024-09-11T06:49:53.437 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002407686 2024-09-11T06:49:53.437 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.437 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.437 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:53.437 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284615+0000", 2024-09-11T06:49:53.437 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031675499999999999 2024-09-11T06:49:53.437 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.437 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.438 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:53.438 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284682+0000", 2024-09-11T06:49:53.438 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7032000000000005e-05 2024-09-11T06:49:53.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.438 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.438 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:53.438 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284730+0000", 2024-09-11T06:49:53.438 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7790000000000002e-05 2024-09-11T06:49:53.438 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.438 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:53.438 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.438 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.438 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:53.439 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:49:53.439 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 46.979843672000001, 2024-09-11T06:49:53.439 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011114877, 2024-09-11T06:49:53.439 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:53.439 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:53.439 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:53.439 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:53.439 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:53.439 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:53.439 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 107 2024-09-11T06:49:53.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.439 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:53.439 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.439 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:53.440 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:49:53.440 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.440 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.440 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:53.440 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:49:53.440 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.440 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.440 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:53.440 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439799+0000", 2024-09-11T06:49:53.440 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.16e-07 2024-09-11T06:49:53.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.441 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.441 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:53.441 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439806+0000", 2024-09-11T06:49:53.441 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3640000000000004e-06 2024-09-11T06:49:53.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.441 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.441 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:53.441 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439816+0000", 2024-09-11T06:49:53.441 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0485999999999999e-05 2024-09-11T06:49:53.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.441 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.441 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:53.441 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439824+0000", 2024-09-11T06:49:53.441 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.6699999999999994e-06 2024-09-11T06:49:53.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.442 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.442 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:53.442 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443328+0000", 2024-09-11T06:49:53.442 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035044329999999999 2024-09-11T06:49:53.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.442 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.442 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:53.442 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443461+0000", 2024-09-11T06:49:53.442 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013272199999999999 2024-09-11T06:49:53.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.442 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.442 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:53.442 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443678+0000", 2024-09-11T06:49:53.443 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021674500000000001 2024-09-11T06:49:53.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.443 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:53.443 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449635+0000", 2024-09-11T06:49:53.443 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0059572119999999999 2024-09-11T06:49:53.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.443 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:53.443 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450750+0000", 2024-09-11T06:49:53.443 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001115654 2024-09-11T06:49:53.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.443 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:53.444 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450832+0000", 2024-09-11T06:49:53.444 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1188999999999998e-05 2024-09-11T06:49:53.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.444 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.444 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:53.444 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450913+0000", 2024-09-11T06:49:53.444 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1485999999999999e-05 2024-09-11T06:49:53.444 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.444 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:53.444 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.444 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.445 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:53.445 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:49:53.445 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 46.979252697, 2024-09-11T06:49:53.445 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011236556, 2024-09-11T06:49:53.445 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:53.445 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:53.445 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:53.445 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:53.445 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:53.445 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:53.445 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 109 2024-09-11T06:49:53.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.445 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:53.446 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.446 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:53.446 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:49:53.446 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.446 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.446 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:53.446 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:49:53.446 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.446 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.446 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:53.446 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440391+0000", 2024-09-11T06:49:53.446 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8360000000000001e-06 2024-09-11T06:49:53.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.447 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:53.447 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440560+0000", 2024-09-11T06:49:53.447 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000169085 2024-09-11T06:49:53.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.447 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:53.447 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440585+0000", 2024-09-11T06:49:53.447 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4530000000000001e-05 2024-09-11T06:49:53.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.447 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:53.447 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440606+0000", 2024-09-11T06:49:53.448 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1277e-05 2024-09-11T06:49:53.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.448 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:53.448 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446479+0000", 2024-09-11T06:49:53.448 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0058734989999999999 2024-09-11T06:49:53.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.448 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:53.448 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446649+0000", 2024-09-11T06:49:53.448 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016948199999999999 2024-09-11T06:49:53.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.448 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:53.449 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446854+0000", 2024-09-11T06:49:53.449 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020491500000000001 2024-09-11T06:49:53.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.449 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:53.449 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451380+0000", 2024-09-11T06:49:53.449 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0045264110000000001 2024-09-11T06:49:53.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.449 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:53.449 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451479+0000", 2024-09-11T06:49:53.449 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.8975999999999996e-05 2024-09-11T06:49:53.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.450 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.450 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:53.450 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451562+0000", 2024-09-11T06:49:53.450 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3213000000000004e-05 2024-09-11T06:49:53.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.450 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.450 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:53.450 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451626+0000", 2024-09-11T06:49:53.450 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3331999999999996e-05 2024-09-11T06:49:53.450 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.450 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:53.450 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.450 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.451 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:53.451 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:49:53.451 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 46.979003569, 2024-09-11T06:49:53.451 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011252866, 2024-09-11T06:49:53.451 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:53.451 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:53.451 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:53.451 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:53.451 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:53.451 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:53.451 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 110 2024-09-11T06:49:53.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.451 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:53.451 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.452 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:53.452 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:49:53.452 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.452 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:53.452 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:49:53.452 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.452 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:53.452 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440640+0000", 2024-09-11T06:49:53.452 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5430000000000001e-06 2024-09-11T06:49:53.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.453 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:53.453 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440651+0000", 2024-09-11T06:49:53.453 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0672999999999999e-05 2024-09-11T06:49:53.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.453 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:53.453 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440670+0000", 2024-09-11T06:49:53.453 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9726999999999999e-05 2024-09-11T06:49:53.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.453 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:53.454 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440691+0000", 2024-09-11T06:49:53.454 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0526e-05 2024-09-11T06:49:53.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.454 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:53.454 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448191+0000", 2024-09-11T06:49:53.454 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0075005990000000002 2024-09-11T06:49:53.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.454 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:53.454 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448390+0000", 2024-09-11T06:49:53.454 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000198815 2024-09-11T06:49:53.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.455 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:53.455 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448735+0000", 2024-09-11T06:49:53.455 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034424999999999999 2024-09-11T06:49:53.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.455 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:53.455 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451742+0000", 2024-09-11T06:49:53.455 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030077440000000001 2024-09-11T06:49:53.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.455 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:53.455 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451761+0000", 2024-09-11T06:49:53.456 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9196999999999999e-05 2024-09-11T06:49:53.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.456 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:53.456 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451834+0000", 2024-09-11T06:49:53.456 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2071999999999997e-05 2024-09-11T06:49:53.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.456 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:53.456 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451891+0000", 2024-09-11T06:49:53.456 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7720000000000003e-05 2024-09-11T06:49:53.456 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.456 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:53.457 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.457 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.457 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:53.457 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:49:53.457 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 46.978895694000002, 2024-09-11T06:49:53.457 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014944941, 2024-09-11T06:49:53.457 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:53.457 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:53.457 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:53.457 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:53.457 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:53.457 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:53.458 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 111 2024-09-11T06:49:53.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.458 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:53.458 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.458 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:53.458 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:49:53.458 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.458 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.458 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:53.458 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:49:53.458 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.459 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:53.459 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440748+0000", 2024-09-11T06:49:53.459 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:49:53.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.459 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:53.459 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440760+0000", 2024-09-11T06:49:53.459 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1897e-05 2024-09-11T06:49:53.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.459 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:53.460 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440782+0000", 2024-09-11T06:49:53.460 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2552999999999999e-05 2024-09-11T06:49:53.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.460 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.460 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:53.460 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440793+0000", 2024-09-11T06:49:53.460 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0643e-05 2024-09-11T06:49:53.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.460 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.460 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:53.460 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449755+0000", 2024-09-11T06:49:53.460 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089622620000000004 2024-09-11T06:49:53.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.461 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.461 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:53.461 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449918+0000", 2024-09-11T06:49:53.461 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163086 2024-09-11T06:49:53.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.461 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.461 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:53.461 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450150+0000", 2024-09-11T06:49:53.461 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023204500000000001 2024-09-11T06:49:53.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.461 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.461 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:53.461 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454533+0000", 2024-09-11T06:49:53.462 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0043831349999999998 2024-09-11T06:49:53.462 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.462 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.462 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:53.462 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455511+0000", 2024-09-11T06:49:53.462 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0009776050000000001 2024-09-11T06:49:53.462 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.462 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.462 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:53.462 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455597+0000", 2024-09-11T06:49:53.462 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6446000000000005e-05 2024-09-11T06:49:53.462 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.462 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.462 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:53.463 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455691+0000", 2024-09-11T06:49:53.463 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3832000000000006e-05 2024-09-11T06:49:53.463 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.463 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:53.463 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.463 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.463 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.463 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:53.463 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:49:53.463 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 46.978825931000003, 2024-09-11T06:49:53.463 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018885411000000001, 2024-09-11T06:49:53.463 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:53.464 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:53.464 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:53.464 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:53.464 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:53.464 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:53.464 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 112 2024-09-11T06:49:53.464 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.464 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:53.464 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.464 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:53.464 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:49:53.464 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.464 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.465 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.465 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:53.465 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:49:53.465 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.465 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.465 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:53.465 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440817+0000", 2024-09-11T06:49:53.465 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:49:53.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.465 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.465 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:53.465 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441003+0000", 2024-09-11T06:49:53.465 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018661200000000001 2024-09-11T06:49:53.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.466 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.466 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:53.466 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441027+0000", 2024-09-11T06:49:53.466 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3920000000000001e-05 2024-09-11T06:49:53.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.466 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.466 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:53.466 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441062+0000", 2024-09-11T06:49:53.466 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4663000000000003e-05 2024-09-11T06:49:53.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.466 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.466 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:53.467 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451981+0000", 2024-09-11T06:49:53.467 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010918926000000001 2024-09-11T06:49:53.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.467 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.467 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:53.467 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452111+0000", 2024-09-11T06:49:53.467 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129529 2024-09-11T06:49:53.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.467 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.467 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:53.467 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452311+0000", 2024-09-11T06:49:53.467 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020013799999999999 2024-09-11T06:49:53.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.467 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.467 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:53.468 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455809+0000", 2024-09-11T06:49:53.468 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003498443 2024-09-11T06:49:53.468 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.468 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.468 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:53.468 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459566+0000", 2024-09-11T06:49:53.468 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003757271 2024-09-11T06:49:53.468 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.468 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.469 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:53.469 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459640+0000", 2024-09-11T06:49:53.469 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3949000000000006e-05 2024-09-11T06:49:53.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.469 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.469 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:53.469 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459701+0000", 2024-09-11T06:49:53.469 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1117000000000005e-05 2024-09-11T06:49:53.469 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.469 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:53.469 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.469 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.469 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:53.470 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:49:53.470 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 46.978547079999998, 2024-09-11T06:49:53.470 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018760915, 2024-09-11T06:49:53.470 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:53.470 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:53.470 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:53.470 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:53.470 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:53.470 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:53.470 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 113 2024-09-11T06:49:53.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.470 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:53.470 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.470 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:53.471 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:49:53.471 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.471 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.471 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.471 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:53.471 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:49:53.471 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.471 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.471 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.471 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:53.471 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441096+0000", 2024-09-11T06:49:53.471 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4130000000000001e-06 2024-09-11T06:49:53.471 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.472 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.472 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:53.472 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441110+0000", 2024-09-11T06:49:53.472 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3900000000000001e-05 2024-09-11T06:49:53.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.472 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.472 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:53.472 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441131+0000", 2024-09-11T06:49:53.472 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0636999999999999e-05 2024-09-11T06:49:53.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.473 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.473 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:53.473 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441147+0000", 2024-09-11T06:49:53.473 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6512999999999999e-05 2024-09-11T06:49:53.473 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.473 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.473 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:53.473 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453274+0000", 2024-09-11T06:49:53.473 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012126764999999999 2024-09-11T06:49:53.473 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.473 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.473 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:53.473 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453498+0000", 2024-09-11T06:49:53.474 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000223568 2024-09-11T06:49:53.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.474 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.474 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:53.474 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453872+0000", 2024-09-11T06:49:53.474 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037467099999999998 2024-09-11T06:49:53.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.474 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.474 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:53.474 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458709+0000", 2024-09-11T06:49:53.474 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0048365680000000003 2024-09-11T06:49:53.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.474 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.474 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:53.475 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459716+0000", 2024-09-11T06:49:53.475 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010075240000000001 2024-09-11T06:49:53.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.475 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:53.475 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459779+0000", 2024-09-11T06:49:53.475 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2336999999999996e-05 2024-09-11T06:49:53.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.475 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:53.475 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459856+0000", 2024-09-11T06:49:53.475 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7019000000000002e-05 2024-09-11T06:49:53.475 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.475 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:53.475 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.476 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:53.476 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:53.476 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 46.978470340999998, 2024-09-11T06:49:53.476 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018963114, 2024-09-11T06:49:53.476 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:53.476 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:53.476 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:53.476 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:53.476 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:53.476 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:53.476 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 114 2024-09-11T06:49:53.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.477 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:53.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.477 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:53.477 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:53.477 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.477 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:53.477 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:53.477 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.478 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:53.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:53.478 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9299999999999996e-07 2024-09-11T06:49:53.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.478 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:53.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441180+0000", 2024-09-11T06:49:53.478 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7200000000000006e-06 2024-09-11T06:49:53.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.478 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:53.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441265+0000", 2024-09-11T06:49:53.478 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4753000000000007e-05 2024-09-11T06:49:53.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.479 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.479 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:53.479 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441288+0000", 2024-09-11T06:49:53.479 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2860000000000001e-05 2024-09-11T06:49:53.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.479 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.479 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:53.479 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454585+0000", 2024-09-11T06:49:53.480 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013297286 2024-09-11T06:49:53.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.480 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:53.480 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454714+0000", 2024-09-11T06:49:53.480 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000128465 2024-09-11T06:49:53.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.480 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:53.480 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454909+0000", 2024-09-11T06:49:53.480 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019522499999999999 2024-09-11T06:49:53.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.481 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:53.481 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459868+0000", 2024-09-11T06:49:53.481 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0049591239999999997 2024-09-11T06:49:53.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.481 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:53.481 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459988+0000", 2024-09-11T06:49:53.481 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000120532 2024-09-11T06:49:53.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.481 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:53.481 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460071+0000", 2024-09-11T06:49:53.481 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2256000000000007e-05 2024-09-11T06:49:53.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.482 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:53.482 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460135+0000", 2024-09-11T06:49:53.482 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3999999999999997e-05 2024-09-11T06:49:53.482 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.482 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:53.482 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.482 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:53.482 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:49:53.482 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 46.978312711999997, 2024-09-11T06:49:53.482 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.019036849000000002, 2024-09-11T06:49:53.483 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:53.483 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:53.483 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:53.483 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:53.483 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:53.483 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:53.483 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 115 2024-09-11T06:49:53.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.483 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:53.483 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.483 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:53.483 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:49:53.483 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.484 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.484 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:53.484 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:49:53.484 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.484 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.484 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:53.484 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441330+0000", 2024-09-11T06:49:53.484 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1930000000000001e-06 2024-09-11T06:49:53.485 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.485 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.485 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:53.485 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441544+0000", 2024-09-11T06:49:53.485 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021405799999999999 2024-09-11T06:49:53.485 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.485 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.485 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:53.485 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441572+0000", 2024-09-11T06:49:53.485 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7317e-05 2024-09-11T06:49:53.485 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.485 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.485 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:53.486 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441593+0000", 2024-09-11T06:49:53.486 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1702999999999999e-05 2024-09-11T06:49:53.486 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.486 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.486 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:53.486 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455891+0000", 2024-09-11T06:49:53.486 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01429708 2024-09-11T06:49:53.486 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.486 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.486 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:53.486 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456046+0000", 2024-09-11T06:49:53.486 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000155682 2024-09-11T06:49:53.486 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.487 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.487 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:53.487 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456242+0000", 2024-09-11T06:49:53.487 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000195692 2024-09-11T06:49:53.487 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.487 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.487 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:53.487 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459876+0000", 2024-09-11T06:49:53.487 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036336480000000002 2024-09-11T06:49:53.487 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.487 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.487 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:53.487 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460245+0000", 2024-09-11T06:49:53.487 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036901700000000002 2024-09-11T06:49:53.488 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.488 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.488 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:53.488 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460316+0000", 2024-09-11T06:49:53.488 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1846000000000003e-05 2024-09-11T06:49:53.488 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.488 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.488 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:53.488 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460366+0000", 2024-09-11T06:49:53.488 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9613000000000001e-05 2024-09-11T06:49:53.488 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.488 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:53.488 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.489 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.489 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.489 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:53.489 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:49:53.489 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 46.977990632000001, 2024-09-11T06:49:53.489 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01904432, 2024-09-11T06:49:53.489 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:53.489 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:53.489 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:53.489 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:53.489 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:53.489 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:53.489 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 116 2024-09-11T06:49:53.490 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.490 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:53.490 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.490 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:53.490 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:49:53.490 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.490 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.490 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.490 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:53.490 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:49:53.490 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.490 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.490 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.490 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:53.491 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441653+0000", 2024-09-11T06:49:53.491 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:49:53.491 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.491 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.491 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:53.491 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441665+0000", 2024-09-11T06:49:53.491 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1653e-05 2024-09-11T06:49:53.491 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.491 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.491 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:53.491 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441684+0000", 2024-09-11T06:49:53.491 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9490000000000001e-05 2024-09-11T06:49:53.492 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.492 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.492 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:53.492 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441699+0000", 2024-09-11T06:49:53.492 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5044e-05 2024-09-11T06:49:53.492 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.492 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.492 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:53.492 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457297+0000", 2024-09-11T06:49:53.492 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015598241000000001 2024-09-11T06:49:53.492 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.492 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.492 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:53.492 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457577+0000", 2024-09-11T06:49:53.493 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027930799999999998 2024-09-11T06:49:53.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.493 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.493 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:53.493 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457957+0000", 2024-09-11T06:49:53.493 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00038088 2024-09-11T06:49:53.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.493 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.493 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:53.493 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460432+0000", 2024-09-11T06:49:53.493 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024743349999999998 2024-09-11T06:49:53.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.493 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.494 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:53.494 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460584+0000", 2024-09-11T06:49:53.494 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015232900000000001 2024-09-11T06:49:53.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.494 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.494 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:53.494 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460646+0000", 2024-09-11T06:49:53.494 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1883000000000004e-05 2024-09-11T06:49:53.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.494 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.494 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:53.494 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460696+0000", 2024-09-11T06:49:53.494 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9589999999999998e-05 2024-09-11T06:49:53.494 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.495 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:53.495 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.495 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.495 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.495 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:53.495 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:49:53.495 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 46.977918566, 2024-09-11T06:49:53.495 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.021834473, 2024-09-11T06:49:53.495 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:53.495 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:53.495 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:53.495 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:53.495 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:53.495 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:53.496 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 117 2024-09-11T06:49:53.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.496 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:53.496 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.496 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:53.496 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:49:53.496 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.496 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.496 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:53.496 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:49:53.496 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.497 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.497 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:53.497 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441725+0000", 2024-09-11T06:49:53.497 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5069999999999999e-06 2024-09-11T06:49:53.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.497 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.497 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:53.497 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441738+0000", 2024-09-11T06:49:53.497 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2813000000000001e-05 2024-09-11T06:49:53.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.497 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.497 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:53.497 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441759+0000", 2024-09-11T06:49:53.498 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1246999999999998e-05 2024-09-11T06:49:53.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.498 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.498 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:53.498 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441779+0000", 2024-09-11T06:49:53.498 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0392999999999999e-05 2024-09-11T06:49:53.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.498 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.498 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:53.498 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458804+0000", 2024-09-11T06:49:53.498 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.017024182999999998 2024-09-11T06:49:53.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.498 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.498 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:53.499 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458977+0000", 2024-09-11T06:49:53.499 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000173926 2024-09-11T06:49:53.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.499 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.499 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:53.499 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459170+0000", 2024-09-11T06:49:53.499 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000192184 2024-09-11T06:49:53.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.499 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.499 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:53.499 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.462300+0000", 2024-09-11T06:49:53.499 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031304969999999999 2024-09-11T06:49:53.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.500 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.500 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:53.500 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463408+0000", 2024-09-11T06:49:53.500 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011077839999999999 2024-09-11T06:49:53.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.500 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.500 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:53.500 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463497+0000", 2024-09-11T06:49:53.500 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8828999999999999e-05 2024-09-11T06:49:53.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.500 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.500 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:53.500 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463558+0000", 2024-09-11T06:49:53.500 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1110000000000003e-05 2024-09-11T06:49:53.501 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.501 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:53.501 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.501 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.501 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.501 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:53.501 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:49:53.501 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 46.601410878000003, 2024-09-11T06:49:53.501 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012216898, 2024-09-11T06:49:53.501 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:53.501 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:53.501 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:53.501 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:53.501 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:53.502 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:53.502 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 137 2024-09-11T06:49:53.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.502 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:53.502 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.502 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:53.502 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:49:53.502 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.502 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.502 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:53.502 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:49:53.502 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.503 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:53.503 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818232+0000", 2024-09-11T06:49:53.503 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2839999999999999e-06 2024-09-11T06:49:53.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.503 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:53.503 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818326+0000", 2024-09-11T06:49:53.503 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3531999999999998e-05 2024-09-11T06:49:53.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.503 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:53.503 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818337+0000", 2024-09-11T06:49:53.503 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0655999999999999e-05 2024-09-11T06:49:53.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.504 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.504 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:53.504 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818342+0000", 2024-09-11T06:49:53.504 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4600000000000002e-06 2024-09-11T06:49:53.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.504 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.504 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:53.504 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823890+0000", 2024-09-11T06:49:53.504 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0055480700000000004 2024-09-11T06:49:53.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.504 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.504 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:53.505 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823953+0000", 2024-09-11T06:49:53.505 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2848999999999994e-05 2024-09-11T06:49:53.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.505 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:53.505 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824074+0000", 2024-09-11T06:49:53.505 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121309 2024-09-11T06:49:53.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.505 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:53.505 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828043+0000", 2024-09-11T06:49:53.505 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0039685240000000002 2024-09-11T06:49:53.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.506 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:53.506 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830293+0000", 2024-09-11T06:49:53.506 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0022503179999999998 2024-09-11T06:49:53.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.506 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.506 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:53.506 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830366+0000", 2024-09-11T06:49:53.506 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2539999999999993e-05 2024-09-11T06:49:53.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.506 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.506 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:53.506 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830448+0000", 2024-09-11T06:49:53.506 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2355999999999996e-05 2024-09-11T06:49:53.506 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.507 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:53.507 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.507 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.507 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.507 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:53.507 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:53.507 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 46.601277064000001, 2024-09-11T06:49:53.507 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013136297, 2024-09-11T06:49:53.507 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:53.507 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:53.507 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:53.507 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:53.507 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:53.508 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:53.508 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 138 2024-09-11T06:49:53.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.508 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:53.508 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.508 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:53.508 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:53.508 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.508 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.508 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:53.508 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:53.508 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.509 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.509 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:53.509 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:53.509 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.89e-07 2024-09-11T06:49:53.509 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.509 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.509 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:53.509 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818369+0000", 2024-09-11T06:49:53.509 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4170000000000001e-06 2024-09-11T06:49:53.509 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.509 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.509 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:53.509 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818376+0000", 2024-09-11T06:49:53.510 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4270000000000002e-06 2024-09-11T06:49:53.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.510 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:53.510 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818383+0000", 2024-09-11T06:49:53.510 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3060000000000003e-06 2024-09-11T06:49:53.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.510 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:53.510 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824783+0000", 2024-09-11T06:49:53.510 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064003020000000001 2024-09-11T06:49:53.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.510 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:53.511 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825013+0000", 2024-09-11T06:49:53.511 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023039499999999999 2024-09-11T06:49:53.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.511 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.511 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:53.511 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825484+0000", 2024-09-11T06:49:53.511 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000470562 2024-09-11T06:49:53.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.511 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.511 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:53.511 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828633+0000", 2024-09-11T06:49:53.511 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031496689999999999 2024-09-11T06:49:53.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.512 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.512 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:53.512 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831413+0000", 2024-09-11T06:49:53.512 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027793219999999999 2024-09-11T06:49:53.512 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.512 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.512 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:53.512 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831450+0000", 2024-09-11T06:49:53.512 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7370999999999998e-05 2024-09-11T06:49:53.512 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.512 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.512 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:53.512 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831501+0000", 2024-09-11T06:49:53.513 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1036999999999999e-05 2024-09-11T06:49:53.513 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.513 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:53.513 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.513 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.513 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.513 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:53.513 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:49:53.513 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 46.601252584000001, 2024-09-11T06:49:53.513 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013223011, 2024-09-11T06:49:53.513 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:53.513 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:53.513 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:53.513 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:53.514 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:53.514 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:53.514 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 139 2024-09-11T06:49:53.514 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.514 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:53.514 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.514 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:53.514 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:49:53.514 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.514 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.514 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.514 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:53.514 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:49:53.515 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.515 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.515 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:53.515 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818390+0000", 2024-09-11T06:49:53.515 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5600000000000001e-07 2024-09-11T06:49:53.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.515 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.515 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:53.515 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818394+0000", 2024-09-11T06:49:53.515 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5929999999999999e-06 2024-09-11T06:49:53.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.515 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.515 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:53.516 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818399+0000", 2024-09-11T06:49:53.516 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7740000000000004e-06 2024-09-11T06:49:53.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.516 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.516 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:53.516 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818412+0000", 2024-09-11T06:49:53.516 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2625e-05 2024-09-11T06:49:53.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.516 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.516 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:53.516 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826205+0000", 2024-09-11T06:49:53.516 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0077930730000000002 2024-09-11T06:49:53.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.517 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.517 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:53.517 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826281+0000", 2024-09-11T06:49:53.517 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5876000000000003e-05 2024-09-11T06:49:53.517 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.517 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.517 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:53.517 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826384+0000", 2024-09-11T06:49:53.517 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103838 2024-09-11T06:49:53.517 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.517 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.517 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:53.517 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830642+0000", 2024-09-11T06:49:53.518 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004257083 2024-09-11T06:49:53.518 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.518 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.518 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:53.518 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831509+0000", 2024-09-11T06:49:53.518 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086730900000000003 2024-09-11T06:49:53.518 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.518 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.518 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:53.518 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831547+0000", 2024-09-11T06:49:53.518 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7793000000000002e-05 2024-09-11T06:49:53.518 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.518 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.518 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:53.519 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831612+0000", 2024-09-11T06:49:53.519 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5691000000000005e-05 2024-09-11T06:49:53.519 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.519 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:53.519 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.519 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.519 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.519 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:53.519 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:53.519 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 46.60122389, 2024-09-11T06:49:53.519 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016098024999999998, 2024-09-11T06:49:53.519 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:53.519 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:53.520 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:53.520 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:53.520 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:53.520 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:53.520 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 140 2024-09-11T06:49:53.520 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.520 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:53.520 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.520 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:53.520 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:53.520 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.520 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.520 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.521 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:53.521 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:53.521 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.521 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.521 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.521 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:53.521 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:53.521 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5199999999999998e-07 2024-09-11T06:49:53.521 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.521 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.521 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:53.521 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818479+0000", 2024-09-11T06:49:53.521 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0368000000000002e-05 2024-09-11T06:49:53.522 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.522 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.522 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:53.522 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818487+0000", 2024-09-11T06:49:53.522 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2339999999999994e-06 2024-09-11T06:49:53.522 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.522 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.522 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:53.522 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818491+0000", 2024-09-11T06:49:53.522 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4329999999999997e-06 2024-09-11T06:49:53.522 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.522 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.522 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:53.522 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828080+0000", 2024-09-11T06:49:53.523 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0095890449999999992 2024-09-11T06:49:53.523 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.523 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.523 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:53.523 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828130+0000", 2024-09-11T06:49:53.523 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9239000000000001e-05 2024-09-11T06:49:53.523 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.523 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.523 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:53.523 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828208+0000", 2024-09-11T06:49:53.523 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.852e-05 2024-09-11T06:49:53.523 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.523 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.524 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:53.524 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831625+0000", 2024-09-11T06:49:53.524 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034169410000000002 2024-09-11T06:49:53.524 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.524 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.524 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:53.524 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834411+0000", 2024-09-11T06:49:53.524 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002786111 2024-09-11T06:49:53.524 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.524 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.524 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:53.524 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834457+0000", 2024-09-11T06:49:53.524 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6041000000000002e-05 2024-09-11T06:49:53.525 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.525 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.525 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:53.525 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834516+0000", 2024-09-11T06:49:53.525 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8740999999999999e-05 2024-09-11T06:49:53.525 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.525 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:53.525 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.525 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.525 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.525 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:53.525 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:49:53.525 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 46.601142823000004, 2024-09-11T06:49:53.525 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016588772000000002, 2024-09-11T06:49:53.526 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:53.526 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:53.526 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:53.526 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:53.526 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:53.526 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:53.526 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 141 2024-09-11T06:49:53.526 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.526 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:53.526 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.526 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:53.526 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:49:53.526 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.527 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.527 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.527 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:53.527 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:49:53.527 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.527 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.527 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.527 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:53.527 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818500+0000", 2024-09-11T06:49:53.527 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0800000000000005e-07 2024-09-11T06:49:53.527 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.528 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.528 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:53.528 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818505+0000", 2024-09-11T06:49:53.528 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2429999999999996e-06 2024-09-11T06:49:53.528 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.528 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.528 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:53.528 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818510+0000", 2024-09-11T06:49:53.528 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9339999999999997e-06 2024-09-11T06:49:53.528 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.528 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.528 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:53.529 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818514+0000", 2024-09-11T06:49:53.529 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7270000000000001e-06 2024-09-11T06:49:53.529 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.529 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.529 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:53.529 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828711+0000", 2024-09-11T06:49:53.529 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010197031 2024-09-11T06:49:53.529 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.529 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.529 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:53.529 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828853+0000", 2024-09-11T06:49:53.529 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014201200000000001 2024-09-11T06:49:53.530 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.530 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.530 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:53.530 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.829115+0000", 2024-09-11T06:49:53.530 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026257500000000003 2024-09-11T06:49:53.530 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.530 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.530 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:53.530 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831629+0000", 2024-09-11T06:49:53.530 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025138169999999998 2024-09-11T06:49:53.530 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.530 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.530 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:53.530 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834991+0000", 2024-09-11T06:49:53.531 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033624459999999998 2024-09-11T06:49:53.531 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.531 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.531 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:53.531 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835032+0000", 2024-09-11T06:49:53.531 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1001000000000003e-05 2024-09-11T06:49:53.531 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.531 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.531 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:53.531 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835088+0000", 2024-09-11T06:49:53.531 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5478e-05 2024-09-11T06:49:53.531 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.531 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:53.531 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.531 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.532 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.532 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:49:53.532 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:49:53.532 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 45.977507920000001, 2024-09-11T06:49:53.532 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011161912, 2024-09-11T06:49:53.532 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:53.532 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:53.532 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:53.532 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:53.532 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:53.532 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:53.532 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 158 2024-09-11T06:49:53.532 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.532 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:53.532 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.533 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:53.533 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:49:53.533 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.533 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.533 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.533 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:53.533 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:49:53.533 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.533 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.533 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.533 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:53.533 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442135+0000", 2024-09-11T06:49:53.533 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3e-07 2024-09-11T06:49:53.534 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.534 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.534 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:53.534 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442259+0000", 2024-09-11T06:49:53.534 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000123903 2024-09-11T06:49:53.534 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.534 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.534 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:53.534 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442289+0000", 2024-09-11T06:49:53.534 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0833000000000001e-05 2024-09-11T06:49:53.534 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.534 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.534 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:53.534 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442301+0000", 2024-09-11T06:49:53.535 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1433e-05 2024-09-11T06:49:53.535 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.535 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.535 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:53.535 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449796+0000", 2024-09-11T06:49:53.535 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0074954959999999999 2024-09-11T06:49:53.535 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.535 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.535 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:53.535 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449917+0000", 2024-09-11T06:49:53.535 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121033 2024-09-11T06:49:53.535 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.535 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.535 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:53.536 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.450166+0000", 2024-09-11T06:49:53.536 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00024832400000000002 2024-09-11T06:49:53.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.536 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.536 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:53.536 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.452630+0000", 2024-09-11T06:49:53.536 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002464188 2024-09-11T06:49:53.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.536 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.536 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:53.536 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453149+0000", 2024-09-11T06:49:53.536 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00051860999999999997 2024-09-11T06:49:53.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.537 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.537 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:53.537 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453234+0000", 2024-09-11T06:49:53.537 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.5865000000000005e-05 2024-09-11T06:49:53.537 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.537 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.537 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:53.537 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453296+0000", 2024-09-11T06:49:53.537 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1496999999999998e-05 2024-09-11T06:49:53.537 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.537 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:53.537 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.537 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.538 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.538 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:49:53.538 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:49:53.538 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 45.977321818, 2024-09-11T06:49:53.538 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011561900999999999, 2024-09-11T06:49:53.538 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:53.538 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:53.538 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:53.538 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:53.538 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:53.538 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:53.538 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 159 2024-09-11T06:49:53.538 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.539 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:53.539 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.539 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:53.539 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:49:53.539 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.539 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.539 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:53.539 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:49:53.539 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:53.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.539 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.539 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:53.540 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442321+0000", 2024-09-11T06:49:53.540 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8400000000000003e-07 2024-09-11T06:49:53.540 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.540 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.540 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:53.540 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442328+0000", 2024-09-11T06:49:53.540 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5130000000000002e-06 2024-09-11T06:49:53.540 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.540 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.540 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:53.540 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442341+0000", 2024-09-11T06:49:53.540 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.363e-05 2024-09-11T06:49:53.540 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.541 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.541 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:53.541 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442349+0000", 2024-09-11T06:49:53.541 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4760000000000001e-06 2024-09-11T06:49:53.541 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.541 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.541 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:53.541 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451257+0000", 2024-09-11T06:49:53.541 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089088569999999992 2024-09-11T06:49:53.541 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.541 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.541 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:53.541 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451389+0000", 2024-09-11T06:49:53.541 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013110899999999999 2024-09-11T06:49:53.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.542 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.542 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:53.542 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451608+0000", 2024-09-11T06:49:53.542 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000219588 2024-09-11T06:49:53.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.542 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.542 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:53.542 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453517+0000", 2024-09-11T06:49:53.542 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0019088779999999999 2024-09-11T06:49:53.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.542 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.542 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:53.543 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453727+0000", 2024-09-11T06:49:53.543 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020955100000000001 2024-09-11T06:49:53.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.543 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.543 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:53.543 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453815+0000", 2024-09-11T06:49:53.543 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8858999999999997e-05 2024-09-11T06:49:53.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:53.543 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:53.543 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:53.543 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453882+0000", 2024-09-11T06:49:53.543 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.6556000000000004e-05 2024-09-11T06:49:53.543 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.543 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:53.543 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.544 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:53.544 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:53.544 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:53.545 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:49:53.546 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:53.541+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:53.645 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:53.641+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:53.713 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:53.713 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:53.713 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:49:53.713 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:53.727 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:49:53.771 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:53.769+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:53.847 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:53.845+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:53.891 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:53.891 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:53.892 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:49:53.892 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:49:53.892 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:53.905 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:49:53.949 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:53.945+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:54.049 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:54.049+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:54.070 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:54.070 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:49:54.070 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:49:54.070 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [] 2024-09-11T06:49:54.070 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:54.084 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:49:54.150 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:54.149+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:54.207 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:54.207 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:54.207 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:49:54.207 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:54.221 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:49:54.270 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:54.269+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:54.343 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:54.343 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:54.343 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:49:54.343 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:49:54.344 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:54.351 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:54.349+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:54.357 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:49:54.452 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:54.449+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:54.526 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:54.527 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:49:54.527 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:49:54.527 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:49:54.527 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.527 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:49:54.527 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:49:54.527 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 43.297297884000002, 2024-09-11T06:49:54.527 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010553425999999999, 2024-09-11T06:49:54.527 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:54.527 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:54.527 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:54.528 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:54.528 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:49:54.528 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:49:54.528 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:49:54.528 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.528 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:54.528 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.528 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:54.528 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:49:54.528 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:54.528 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.528 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.529 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:54.529 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:49:54.529 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:54.529 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.529 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.529 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:54.529 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230274+0000", 2024-09-11T06:49:54.529 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.533e-06 2024-09-11T06:49:54.529 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.529 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.529 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:54.529 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230285+0000", 2024-09-11T06:49:54.529 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0540000000000001e-05 2024-09-11T06:49:54.530 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.530 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.530 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:54.530 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230309+0000", 2024-09-11T06:49:54.530 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4223000000000002e-05 2024-09-11T06:49:54.530 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.530 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.530 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:54.530 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230327+0000", 2024-09-11T06:49:54.530 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:49:54.530 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.530 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.530 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:54.530 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230478+0000", 2024-09-11T06:49:54.530 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000151459 2024-09-11T06:49:54.531 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.531 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.531 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:49:54.531 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230490+0000", 2024-09-11T06:49:54.531 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.147e-05 2024-09-11T06:49:54.531 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.531 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.531 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:54.531 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231803+0000", 2024-09-11T06:49:54.531 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001313158 2024-09-11T06:49:54.531 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.531 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.531 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:49:54.531 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231817+0000", 2024-09-11T06:49:54.532 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4197e-05 2024-09-11T06:49:54.532 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.532 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.532 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:54.532 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.235945+0000", 2024-09-11T06:49:54.532 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0041282480000000002 2024-09-11T06:49:54.532 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.532 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.532 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:54.532 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236312+0000", 2024-09-11T06:49:54.532 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036652700000000001 2024-09-11T06:49:54.532 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.532 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.533 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:49:54.533 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236689+0000", 2024-09-11T06:49:54.533 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000377117 2024-09-11T06:49:54.533 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.533 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.533 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:54.533 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238460+0000", 2024-09-11T06:49:54.533 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001770918 2024-09-11T06:49:54.533 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.533 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.533 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:54.533 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240627+0000", 2024-09-11T06:49:54.533 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021670779999999998 2024-09-11T06:49:54.533 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.534 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.534 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:54.534 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240751+0000", 2024-09-11T06:49:54.534 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012384900000000001 2024-09-11T06:49:54.534 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.534 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.534 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:54.534 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240821+0000", 2024-09-11T06:49:54.534 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0516e-05 2024-09-11T06:49:54.535 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:54.535 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:54.535 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:54.535 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.535 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.535 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop_reply(client.4230.0:1 2.3 e16/15 ondisk, result = 0)", 2024-09-11T06:49:54.535 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.238282+0000", 2024-09-11T06:49:54.535 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 43.289283558999998, 2024-09-11T06:49:54.535 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018970799999999999, 2024-09-11T06:49:54.535 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:54.535 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:54.535 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:49:54.535 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:54.536 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.536 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:54.536 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238282+0000", 2024-09-11T06:49:54.536 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:54.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.536 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.536 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:54.536 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238282+0000", 2024-09-11T06:49:54.536 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:54.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.536 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.536 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:54.536 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238284+0000", 2024-09-11T06:49:54.536 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:49:54.537 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.537 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.537 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:54.537 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238292+0000", 2024-09-11T06:49:54.537 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2390000000000002e-06 2024-09-11T06:49:54.537 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.537 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.537 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:54.537 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238314+0000", 2024-09-11T06:49:54.537 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1776999999999998e-05 2024-09-11T06:49:54.537 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.537 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.537 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:54.538 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238339+0000", 2024-09-11T06:49:54.538 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5119000000000001e-05 2024-09-11T06:49:54.538 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.538 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.538 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:54.538 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238438+0000", 2024-09-11T06:49:54.538 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9553e-05 2024-09-11T06:49:54.538 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.538 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.538 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:54.538 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238451+0000", 2024-09-11T06:49:54.538 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2286000000000001e-05 2024-09-11T06:49:54.538 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.538 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.539 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:54.539 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238472+0000", 2024-09-11T06:49:54.539 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1166999999999999e-05 2024-09-11T06:49:54.539 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:54.539 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:54.539 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:54.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.539 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.539 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:49:54.539 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.244627+0000", 2024-09-11T06:49:54.539 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 43.282938860999998, 2024-09-11T06:49:54.539 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00046141999999999998, 2024-09-11T06:49:54.540 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:54.540 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:54.540 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:49:54.540 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:54.540 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:49:54.540 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:49:54.541 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 2 2024-09-11T06:49:54.541 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.541 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:54.541 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.541 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:54.541 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244627+0000", 2024-09-11T06:49:54.542 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:54.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.542 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.542 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:54.542 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244627+0000", 2024-09-11T06:49:54.542 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:54.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.543 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.543 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:54.543 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244629+0000", 2024-09-11T06:49:54.543 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.95e-06 2024-09-11T06:49:54.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.543 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.543 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:54.544 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244640+0000", 2024-09-11T06:49:54.544 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1494e-05 2024-09-11T06:49:54.544 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.544 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.544 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:54.544 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244664+0000", 2024-09-11T06:49:54.544 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4006e-05 2024-09-11T06:49:54.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.545 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.545 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:54.545 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244684+0000", 2024-09-11T06:49:54.545 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.961e-05 2024-09-11T06:49:54.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.546 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.546 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:54.546 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244862+0000", 2024-09-11T06:49:54.546 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017863500000000001 2024-09-11T06:49:54.546 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.546 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.547 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:54.547 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245088+0000", 2024-09-11T06:49:54.547 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000225725 2024-09-11T06:49:54.547 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:54.547 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:54.547 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:54.547 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.548 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.548 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:49:54.548 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.245492+0000", 2024-09-11T06:49:54.548 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 43.282073918000002, 2024-09-11T06:49:54.548 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00042092600000000001, 2024-09-11T06:49:54.548 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:54.548 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:54.548 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:49:54.548 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:54.548 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:49:54.548 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:49:54.548 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 3 2024-09-11T06:49:54.549 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.549 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:54.549 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.549 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:54.549 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245492+0000", 2024-09-11T06:49:54.549 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:54.549 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.549 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.549 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:54.549 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245492+0000", 2024-09-11T06:49:54.549 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:54.549 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.549 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.549 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:54.550 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245493+0000", 2024-09-11T06:49:54.550 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.657e-06 2024-09-11T06:49:54.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.550 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:54.550 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245501+0000", 2024-09-11T06:49:54.550 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2490000000000004e-06 2024-09-11T06:49:54.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.550 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:54.550 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245523+0000", 2024-09-11T06:49:54.550 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2016999999999999e-05 2024-09-11T06:49:54.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.551 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:54.551 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245548+0000", 2024-09-11T06:49:54.551 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5060000000000001e-05 2024-09-11T06:49:54.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.551 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.551 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:54.551 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245720+0000", 2024-09-11T06:49:54.551 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017190200000000001 2024-09-11T06:49:54.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.551 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.551 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:54.551 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245913+0000", 2024-09-11T06:49:54.551 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193041 2024-09-11T06:49:54.552 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:54.552 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:54.552 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:54.552 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.552 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.552 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-11T06:49:54.552 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:49:54.552 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 42.295127041000001, 2024-09-11T06:49:54.552 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069904770000000001, 2024-09-11T06:49:54.552 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:54.552 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:54.552 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:54.552 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:54.553 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:49:54.553 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:49:54.553 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 5 2024-09-11T06:49:54.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.553 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:54.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.553 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:54.553 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:49:54.553 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:54.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.553 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:54.553 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:49:54.554 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:54.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.554 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.554 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:54.554 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232442+0000", 2024-09-11T06:49:54.554 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.044e-06 2024-09-11T06:49:54.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.554 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.554 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:54.554 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232460+0000", 2024-09-11T06:49:54.554 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7863000000000002e-05 2024-09-11T06:49:54.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.554 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.554 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:54.555 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232492+0000", 2024-09-11T06:49:54.555 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2812999999999999e-05 2024-09-11T06:49:54.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.555 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.555 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:54.555 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232510+0000", 2024-09-11T06:49:54.555 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7552999999999999e-05 2024-09-11T06:49:54.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.555 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.555 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:54.555 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232640+0000", 2024-09-11T06:49:54.555 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013029900000000001 2024-09-11T06:49:54.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.555 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.556 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:49:54.556 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232650+0000", 2024-09-11T06:49:54.556 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0023999999999999e-05 2024-09-11T06:49:54.556 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.556 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.556 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:54.556 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235641+0000", 2024-09-11T06:49:54.556 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00299089 2024-09-11T06:49:54.556 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.556 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.556 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:54.556 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235980+0000", 2024-09-11T06:49:54.556 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000338907 2024-09-11T06:49:54.557 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.557 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.557 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:49:54.557 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.236351+0000", 2024-09-11T06:49:54.557 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037087699999999998 2024-09-11T06:49:54.557 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.557 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.557 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:54.557 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238074+0000", 2024-09-11T06:49:54.557 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001722859 2024-09-11T06:49:54.557 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.557 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.557 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:54.558 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239160+0000", 2024-09-11T06:49:54.558 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001086074 2024-09-11T06:49:54.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.558 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.558 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:54.558 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239299+0000", 2024-09-11T06:49:54.558 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013933500000000001 2024-09-11T06:49:54.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.558 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.558 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:54.558 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239429+0000", 2024-09-11T06:49:54.558 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129939 2024-09-11T06:49:54.558 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:54.558 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:54.559 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:54.559 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.559 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.559 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop_reply(client.4230.0:5 2.2 e16/15 ondisk, result = 0)", 2024-09-11T06:49:54.559 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:12.238911+0000", 2024-09-11T06:49:54.559 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 42.288654753000003, 2024-09-11T06:49:54.559 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00053624900000000004, 2024-09-11T06:49:54.559 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:54.559 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:54.559 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:49:54.559 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:54.559 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.559 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:54.559 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238911+0000", 2024-09-11T06:49:54.560 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:54.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.560 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:54.560 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238911+0000", 2024-09-11T06:49:54.560 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:54.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.560 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:54.560 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238913+0000", 2024-09-11T06:49:54.560 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2069999999999998e-06 2024-09-11T06:49:54.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.561 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:54.561 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238923+0000", 2024-09-11T06:49:54.561 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0360000000000001e-05 2024-09-11T06:49:54.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.561 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.561 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:54.561 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238953+0000", 2024-09-11T06:49:54.561 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9258999999999999e-05 2024-09-11T06:49:54.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.561 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.561 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:54.561 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238967+0000", 2024-09-11T06:49:54.562 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3797e-05 2024-09-11T06:49:54.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.562 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:54.562 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239117+0000", 2024-09-11T06:49:54.562 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000150765 2024-09-11T06:49:54.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.562 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:54.562 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239143+0000", 2024-09-11T06:49:54.562 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5514e-05 2024-09-11T06:49:54.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.562 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:54.563 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239447+0000", 2024-09-11T06:49:54.563 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00030434700000000001 2024-09-11T06:49:54.563 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:54.563 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:54.563 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:54.563 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.563 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-11T06:49:54.563 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:49:54.563 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 41.281039088999997, 2024-09-11T06:49:54.563 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069253250000000004, 2024-09-11T06:49:54.563 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:54.563 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:54.564 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:54.564 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:54.564 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:49:54.564 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:49:54.564 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 7 2024-09-11T06:49:54.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.564 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:54.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.564 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:54.564 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:49:54.564 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:54.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.565 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.565 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:54.565 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:49:54.565 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:54.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.565 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.565 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:54.565 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246530+0000", 2024-09-11T06:49:54.565 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9469999999999999e-06 2024-09-11T06:49:54.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.565 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.565 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:54.565 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246545+0000", 2024-09-11T06:49:54.566 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5973000000000001e-05 2024-09-11T06:49:54.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.566 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.566 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:54.566 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246578+0000", 2024-09-11T06:49:54.566 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2759999999999998e-05 2024-09-11T06:49:54.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.566 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.566 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:54.566 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246595+0000", 2024-09-11T06:49:54.566 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6939000000000002e-05 2024-09-11T06:49:54.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.566 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.566 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:54.567 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246770+0000", 2024-09-11T06:49:54.567 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000174723 2024-09-11T06:49:54.567 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.567 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.567 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:49:54.567 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246789+0000", 2024-09-11T06:49:54.567 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9159000000000001e-05 2024-09-11T06:49:54.567 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.567 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.567 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:54.567 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250337+0000", 2024-09-11T06:49:54.567 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035480759999999998 2024-09-11T06:49:54.567 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.567 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.568 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:54.568 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250549+0000", 2024-09-11T06:49:54.568 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021192199999999999 2024-09-11T06:49:54.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.568 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.568 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:49:54.568 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250750+0000", 2024-09-11T06:49:54.568 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020104199999999999 2024-09-11T06:49:54.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.568 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.568 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:54.568 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.252455+0000", 2024-09-11T06:49:54.568 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001705249 2024-09-11T06:49:54.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.569 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:54.569 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253305+0000", 2024-09-11T06:49:54.569 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00084941200000000002 2024-09-11T06:49:54.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.569 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:54.569 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253388+0000", 2024-09-11T06:49:54.569 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3642999999999999e-05 2024-09-11T06:49:54.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.569 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:54.569 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253452+0000", 2024-09-11T06:49:54.570 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3479999999999995e-05 2024-09-11T06:49:54.570 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:54.570 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:54.570 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:54.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.570 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop_reply(client.4230.0:7 2.2 e18/15 ondisk, result = 0)", 2024-09-11T06:49:54.570 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:13.253128+0000", 2024-09-11T06:49:54.570 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 41.274437515000002, 2024-09-11T06:49:54.570 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034023700000000002, 2024-09-11T06:49:54.570 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:54.570 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:54.570 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:49:54.571 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:54.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.571 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:54.571 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253128+0000", 2024-09-11T06:49:54.571 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:54.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.571 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:54.571 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253128+0000", 2024-09-11T06:49:54.571 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:54.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.571 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:54.572 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253135+0000", 2024-09-11T06:49:54.572 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.657e-06 2024-09-11T06:49:54.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.572 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:54.572 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253144+0000", 2024-09-11T06:49:54.572 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0869999999999996e-06 2024-09-11T06:49:54.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.572 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:54.573 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253167+0000", 2024-09-11T06:49:54.573 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3252e-05 2024-09-11T06:49:54.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.573 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:54.573 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253186+0000", 2024-09-11T06:49:54.573 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9239999999999999e-05 2024-09-11T06:49:54.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.573 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:54.574 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253282+0000", 2024-09-11T06:49:54.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.6093000000000003e-05 2024-09-11T06:49:54.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.574 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:54.574 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253294+0000", 2024-09-11T06:49:54.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1986e-05 2024-09-11T06:49:54.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.574 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:54.575 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253468+0000", 2024-09-11T06:49:54.575 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017392199999999999 2024-09-11T06:49:54.575 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:54.575 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:54.575 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:54.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.575 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPushReply(2.2 37/36 [PushReplyOp(2:5cea7035:::rbd_info:head)])", 2024-09-11T06:49:54.575 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:50.557624+0000", 2024-09-11T06:49:54.575 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 3.9699414499999999, 2024-09-11T06:49:54.575 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00069280800000000005, 2024-09-11T06:49:54.575 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:54.575 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:54.576 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:49:54.576 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:54.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.576 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:54.576 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:50.557624+0000", 2024-09-11T06:49:54.576 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:54.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.576 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:54.576 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:50.557624+0000", 2024-09-11T06:49:54.576 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:54.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:54.577 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:50.557625+0000", 2024-09-11T06:49:54.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.37e-07 2024-09-11T06:49:54.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:54.577 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:50.557633+0000", 2024-09-11T06:49:54.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6270000000000006e-06 2024-09-11T06:49:54.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:54.577 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:50.557650+0000", 2024-09-11T06:49:54.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6350000000000001e-05 2024-09-11T06:49:54.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.578 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:54.578 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:50.557667+0000", 2024-09-11T06:49:54.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7668000000000001e-05 2024-09-11T06:49:54.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.578 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:54.578 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:50.557832+0000", 2024-09-11T06:49:54.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016408699999999999 2024-09-11T06:49:54.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:54.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:54.579 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:54.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:50.558317+0000", 2024-09-11T06:49:54.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000485439 2024-09-11T06:49:54.579 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:54.579 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:54.579 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:54.579 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:54.579 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:54.579 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:54.580 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T06:49:54.580 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T06:49:54.580 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-11T06:49:54.580 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-11T06:49:54.582 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:49:54.583 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:54.581+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:54.683 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:54.681+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:54.709 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:54.709 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:54.709 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:49:54.709 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:54.726 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:49:54.784 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:54.785+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:54.839 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:54.839 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:54.839 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:49:54.839 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:49:54.839 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:54.854 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:49:54.885 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:54.881+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:54.899 INFO:tasks.ceph.mon.a.smithi071.stderr:2024-09-11T06:49:54.893+0000 7f1296554640 -1 mon.a@0(leader).osd e40 definitely_dead 0 2024-09-11T06:49:54.935 INFO:teuthology.orchestra.run.smithi071.stderr:marked in osd.1. 2024-09-11T06:49:54.945 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T06:49:54.959 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-11T06:49:54.980 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:54.980 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:49:54.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:49:54.980 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:49:54.980 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.980 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:54 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'32, mlcod=13'29)", 2024-09-11T06:49:54.980 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:49:54.980 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 48.768456239000002, 2024-09-11T06:49:54.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034271670000000001, 2024-09-11T06:49:54.980 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:54.981 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:54.981 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:54.981 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:54.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.981 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:54.981 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:49:54.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:54.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.981 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:54.981 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:49:54.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:54.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.982 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:54.982 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210361+0000", 2024-09-11T06:49:54.982 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1260000000000001e-06 2024-09-11T06:49:54.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.982 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:54.982 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210370+0000", 2024-09-11T06:49:54.982 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8200000000000003e-06 2024-09-11T06:49:54.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.982 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:54.982 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210390+0000", 2024-09-11T06:49:54.982 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0126e-05 2024-09-11T06:49:54.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:54.983 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210404+0000", 2024-09-11T06:49:54.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4589999999999999e-05 2024-09-11T06:49:54.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:54.983 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210538+0000", 2024-09-11T06:49:54.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000133719 2024-09-11T06:49:54.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:54.984 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210593+0000", 2024-09-11T06:49:54.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5016999999999999e-05 2024-09-11T06:49:54.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.984 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:54.984 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213695+0000", 2024-09-11T06:49:54.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003101866 2024-09-11T06:49:54.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:54.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213787+0000", 2024-09-11T06:49:54.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1903000000000004e-05 2024-09-11T06:49:54.985 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:54.985 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:54.985 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:54.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.985 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:55 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'33, mlcod=13'29)", 2024-09-11T06:49:54.986 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:49:54.986 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 48.767437000999998, 2024-09-11T06:49:54.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.005086091, 2024-09-11T06:49:54.986 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:54.986 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:54.986 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:54.986 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:54.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:54.986 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:49:54.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:54.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:54.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:49:54.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:54.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.987 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:54.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211380+0000", 2024-09-11T06:49:54.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.097e-06 2024-09-11T06:49:54.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.987 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:54.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211537+0000", 2024-09-11T06:49:54.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000157369 2024-09-11T06:49:54.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.988 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:54.988 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211559+0000", 2024-09-11T06:49:54.988 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000001e-05 2024-09-11T06:49:54.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.988 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.988 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:54.988 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211573+0000", 2024-09-11T06:49:54.988 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4946000000000001e-05 2024-09-11T06:49:54.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.988 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.988 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:54.988 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211676+0000", 2024-09-11T06:49:54.988 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0001026 2024-09-11T06:49:54.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:54.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211709+0000", 2024-09-11T06:49:54.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3071999999999998e-05 2024-09-11T06:49:54.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:54.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216379+0000", 2024-09-11T06:49:54.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0046693780000000001 2024-09-11T06:49:54.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:54.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216465+0000", 2024-09-11T06:49:54.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6469000000000002e-05 2024-09-11T06:49:54.990 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:54.990 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:54.990 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:54.990 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.990 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:56 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'34, mlcod=13'29)", 2024-09-11T06:49:54.990 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:49:54.990 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 48.766003142000002, 2024-09-11T06:49:54.990 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0060310629999999997, 2024-09-11T06:49:54.990 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:54.990 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:54.990 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:54.990 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:54.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.991 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:54.991 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:49:54.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:54.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.991 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:54.991 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:49:54.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:54.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.991 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:54.991 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212814+0000", 2024-09-11T06:49:54.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0869999999999999e-06 2024-09-11T06:49:54.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.992 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.992 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:54.992 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212822+0000", 2024-09-11T06:49:54.992 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0870000000000007e-06 2024-09-11T06:49:54.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.992 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.992 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:54.992 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212840+0000", 2024-09-11T06:49:54.992 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:49:54.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.992 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.992 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:54.992 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212854+0000", 2024-09-11T06:49:54.992 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4003e-05 2024-09-11T06:49:54.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.993 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:54.993 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213026+0000", 2024-09-11T06:49:54.993 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017207499999999999 2024-09-11T06:49:54.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.993 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:54.993 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213063+0000", 2024-09-11T06:49:54.993 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7187000000000002e-05 2024-09-11T06:49:54.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.993 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:54.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218762+0000", 2024-09-11T06:49:54.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0056993290000000004 2024-09-11T06:49:54.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.994 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:54.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218844+0000", 2024-09-11T06:49:54.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1701999999999998e-05 2024-09-11T06:49:54.994 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:54.994 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:54.994 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:54.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.994 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:60 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'38, mlcod=13'36)", 2024-09-11T06:49:54.994 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:49:54.995 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 48.752061793000003, 2024-09-11T06:49:54.995 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039254270000000004, 2024-09-11T06:49:54.995 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:54.995 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:54.995 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:54.995 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:54.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.995 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:54.995 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:49:54.995 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:54.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.995 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:54.995 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:49:54.996 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:54.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:54.996 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226756+0000", 2024-09-11T06:49:54.996 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-11T06:49:54.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:54.996 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226768+0000", 2024-09-11T06:49:54.996 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1919999999999999e-05 2024-09-11T06:49:54.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:54.997 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226797+0000", 2024-09-11T06:49:54.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9573e-05 2024-09-11T06:49:54.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.997 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.997 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:54.997 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226818+0000", 2024-09-11T06:49:54.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0432999999999999e-05 2024-09-11T06:49:54.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.997 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.997 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:54.997 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226909+0000", 2024-09-11T06:49:54.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0920000000000004e-05 2024-09-11T06:49:54.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.997 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:54.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226936+0000", 2024-09-11T06:49:54.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7472999999999999e-05 2024-09-11T06:49:54.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:54.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230596+0000", 2024-09-11T06:49:54.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0036602520000000001 2024-09-11T06:49:54.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:54.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230680+0000", 2024-09-11T06:49:54.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3106e-05 2024-09-11T06:49:54.999 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:54.999 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:54.999 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:54.999 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:54.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.999 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:107 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'69, mlcod=13'64)", 2024-09-11T06:49:54.999 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:49:54.999 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 48.535614336000002, 2024-09-11T06:49:54.999 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040728830000000002, 2024-09-11T06:49:54.999 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:54.999 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:54.999 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:54.999 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:54.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:54.999 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:55.000 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:49:55.000 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.000 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:55.000 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:49:55.000 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.000 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:55.000 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443203+0000", 2024-09-11T06:49:55.000 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1480000000000001e-06 2024-09-11T06:49:55.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.001 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:55.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443222+0000", 2024-09-11T06:49:55.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9040000000000001e-05 2024-09-11T06:49:55.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.001 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:55.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443246+0000", 2024-09-11T06:49:55.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4606000000000001e-05 2024-09-11T06:49:55.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.001 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:55.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443277+0000", 2024-09-11T06:49:55.002 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0207999999999999e-05 2024-09-11T06:49:55.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:55.002 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443340+0000", 2024-09-11T06:49:55.002 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3602000000000005e-05 2024-09-11T06:49:55.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:55.002 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443363+0000", 2024-09-11T06:49:55.002 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2331e-05 2024-09-11T06:49:55.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:55.003 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447216+0000", 2024-09-11T06:49:55.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0038539690000000001 2024-09-11T06:49:55.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:55.003 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447274+0000", 2024-09-11T06:49:55.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7979000000000002e-05 2024-09-11T06:49:55.003 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.003 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:55.003 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.003 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:111 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'73, mlcod=13'67)", 2024-09-11T06:49:55.003 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:49:55.004 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 48.529102706000003, 2024-09-11T06:49:55.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034372360000000002, 2024-09-11T06:49:55.004 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:55.004 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:55.004 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:55.004 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:55.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.004 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:55.004 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:49:55.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.004 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:55.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:49:55.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:55.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449714+0000", 2024-09-11T06:49:55.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5000000000000001e-07 2024-09-11T06:49:55.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:55.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449721+0000", 2024-09-11T06:49:55.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8530000000000003e-06 2024-09-11T06:49:55.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.006 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:55.006 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449736+0000", 2024-09-11T06:49:55.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5114e-05 2024-09-11T06:49:55.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.006 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:55.006 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449743+0000", 2024-09-11T06:49:55.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0210000000000002e-06 2024-09-11T06:49:55.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.006 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:55.006 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449804+0000", 2024-09-11T06:49:55.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0479999999999997e-05 2024-09-11T06:49:55.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.007 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:55.007 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449826+0000", 2024-09-11T06:49:55.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1922000000000001e-05 2024-09-11T06:49:55.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.007 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:55.007 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453104+0000", 2024-09-11T06:49:55.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003278652 2024-09-11T06:49:55.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.007 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:55.007 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453150+0000", 2024-09-11T06:49:55.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6244e-05 2024-09-11T06:49:55.008 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.008 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:55.008 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.008 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:114 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'76, mlcod=13'71)", 2024-09-11T06:49:55.008 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:49:55.008 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 48.524424635000003, 2024-09-11T06:49:55.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0042576660000000002, 2024-09-11T06:49:55.008 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:55.008 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:55.008 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:55.008 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:55.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.009 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:55.009 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:49:55.009 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.009 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:55.009 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:49:55.009 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.009 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:55.009 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454392+0000", 2024-09-11T06:49:55.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5999999999999991e-07 2024-09-11T06:49:55.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:55.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454399+0000", 2024-09-11T06:49:55.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.878e-06 2024-09-11T06:49:55.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:55.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454415+0000", 2024-09-11T06:49:55.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5659999999999999e-05 2024-09-11T06:49:55.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.011 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:55.011 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454422+0000", 2024-09-11T06:49:55.011 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:49:55.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.011 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:55.011 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454474+0000", 2024-09-11T06:49:55.011 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1860999999999997e-05 2024-09-11T06:49:55.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.011 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:55.011 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454497+0000", 2024-09-11T06:49:55.011 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3686e-05 2024-09-11T06:49:55.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.012 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.012 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:55.012 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458552+0000", 2024-09-11T06:49:55.012 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040544550000000002 2024-09-11T06:49:55.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.012 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.012 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:55.012 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458649+0000", 2024-09-11T06:49:55.012 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7005999999999999e-05 2024-09-11T06:49:55.012 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.012 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:55.012 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.013 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:115 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'77, mlcod=13'72)", 2024-09-11T06:49:55.013 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:55.013 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 48.523084023000003, 2024-09-11T06:49:55.013 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003355584, 2024-09-11T06:49:55.013 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:55.013 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:55.013 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:55.013 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:55.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.013 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:55.013 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:55.013 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.014 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:55.014 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:55.014 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.014 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:55.014 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:55.014 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5999999999999999e-07 2024-09-11T06:49:55.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.014 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:55.014 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455803+0000", 2024-09-11T06:49:55.015 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0310999999999998e-05 2024-09-11T06:49:55.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.015 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:55.015 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455815+0000", 2024-09-11T06:49:55.015 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2787e-05 2024-09-11T06:49:55.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.015 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:55.015 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455823+0000", 2024-09-11T06:49:55.015 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5959999999999998e-06 2024-09-11T06:49:55.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.015 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:55.016 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455875+0000", 2024-09-11T06:49:55.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2054e-05 2024-09-11T06:49:55.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.016 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.016 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:55.016 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455896+0000", 2024-09-11T06:49:55.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0554999999999999e-05 2024-09-11T06:49:55.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.016 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.016 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:55.016 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459000+0000", 2024-09-11T06:49:55.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003104842 2024-09-11T06:49:55.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.017 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:55.017 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459087+0000", 2024-09-11T06:49:55.017 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6978999999999995e-05 2024-09-11T06:49:55.017 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.017 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:55.017 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.017 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:120 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'82, mlcod=13'78)", 2024-09-11T06:49:55.017 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:49:55.017 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 48.512406738000003, 2024-09-11T06:49:55.017 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0064612549999999999, 2024-09-11T06:49:55.018 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:55.018 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:55.018 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:55.018 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:55.018 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.018 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:55.018 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:49:55.018 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.018 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.018 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:55.018 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:49:55.018 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:55.019 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466410+0000", 2024-09-11T06:49:55.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0389999999999999e-06 2024-09-11T06:49:55.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:55.019 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466496+0000", 2024-09-11T06:49:55.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5519000000000006e-05 2024-09-11T06:49:55.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:55.019 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466510+0000", 2024-09-11T06:49:55.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3842999999999999e-05 2024-09-11T06:49:55.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.020 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:55.020 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466517+0000", 2024-09-11T06:49:55.020 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5229999999999998e-06 2024-09-11T06:49:55.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.020 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:55.020 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466595+0000", 2024-09-11T06:49:55.020 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8019e-05 2024-09-11T06:49:55.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.020 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:55.021 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466612+0000", 2024-09-11T06:49:55.021 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7013000000000001e-05 2024-09-11T06:49:55.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.021 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.021 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:55.021 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472787+0000", 2024-09-11T06:49:55.021 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0061750269999999996 2024-09-11T06:49:55.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.021 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.021 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:55.021 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472870+0000", 2024-09-11T06:49:55.021 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3271999999999998e-05 2024-09-11T06:49:55.021 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.021 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:55.022 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.022 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:121 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'83, mlcod=13'78)", 2024-09-11T06:49:55.022 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:49:55.022 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 48.511181942, 2024-09-11T06:49:55.022 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055613069999999997, 2024-09-11T06:49:55.022 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:55.022 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:55.022 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:55.022 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:55.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.022 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:55.022 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:49:55.023 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.023 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.023 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:55.023 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:49:55.023 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.023 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.023 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:55.023 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467635+0000", 2024-09-11T06:49:55.023 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6899999999999996e-07 2024-09-11T06:49:55.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.023 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.024 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:55.024 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467727+0000", 2024-09-11T06:49:55.024 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1800999999999997e-05 2024-09-11T06:49:55.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.024 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.024 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:55.024 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467740+0000", 2024-09-11T06:49:55.024 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3747999999999999e-05 2024-09-11T06:49:55.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.024 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.024 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:55.024 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467753+0000", 2024-09-11T06:49:55.024 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2807e-05 2024-09-11T06:49:55.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.025 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.025 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:55.025 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467812+0000", 2024-09-11T06:49:55.025 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8381999999999998e-05 2024-09-11T06:49:55.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.025 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.025 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:55.025 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467828+0000", 2024-09-11T06:49:55.025 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6728e-05 2024-09-11T06:49:55.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.025 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.025 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:55.025 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473112+0000", 2024-09-11T06:49:55.026 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052833619999999998 2024-09-11T06:49:55.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.026 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:55.026 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473195+0000", 2024-09-11T06:49:55.026 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3609999999999994e-05 2024-09-11T06:49:55.026 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.026 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:55.026 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.026 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:131 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'89, mlcod=13'85)", 2024-09-11T06:49:55.026 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:49:55.026 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 48.159776227000002, 2024-09-11T06:49:55.027 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052492260000000001, 2024-09-11T06:49:55.027 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:55.027 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:55.027 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:55.027 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:55.027 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.027 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:55.027 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:49:55.027 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.027 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.027 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:55.027 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:49:55.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.028 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:55.028 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819041+0000", 2024-09-11T06:49:55.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4360000000000001e-06 2024-09-11T06:49:55.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.028 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:55.028 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819100+0000", 2024-09-11T06:49:55.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9225999999999998e-05 2024-09-11T06:49:55.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.029 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:55.029 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819128+0000", 2024-09-11T06:49:55.029 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.779e-05 2024-09-11T06:49:55.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.029 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:55.029 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819142+0000", 2024-09-11T06:49:55.029 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3573e-05 2024-09-11T06:49:55.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.029 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:55.029 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819263+0000", 2024-09-11T06:49:55.029 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000121086 2024-09-11T06:49:55.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:55.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819292+0000", 2024-09-11T06:49:55.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9363e-05 2024-09-11T06:49:55.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:55.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824185+0000", 2024-09-11T06:49:55.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0048924620000000002 2024-09-11T06:49:55.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:55.031 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824289+0000", 2024-09-11T06:49:55.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010429000000000001 2024-09-11T06:49:55.031 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.031 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:55.031 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.031 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:132 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'90, mlcod=13'85)", 2024-09-11T06:49:55.031 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:49:55.031 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 48.159001537000002, 2024-09-11T06:49:55.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.004532342, 2024-09-11T06:49:55.031 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:55.031 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:55.031 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:55.032 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:55.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:55.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:49:55.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:55.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:49:55.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:55.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819816+0000", 2024-09-11T06:49:55.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.61e-06 2024-09-11T06:49:55.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:55.033 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819827+0000", 2024-09-11T06:49:55.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1e-05 2024-09-11T06:49:55.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:55.033 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819851+0000", 2024-09-11T06:49:55.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4267e-05 2024-09-11T06:49:55.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.034 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:55.034 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819864+0000", 2024-09-11T06:49:55.034 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2767e-05 2024-09-11T06:49:55.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.034 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.034 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:55.034 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820071+0000", 2024-09-11T06:49:55.034 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000207061 2024-09-11T06:49:55.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.034 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.034 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:55.034 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820103+0000", 2024-09-11T06:49:55.034 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2326999999999997e-05 2024-09-11T06:49:55.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.035 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:55.035 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824302+0000", 2024-09-11T06:49:55.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0041986000000000002 2024-09-11T06:49:55.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.035 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:55.035 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824347+0000", 2024-09-11T06:49:55.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4709999999999997e-05 2024-09-11T06:49:55.035 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.035 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:55.035 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.036 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:133 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'91, mlcod=13'86)", 2024-09-11T06:49:55.036 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:49:55.036 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 48.158369878000002, 2024-09-11T06:49:55.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039421459999999997, 2024-09-11T06:49:55.036 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:55.036 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:55.036 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:55.036 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:55.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:55.036 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:49:55.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.037 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:55.037 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:49:55.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.037 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:55.037 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820448+0000", 2024-09-11T06:49:55.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.84e-06 2024-09-11T06:49:55.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.037 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:55.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820463+0000", 2024-09-11T06:49:55.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5356e-05 2024-09-11T06:49:55.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:55.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820499+0000", 2024-09-11T06:49:55.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.625e-05 2024-09-11T06:49:55.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:55.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820546+0000", 2024-09-11T06:49:55.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7017000000000001e-05 2024-09-11T06:49:55.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.039 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:55.039 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820882+0000", 2024-09-11T06:49:55.039 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033512 2024-09-11T06:49:55.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.039 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.039 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:55.039 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820941+0000", 2024-09-11T06:49:55.039 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8922999999999998e-05 2024-09-11T06:49:55.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.039 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.039 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:55.039 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824351+0000", 2024-09-11T06:49:55.039 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034101940000000001 2024-09-11T06:49:55.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.040 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.040 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:55.040 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824388+0000", 2024-09-11T06:49:55.040 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7446e-05 2024-09-11T06:49:55.040 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.040 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:55.040 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.040 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.040 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:134 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'92, mlcod=13'86)", 2024-09-11T06:49:55.040 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:49:55.040 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 48.157710760000001, 2024-09-11T06:49:55.040 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033248179999999998, 2024-09-11T06:49:55.041 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:55.041 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:55.041 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:55.041 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:55.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:55.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:49:55.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:55.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:49:55.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:55.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821107+0000", 2024-09-11T06:49:55.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7430000000000001e-06 2024-09-11T06:49:55.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:55.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821205+0000", 2024-09-11T06:49:55.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7666999999999998e-05 2024-09-11T06:49:55.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:55.043 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821234+0000", 2024-09-11T06:49:55.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9756000000000001e-05 2024-09-11T06:49:55.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.043 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:55.043 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821254+0000", 2024-09-11T06:49:55.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9446e-05 2024-09-11T06:49:55.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.043 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:55.043 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821846+0000", 2024-09-11T06:49:55.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00059187599999999997 2024-09-11T06:49:55.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.044 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:55.044 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821894+0000", 2024-09-11T06:49:55.044 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8766e-05 2024-09-11T06:49:55.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.044 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.044 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:55.044 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824392+0000", 2024-09-11T06:49:55.044 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0024977509999999999 2024-09-11T06:49:55.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.044 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.044 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:55.044 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824430+0000", 2024-09-11T06:49:55.044 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7812999999999999e-05 2024-09-11T06:49:55.045 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.045 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:55.045 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.045 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.045 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:137 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'95, mlcod=13'86)", 2024-09-11T06:49:55.045 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:49:55.045 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 48.155248618999998, 2024-09-11T06:49:55.045 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003084299, 2024-09-11T06:49:55.045 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:55.045 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:55.045 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:55.045 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:55.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:55.046 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:49:55.046 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:55.046 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:49:55.046 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:55.046 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823568+0000", 2024-09-11T06:49:55.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1030000000000001e-06 2024-09-11T06:49:55.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.047 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:55.047 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823662+0000", 2024-09-11T06:49:55.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3529999999999994e-05 2024-09-11T06:49:55.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.047 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:55.047 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823686+0000", 2024-09-11T06:49:55.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4358999999999998e-05 2024-09-11T06:49:55.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.047 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:55.048 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823701+0000", 2024-09-11T06:49:55.048 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4847e-05 2024-09-11T06:49:55.048 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.048 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.048 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:55.048 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824518+0000", 2024-09-11T06:49:55.048 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00081716700000000005 2024-09-11T06:49:55.048 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.048 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.048 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:55.048 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824546+0000", 2024-09-11T06:49:55.048 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7860000000000001e-05 2024-09-11T06:49:55.048 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.048 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.049 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:55.049 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826558+0000", 2024-09-11T06:49:55.049 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0020120009999999998 2024-09-11T06:49:55.049 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.049 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.049 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:55.049 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826652+0000", 2024-09-11T06:49:55.049 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3431999999999996e-05 2024-09-11T06:49:55.049 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.049 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:55.049 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.049 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.049 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.049 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:139 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'97, mlcod=13'86)", 2024-09-11T06:49:55.050 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:49:55.050 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 48.152959899999999, 2024-09-11T06:49:55.050 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003732221, 2024-09-11T06:49:55.050 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:55.050 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:55.050 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:55.050 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:55.050 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.050 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:55.050 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:49:55.050 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.050 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:55.051 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:49:55.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:55.051 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825858+0000", 2024-09-11T06:49:55.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5829999999999999e-06 2024-09-11T06:49:55.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:55.051 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825874+0000", 2024-09-11T06:49:55.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5967000000000001e-05 2024-09-11T06:49:55.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.052 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.052 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:55.052 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825909+0000", 2024-09-11T06:49:55.052 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5692999999999999e-05 2024-09-11T06:49:55.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.052 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.052 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:55.052 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825923+0000", 2024-09-11T06:49:55.052 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4080000000000001e-05 2024-09-11T06:49:55.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.052 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.052 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:55.052 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826010+0000", 2024-09-11T06:49:55.053 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6248999999999994e-05 2024-09-11T06:49:55.053 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.053 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.053 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:55.053 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826038+0000", 2024-09-11T06:49:55.053 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8010000000000001e-05 2024-09-11T06:49:55.053 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.053 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.053 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:55.053 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829499+0000", 2024-09-11T06:49:55.053 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003461783 2024-09-11T06:49:55.053 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.053 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.054 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:55.054 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829588+0000", 2024-09-11T06:49:55.054 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8856000000000004e-05 2024-09-11T06:49:55.054 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.054 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:55.054 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.054 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.054 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.054 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:140 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'98, mlcod=13'93)", 2024-09-11T06:49:55.054 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:49:55.054 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 48.151109171000002, 2024-09-11T06:49:55.054 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0057995929999999996, 2024-09-11T06:49:55.054 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:55.054 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:55.055 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:55.055 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:55.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.055 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:55.055 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:49:55.055 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.055 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:55.055 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:49:55.055 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.055 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:55.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827708+0000", 2024-09-11T06:49:55.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.547e-06 2024-09-11T06:49:55.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.056 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.056 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:55.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827765+0000", 2024-09-11T06:49:55.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6266e-05 2024-09-11T06:49:55.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.056 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.056 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:55.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827791+0000", 2024-09-11T06:49:55.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6098999999999999e-05 2024-09-11T06:49:55.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.057 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.057 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:55.057 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827806+0000", 2024-09-11T06:49:55.057 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5537e-05 2024-09-11T06:49:55.057 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.057 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.057 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:55.057 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827928+0000", 2024-09-11T06:49:55.057 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000122012 2024-09-11T06:49:55.057 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.057 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.057 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:55.057 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827961+0000", 2024-09-11T06:49:55.057 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2663999999999998e-05 2024-09-11T06:49:55.058 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.058 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.058 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:55.058 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833409+0000", 2024-09-11T06:49:55.058 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054479350000000001 2024-09-11T06:49:55.058 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.058 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.058 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:55.058 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833506+0000", 2024-09-11T06:49:55.058 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7533000000000003e-05 2024-09-11T06:49:55.058 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.058 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:55.058 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.059 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.059 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.059 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:141 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'99, mlcod=13'93)", 2024-09-11T06:49:55.059 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:49:55.059 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 48.150215787999997, 2024-09-11T06:49:55.059 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055394709999999998, 2024-09-11T06:49:55.059 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:55.059 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:55.059 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:55.059 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:55.059 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.059 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:55.059 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:49:55.059 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.060 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.060 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:55.060 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:49:55.060 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.060 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.060 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:55.060 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828602+0000", 2024-09-11T06:49:55.060 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.46e-06 2024-09-11T06:49:55.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.060 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.060 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:55.061 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828618+0000", 2024-09-11T06:49:55.061 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6592999999999999e-05 2024-09-11T06:49:55.061 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.061 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.061 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:55.061 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828646+0000", 2024-09-11T06:49:55.061 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7552999999999999e-05 2024-09-11T06:49:55.061 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.061 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.061 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:55.061 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828663+0000", 2024-09-11T06:49:55.061 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.719e-05 2024-09-11T06:49:55.061 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.061 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.062 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:55.062 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829023+0000", 2024-09-11T06:49:55.062 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036054799999999998 2024-09-11T06:49:55.062 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.062 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.062 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:55.062 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829062+0000", 2024-09-11T06:49:55.062 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8865999999999997e-05 2024-09-11T06:49:55.062 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.062 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.062 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:55.062 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834078+0000", 2024-09-11T06:49:55.062 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050157480000000004 2024-09-11T06:49:55.062 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.063 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.063 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:55.063 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834140+0000", 2024-09-11T06:49:55.063 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1513000000000005e-05 2024-09-11T06:49:55.063 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.063 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:55.063 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.063 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.063 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.063 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:144 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'102, mlcod=13'98)", 2024-09-11T06:49:55.063 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:49:55.063 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 48.141055174999998, 2024-09-11T06:49:55.063 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003210348, 2024-09-11T06:49:55.064 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:55.064 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:55.064 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:55.064 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:55.064 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.064 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:55.064 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:49:55.064 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.064 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.064 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.064 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:55.064 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:49:55.064 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.064 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.065 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.065 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:55.065 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837768+0000", 2024-09-11T06:49:55.065 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9330000000000004e-06 2024-09-11T06:49:55.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.065 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.065 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:55.065 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837858+0000", 2024-09-11T06:49:55.065 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0185999999999993e-05 2024-09-11T06:49:55.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.065 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.065 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:55.065 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837890+0000", 2024-09-11T06:49:55.065 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2437000000000001e-05 2024-09-11T06:49:55.066 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.066 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.066 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:55.066 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837905+0000", 2024-09-11T06:49:55.066 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:49:55.066 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.066 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.066 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:55.066 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838019+0000", 2024-09-11T06:49:55.066 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000114336 2024-09-11T06:49:55.066 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.066 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.066 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:55.066 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838052+0000", 2024-09-11T06:49:55.066 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3139000000000002e-05 2024-09-11T06:49:55.067 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.067 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.067 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:55.067 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840914+0000", 2024-09-11T06:49:55.067 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028612780000000001 2024-09-11T06:49:55.067 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.067 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.067 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:55.067 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840971+0000", 2024-09-11T06:49:55.067 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7402999999999999e-05 2024-09-11T06:49:55.067 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.067 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:55.067 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.068 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.068 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.068 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:145 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'103, mlcod=13'98)", 2024-09-11T06:49:55.068 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:49:55.068 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 48.140263290999997, 2024-09-11T06:49:55.068 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054455409999999999, 2024-09-11T06:49:55.068 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:55.068 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:55.068 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:55.068 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:55.068 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.068 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:55.068 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:49:55.069 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.069 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.069 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.069 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:55.069 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:49:55.069 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:55.069 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.069 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.069 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:55.069 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838554+0000", 2024-09-11T06:49:55.069 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3430000000000001e-06 2024-09-11T06:49:55.069 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.069 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.069 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:55.070 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838640+0000", 2024-09-11T06:49:55.070 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5569999999999996e-05 2024-09-11T06:49:55.070 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.070 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.070 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:55.070 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838660+0000", 2024-09-11T06:49:55.070 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0576000000000001e-05 2024-09-11T06:49:55.070 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.070 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.070 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:55.070 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838674+0000", 2024-09-11T06:49:55.070 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3529999999999999e-05 2024-09-11T06:49:55.070 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.071 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.071 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:55.071 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838838+0000", 2024-09-11T06:49:55.071 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016468799999999999 2024-09-11T06:49:55.071 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.071 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.071 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:55.071 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838885+0000", 2024-09-11T06:49:55.071 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6884e-05 2024-09-11T06:49:55.071 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.071 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.071 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:55.071 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843901+0000", 2024-09-11T06:49:55.071 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050162310000000003 2024-09-11T06:49:55.072 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:55.072 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:55.072 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:55.072 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843998+0000", 2024-09-11T06:49:55.072 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.6718999999999993e-05 2024-09-11T06:49:55.072 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.072 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:55.072 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.072 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:55.072 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:55.072 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:55.074 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:49:55.074 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:55.069+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:55.175 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:55.173+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:55.233 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:55.233 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:55.233 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:49:55.233 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:55.246 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:49:55.292 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:55.289+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:55.363 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:55.364 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:55.364 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:49:55.364 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:49:55.364 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:55.372 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:49:55.377 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:55.377+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:55.378 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:49:55.478 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:55.473+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:55.519 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:49:55.519 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:49:55.535 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:49:55.535 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:49:55.553 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:49:55.553 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:49:55.555 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:55.555 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:55.568 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:49:55.574 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:49:55.574 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:49:55.579 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:55.577+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:55.594 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:49:55.595 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:49:55.615 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:49:55.615 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:49:55.680 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:55.677+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:55.687 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:55.687 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:55.701 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:49:55.788 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:55.785+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:55.859 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:55.859 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:55.872 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:49:55.882 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:55.881+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:55.942 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is set 2024-09-11T06:49:56.004 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:56.001+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:56.038 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:56.038 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:56.051 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:49:56.099 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:56.097+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:56.184 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:56.185 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:56.185 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:56.181+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:56.199 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:49:56.286 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:56.285+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:56.364 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:56.364 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:56.379 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:49:56.427 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:56.425+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:56.488 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:56.485+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:56.499 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:56.499 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:56.513 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:49:56.589 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:56.589+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:56.677 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:56.677 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:56.690 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:56.685+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:56.691 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:49:56.791 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:56.789+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:56.809 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:56.809 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:56.823 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:49:56.892 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:56.889+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:56.942 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:56.942 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:56.958 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:49:57.003 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:57.001+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:57.083 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:57.083 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:57.094 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:57.093+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:57.097 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:49:57.195 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:57.193+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:57.260 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:57.260 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:57.274 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:49:57.297 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:57.297+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:57.398 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:57.393+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:57.439 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:57.439 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:57.454 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:49:57.499 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:57.497+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:57.586 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:57.586 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:57.600 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:49:57.601 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:57.597+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:57.631 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:49:57.701 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:57.697+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:57.723 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:49:57.723 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:49:57.740 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:49:57.792 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:49:57.792 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:49:57.801 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:57.797+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:57.809 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:49:57.809 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:49:57.825 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:49:57.825 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:49:57.846 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:49:57.846 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:49:57.859 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:57.859 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:49:57.859 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:49:57.859 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:49:57.859 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.859 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:54 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'32, mlcod=13'29)", 2024-09-11T06:49:57.859 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:49:57.859 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 51.647496398999998, 2024-09-11T06:49:57.859 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034271670000000001, 2024-09-11T06:49:57.860 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:57.860 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:57.860 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:57.860 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:57.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.860 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:57.860 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:49:57.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.860 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:57.860 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:49:57.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.861 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:57.861 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210361+0000", 2024-09-11T06:49:57.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1260000000000001e-06 2024-09-11T06:49:57.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.861 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:57.861 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210370+0000", 2024-09-11T06:49:57.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8200000000000003e-06 2024-09-11T06:49:57.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.862 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:57.862 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210390+0000", 2024-09-11T06:49:57.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0126e-05 2024-09-11T06:49:57.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.862 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:57.862 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210404+0000", 2024-09-11T06:49:57.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4589999999999999e-05 2024-09-11T06:49:57.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.862 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:57.862 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210538+0000", 2024-09-11T06:49:57.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000133719 2024-09-11T06:49:57.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:57.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210593+0000", 2024-09-11T06:49:57.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5016999999999999e-05 2024-09-11T06:49:57.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:57.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213695+0000", 2024-09-11T06:49:57.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003101866 2024-09-11T06:49:57.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:57.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213787+0000", 2024-09-11T06:49:57.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1903000000000004e-05 2024-09-11T06:49:57.864 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.864 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:57.864 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.864 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:55 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'33, mlcod=13'29)", 2024-09-11T06:49:57.864 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:49:57.864 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 51.646477161, 2024-09-11T06:49:57.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.005086091, 2024-09-11T06:49:57.864 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:57.864 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:57.864 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:57.864 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:57.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:57.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:49:57.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:57.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:49:57.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:57.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211380+0000", 2024-09-11T06:49:57.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.097e-06 2024-09-11T06:49:57.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.866 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:57.866 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211537+0000", 2024-09-11T06:49:57.866 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000157369 2024-09-11T06:49:57.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.866 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:57.866 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211559+0000", 2024-09-11T06:49:57.866 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000001e-05 2024-09-11T06:49:57.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.867 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:57.867 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211573+0000", 2024-09-11T06:49:57.867 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4946000000000001e-05 2024-09-11T06:49:57.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.867 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:57.867 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211676+0000", 2024-09-11T06:49:57.867 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0001026 2024-09-11T06:49:57.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.867 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:57.867 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211709+0000", 2024-09-11T06:49:57.868 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3071999999999998e-05 2024-09-11T06:49:57.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.868 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:57.868 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216379+0000", 2024-09-11T06:49:57.868 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0046693780000000001 2024-09-11T06:49:57.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.868 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:57.868 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216465+0000", 2024-09-11T06:49:57.868 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6469000000000002e-05 2024-09-11T06:49:57.869 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.869 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:57.869 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.869 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:56 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'34, mlcod=13'29)", 2024-09-11T06:49:57.869 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:49:57.869 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 51.645043301999998, 2024-09-11T06:49:57.869 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0060310629999999997, 2024-09-11T06:49:57.869 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:57.870 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:57.870 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:57.870 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:57.870 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.870 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:57.870 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:49:57.870 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.870 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.870 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:57.870 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:49:57.870 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.870 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.871 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:57.871 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212814+0000", 2024-09-11T06:49:57.871 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0869999999999999e-06 2024-09-11T06:49:57.871 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.871 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.871 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:57.871 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212822+0000", 2024-09-11T06:49:57.871 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0870000000000007e-06 2024-09-11T06:49:57.871 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.872 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:57.872 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212840+0000", 2024-09-11T06:49:57.872 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:49:57.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.872 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:57.872 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212854+0000", 2024-09-11T06:49:57.872 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4003e-05 2024-09-11T06:49:57.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.872 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:57.872 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213026+0000", 2024-09-11T06:49:57.872 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017207499999999999 2024-09-11T06:49:57.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.873 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.873 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:57.873 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213063+0000", 2024-09-11T06:49:57.873 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7187000000000002e-05 2024-09-11T06:49:57.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.873 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.873 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:57.873 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218762+0000", 2024-09-11T06:49:57.874 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0056993290000000004 2024-09-11T06:49:57.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.874 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:57.874 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218844+0000", 2024-09-11T06:49:57.874 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1701999999999998e-05 2024-09-11T06:49:57.874 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.874 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:57.874 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.874 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:60 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'38, mlcod=13'36)", 2024-09-11T06:49:57.875 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:49:57.875 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 51.631101952999998, 2024-09-11T06:49:57.875 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039254270000000004, 2024-09-11T06:49:57.875 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:57.875 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:57.875 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:57.875 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:57.875 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.875 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:57.875 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:49:57.875 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.876 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:57.876 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:49:57.876 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.876 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:57.876 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226756+0000", 2024-09-11T06:49:57.876 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-11T06:49:57.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.876 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:57.877 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226768+0000", 2024-09-11T06:49:57.877 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1919999999999999e-05 2024-09-11T06:49:57.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.877 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:57.877 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226797+0000", 2024-09-11T06:49:57.877 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9573e-05 2024-09-11T06:49:57.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.877 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:57.877 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226818+0000", 2024-09-11T06:49:57.877 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0432999999999999e-05 2024-09-11T06:49:57.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.878 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:57.878 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226909+0000", 2024-09-11T06:49:57.878 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0920000000000004e-05 2024-09-11T06:49:57.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.878 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.878 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:57.878 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226936+0000", 2024-09-11T06:49:57.878 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7472999999999999e-05 2024-09-11T06:49:57.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.878 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.878 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:57.878 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230596+0000", 2024-09-11T06:49:57.878 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0036602520000000001 2024-09-11T06:49:57.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.879 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:57.879 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230680+0000", 2024-09-11T06:49:57.879 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3106e-05 2024-09-11T06:49:57.879 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.879 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:57.879 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.879 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.879 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:107 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'69, mlcod=13'64)", 2024-09-11T06:49:57.879 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:49:57.879 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 51.414654495999997, 2024-09-11T06:49:57.879 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040728830000000002, 2024-09-11T06:49:57.880 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:57.880 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:57.880 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:57.880 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:57.880 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.880 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:57.880 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:49:57.880 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.880 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.880 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:57.880 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:49:57.880 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.881 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:57.881 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443203+0000", 2024-09-11T06:49:57.881 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1480000000000001e-06 2024-09-11T06:49:57.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.884 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:57.884 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443222+0000", 2024-09-11T06:49:57.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9040000000000001e-05 2024-09-11T06:49:57.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.884 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:57.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443246+0000", 2024-09-11T06:49:57.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4606000000000001e-05 2024-09-11T06:49:57.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:57.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443277+0000", 2024-09-11T06:49:57.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0207999999999999e-05 2024-09-11T06:49:57.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:57.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443340+0000", 2024-09-11T06:49:57.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3602000000000005e-05 2024-09-11T06:49:57.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:57.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443363+0000", 2024-09-11T06:49:57.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2331e-05 2024-09-11T06:49:57.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:57.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447216+0000", 2024-09-11T06:49:57.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0038539690000000001 2024-09-11T06:49:57.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:57.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447274+0000", 2024-09-11T06:49:57.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7979000000000002e-05 2024-09-11T06:49:57.887 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.887 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:57.887 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.887 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:111 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'73, mlcod=13'67)", 2024-09-11T06:49:57.887 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:49:57.887 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 51.408142865999999, 2024-09-11T06:49:57.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034372360000000002, 2024-09-11T06:49:57.887 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:57.887 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:57.888 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:57.888 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:57.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.888 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:57.888 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:49:57.888 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.888 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:57.888 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:49:57.888 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.889 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:57.889 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449714+0000", 2024-09-11T06:49:57.889 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5000000000000001e-07 2024-09-11T06:49:57.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.889 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:57.889 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449721+0000", 2024-09-11T06:49:57.889 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8530000000000003e-06 2024-09-11T06:49:57.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.889 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:57.889 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449736+0000", 2024-09-11T06:49:57.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5114e-05 2024-09-11T06:49:57.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:57.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449743+0000", 2024-09-11T06:49:57.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0210000000000002e-06 2024-09-11T06:49:57.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:57.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449804+0000", 2024-09-11T06:49:57.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0479999999999997e-05 2024-09-11T06:49:57.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:57.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449826+0000", 2024-09-11T06:49:57.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1922000000000001e-05 2024-09-11T06:49:57.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:57.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453104+0000", 2024-09-11T06:49:57.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003278652 2024-09-11T06:49:57.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:57.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453150+0000", 2024-09-11T06:49:57.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6244e-05 2024-09-11T06:49:57.892 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.892 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:57.892 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.892 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:114 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'76, mlcod=13'71)", 2024-09-11T06:49:57.892 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:49:57.892 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 51.403464794999998, 2024-09-11T06:49:57.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0042576660000000002, 2024-09-11T06:49:57.893 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:57.893 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:57.893 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:57.893 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:57.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.893 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:57.893 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:49:57.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.893 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:57.894 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:49:57.894 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.894 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.894 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:57.894 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454392+0000", 2024-09-11T06:49:57.894 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5999999999999991e-07 2024-09-11T06:49:57.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.894 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.894 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:57.894 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454399+0000", 2024-09-11T06:49:57.894 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.878e-06 2024-09-11T06:49:57.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:57.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454415+0000", 2024-09-11T06:49:57.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5659999999999999e-05 2024-09-11T06:49:57.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:57.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454422+0000", 2024-09-11T06:49:57.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:49:57.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.896 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:57.896 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454474+0000", 2024-09-11T06:49:57.896 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1860999999999997e-05 2024-09-11T06:49:57.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.896 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:57.896 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454497+0000", 2024-09-11T06:49:57.896 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3686e-05 2024-09-11T06:49:57.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.896 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:57.896 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458552+0000", 2024-09-11T06:49:57.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040544550000000002 2024-09-11T06:49:57.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:57.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458649+0000", 2024-09-11T06:49:57.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7005999999999999e-05 2024-09-11T06:49:57.897 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.897 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:57.897 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.897 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:115 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'77, mlcod=13'72)", 2024-09-11T06:49:57.897 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:57.897 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 51.402124182999998, 2024-09-11T06:49:57.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003355584, 2024-09-11T06:49:57.898 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:57.898 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:57.898 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:57.898 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:57.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.898 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:57.898 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:57.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.898 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:57.899 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:57.899 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.899 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:57.899 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:49:57.899 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5999999999999999e-07 2024-09-11T06:49:57.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.899 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:57.899 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455803+0000", 2024-09-11T06:49:57.899 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0310999999999998e-05 2024-09-11T06:49:57.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:57.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455815+0000", 2024-09-11T06:49:57.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2787e-05 2024-09-11T06:49:57.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:57.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455823+0000", 2024-09-11T06:49:57.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5959999999999998e-06 2024-09-11T06:49:57.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:57.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455875+0000", 2024-09-11T06:49:57.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2054e-05 2024-09-11T06:49:57.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:57.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455896+0000", 2024-09-11T06:49:57.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0554999999999999e-05 2024-09-11T06:49:57.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:57.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459000+0000", 2024-09-11T06:49:57.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003104842 2024-09-11T06:49:57.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:57.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459087+0000", 2024-09-11T06:49:57.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6978999999999995e-05 2024-09-11T06:49:57.902 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.902 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:57.902 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.902 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:120 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'82, mlcod=13'78)", 2024-09-11T06:49:57.902 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:49:57.902 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 51.391446897999998, 2024-09-11T06:49:57.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0064612549999999999, 2024-09-11T06:49:57.902 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:57.902 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:57.902 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:57.903 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:57.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.903 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:57.903 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:49:57.903 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.903 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:57.903 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:49:57.903 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.903 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:57.903 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466410+0000", 2024-09-11T06:49:57.904 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0389999999999999e-06 2024-09-11T06:49:57.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.904 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:57.904 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466496+0000", 2024-09-11T06:49:57.904 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5519000000000006e-05 2024-09-11T06:49:57.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.904 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:57.904 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466510+0000", 2024-09-11T06:49:57.904 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3842999999999999e-05 2024-09-11T06:49:57.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.905 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:57.905 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466517+0000", 2024-09-11T06:49:57.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5229999999999998e-06 2024-09-11T06:49:57.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.905 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:57.905 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466595+0000", 2024-09-11T06:49:57.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8019e-05 2024-09-11T06:49:57.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.905 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:57.905 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466612+0000", 2024-09-11T06:49:57.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7013000000000001e-05 2024-09-11T06:49:57.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:57.906 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472787+0000", 2024-09-11T06:49:57.906 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0061750269999999996 2024-09-11T06:49:57.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:57.906 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472870+0000", 2024-09-11T06:49:57.906 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3271999999999998e-05 2024-09-11T06:49:57.906 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.906 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:57.906 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.907 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:121 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'83, mlcod=13'78)", 2024-09-11T06:49:57.907 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:49:57.907 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 51.390222102000003, 2024-09-11T06:49:57.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055613069999999997, 2024-09-11T06:49:57.907 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:57.907 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:57.907 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:57.907 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:57.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:57.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:49:57.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:57.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:49:57.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:57.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467635+0000", 2024-09-11T06:49:57.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6899999999999996e-07 2024-09-11T06:49:57.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.909 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:57.909 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467727+0000", 2024-09-11T06:49:57.909 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1800999999999997e-05 2024-09-11T06:49:57.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.909 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:57.909 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467740+0000", 2024-09-11T06:49:57.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3747999999999999e-05 2024-09-11T06:49:57.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.910 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:57.910 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467753+0000", 2024-09-11T06:49:57.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2807e-05 2024-09-11T06:49:57.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.910 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:57.910 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467812+0000", 2024-09-11T06:49:57.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8381999999999998e-05 2024-09-11T06:49:57.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.910 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:57.911 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467828+0000", 2024-09-11T06:49:57.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6728e-05 2024-09-11T06:49:57.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.911 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:57.911 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473112+0000", 2024-09-11T06:49:57.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052833619999999998 2024-09-11T06:49:57.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.912 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:57.912 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473195+0000", 2024-09-11T06:49:57.912 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3609999999999994e-05 2024-09-11T06:49:57.912 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.912 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:57.912 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.912 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.912 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:131 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'89, mlcod=13'85)", 2024-09-11T06:49:57.912 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:49:57.912 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 51.038816386999997, 2024-09-11T06:49:57.912 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052492260000000001, 2024-09-11T06:49:57.913 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:57.913 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:57.913 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:57.913 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:57.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:57.913 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:49:57.913 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:57.914 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:49:57.914 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.914 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.914 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:57.914 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819041+0000", 2024-09-11T06:49:57.914 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4360000000000001e-06 2024-09-11T06:49:57.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.914 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.914 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:57.914 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819100+0000", 2024-09-11T06:49:57.914 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9225999999999998e-05 2024-09-11T06:49:57.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.915 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:57.915 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819128+0000", 2024-09-11T06:49:57.915 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.779e-05 2024-09-11T06:49:57.915 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.915 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:57.915 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819142+0000", 2024-09-11T06:49:57.915 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3573e-05 2024-09-11T06:49:57.915 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.915 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:57.915 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819263+0000", 2024-09-11T06:49:57.915 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000121086 2024-09-11T06:49:57.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.916 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.916 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:57.916 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819292+0000", 2024-09-11T06:49:57.916 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9363e-05 2024-09-11T06:49:57.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.916 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.916 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:57.916 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824185+0000", 2024-09-11T06:49:57.916 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0048924620000000002 2024-09-11T06:49:57.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.916 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.916 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:57.917 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824289+0000", 2024-09-11T06:49:57.917 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010429000000000001 2024-09-11T06:49:57.917 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.917 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:57.917 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.917 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.917 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:132 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'90, mlcod=13'85)", 2024-09-11T06:49:57.917 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:49:57.917 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 51.038041696999997, 2024-09-11T06:49:57.917 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.004532342, 2024-09-11T06:49:57.917 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:57.917 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:57.918 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:57.918 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:57.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.918 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:57.918 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:49:57.918 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.918 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:57.918 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:49:57.918 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.918 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:57.919 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819816+0000", 2024-09-11T06:49:57.919 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.61e-06 2024-09-11T06:49:57.919 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.919 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.919 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:57.919 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819827+0000", 2024-09-11T06:49:57.919 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1e-05 2024-09-11T06:49:57.919 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.919 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.919 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:57.919 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819851+0000", 2024-09-11T06:49:57.919 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4267e-05 2024-09-11T06:49:57.919 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.920 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.920 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:57.920 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819864+0000", 2024-09-11T06:49:57.920 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2767e-05 2024-09-11T06:49:57.920 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.920 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.920 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:57.920 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820071+0000", 2024-09-11T06:49:57.920 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000207061 2024-09-11T06:49:57.920 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.920 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.921 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:57.921 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820103+0000", 2024-09-11T06:49:57.921 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2326999999999997e-05 2024-09-11T06:49:57.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.921 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.921 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:57.921 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824302+0000", 2024-09-11T06:49:57.921 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0041986000000000002 2024-09-11T06:49:57.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.921 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.921 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:57.921 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824347+0000", 2024-09-11T06:49:57.922 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4709999999999997e-05 2024-09-11T06:49:57.922 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.922 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:57.922 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.922 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.922 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:133 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'91, mlcod=13'86)", 2024-09-11T06:49:57.922 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:49:57.922 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 51.037410037999997, 2024-09-11T06:49:57.922 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039421459999999997, 2024-09-11T06:49:57.922 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:57.922 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:57.922 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:57.923 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:57.923 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.923 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:57.923 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:49:57.923 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.923 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.923 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.923 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:57.923 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:49:57.923 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.923 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.923 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.924 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:57.924 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820448+0000", 2024-09-11T06:49:57.924 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.84e-06 2024-09-11T06:49:57.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.924 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.924 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:57.924 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820463+0000", 2024-09-11T06:49:57.924 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5356e-05 2024-09-11T06:49:57.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.924 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.924 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:57.924 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820499+0000", 2024-09-11T06:49:57.924 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.625e-05 2024-09-11T06:49:57.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.925 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.925 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:57.925 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820546+0000", 2024-09-11T06:49:57.925 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7017000000000001e-05 2024-09-11T06:49:57.925 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.925 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.925 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:57.925 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820882+0000", 2024-09-11T06:49:57.925 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033512 2024-09-11T06:49:57.925 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.925 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.926 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:57.926 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820941+0000", 2024-09-11T06:49:57.926 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8922999999999998e-05 2024-09-11T06:49:57.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.926 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.926 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:57.926 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824351+0000", 2024-09-11T06:49:57.926 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034101940000000001 2024-09-11T06:49:57.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.926 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.926 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:57.926 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824388+0000", 2024-09-11T06:49:57.926 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7446e-05 2024-09-11T06:49:57.927 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.927 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:57.927 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.927 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.927 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:134 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'92, mlcod=13'86)", 2024-09-11T06:49:57.927 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:49:57.927 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 51.036750920000003, 2024-09-11T06:49:57.927 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033248179999999998, 2024-09-11T06:49:57.927 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:57.927 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:57.927 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:57.927 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:57.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.928 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:57.928 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:49:57.928 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.928 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:57.928 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:49:57.928 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.929 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:57.929 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821107+0000", 2024-09-11T06:49:57.929 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7430000000000001e-06 2024-09-11T06:49:57.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.929 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:57.929 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821205+0000", 2024-09-11T06:49:57.929 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7666999999999998e-05 2024-09-11T06:49:57.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.929 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:57.929 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821234+0000", 2024-09-11T06:49:57.929 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9756000000000001e-05 2024-09-11T06:49:57.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.930 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:57.930 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821254+0000", 2024-09-11T06:49:57.930 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9446e-05 2024-09-11T06:49:57.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.930 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:57.930 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821846+0000", 2024-09-11T06:49:57.930 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00059187599999999997 2024-09-11T06:49:57.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.930 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:57.931 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821894+0000", 2024-09-11T06:49:57.931 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8766e-05 2024-09-11T06:49:57.931 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.931 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.931 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:57.931 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824392+0000", 2024-09-11T06:49:57.931 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0024977509999999999 2024-09-11T06:49:57.931 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.931 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.931 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:57.931 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824430+0000", 2024-09-11T06:49:57.931 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7812999999999999e-05 2024-09-11T06:49:57.932 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.932 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:57.932 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.932 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:137 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'95, mlcod=13'86)", 2024-09-11T06:49:57.932 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:49:57.932 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 51.034288779000001, 2024-09-11T06:49:57.932 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003084299, 2024-09-11T06:49:57.932 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:57.932 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:57.932 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:57.933 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:57.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.933 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:57.933 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:49:57.933 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.933 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:57.933 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:49:57.933 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.933 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:57.933 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823568+0000", 2024-09-11T06:49:57.934 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1030000000000001e-06 2024-09-11T06:49:57.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.934 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:57.934 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823662+0000", 2024-09-11T06:49:57.934 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3529999999999994e-05 2024-09-11T06:49:57.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.934 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:57.934 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823686+0000", 2024-09-11T06:49:57.934 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4358999999999998e-05 2024-09-11T06:49:57.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.935 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:57.935 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823701+0000", 2024-09-11T06:49:57.935 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4847e-05 2024-09-11T06:49:57.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.935 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.935 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:57.935 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824518+0000", 2024-09-11T06:49:57.935 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00081716700000000005 2024-09-11T06:49:57.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.935 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.935 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:57.935 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824546+0000", 2024-09-11T06:49:57.935 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7860000000000001e-05 2024-09-11T06:49:57.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.936 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:57.936 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826558+0000", 2024-09-11T06:49:57.936 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0020120009999999998 2024-09-11T06:49:57.936 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.936 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:57.936 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826652+0000", 2024-09-11T06:49:57.936 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3431999999999996e-05 2024-09-11T06:49:57.936 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.936 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:57.936 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.936 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.937 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:139 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'97, mlcod=13'86)", 2024-09-11T06:49:57.937 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:49:57.937 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 51.032000060000001, 2024-09-11T06:49:57.937 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003732221, 2024-09-11T06:49:57.937 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:57.937 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:57.937 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:57.937 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:57.937 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.937 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:57.937 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:49:57.937 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.938 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:57.938 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:49:57.938 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.938 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:57.938 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825858+0000", 2024-09-11T06:49:57.938 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5829999999999999e-06 2024-09-11T06:49:57.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.938 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:57.938 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825874+0000", 2024-09-11T06:49:57.938 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5967000000000001e-05 2024-09-11T06:49:57.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.939 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.939 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:57.939 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825909+0000", 2024-09-11T06:49:57.939 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5692999999999999e-05 2024-09-11T06:49:57.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.939 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.939 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:57.939 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825923+0000", 2024-09-11T06:49:57.939 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4080000000000001e-05 2024-09-11T06:49:57.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.939 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.939 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:57.939 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826010+0000", 2024-09-11T06:49:57.940 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6248999999999994e-05 2024-09-11T06:49:57.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.940 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:57.940 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826038+0000", 2024-09-11T06:49:57.940 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8010000000000001e-05 2024-09-11T06:49:57.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.940 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:57.941 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829499+0000", 2024-09-11T06:49:57.941 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003461783 2024-09-11T06:49:57.941 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.941 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.941 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:57.941 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829588+0000", 2024-09-11T06:49:57.941 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8856000000000004e-05 2024-09-11T06:49:57.941 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.941 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:57.941 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.941 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.941 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.941 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:140 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'98, mlcod=13'93)", 2024-09-11T06:49:57.941 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:49:57.942 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 51.030149330999997, 2024-09-11T06:49:57.942 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0057995929999999996, 2024-09-11T06:49:57.942 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:57.942 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:57.942 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:57.942 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:57.942 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.942 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:57.942 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:49:57.942 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.943 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.943 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:57.943 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:49:57.943 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.943 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.943 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:57.943 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827708+0000", 2024-09-11T06:49:57.943 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.547e-06 2024-09-11T06:49:57.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.943 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.943 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:57.944 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827765+0000", 2024-09-11T06:49:57.944 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6266e-05 2024-09-11T06:49:57.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.944 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:57.944 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827791+0000", 2024-09-11T06:49:57.944 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6098999999999999e-05 2024-09-11T06:49:57.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.945 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.945 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:57.945 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827806+0000", 2024-09-11T06:49:57.945 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5537e-05 2024-09-11T06:49:57.945 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.945 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.945 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:57.945 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827928+0000", 2024-09-11T06:49:57.945 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000122012 2024-09-11T06:49:57.945 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.945 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.945 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:57.945 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827961+0000", 2024-09-11T06:49:57.946 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2663999999999998e-05 2024-09-11T06:49:57.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.946 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:57.946 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833409+0000", 2024-09-11T06:49:57.946 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054479350000000001 2024-09-11T06:49:57.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.946 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:57.946 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833506+0000", 2024-09-11T06:49:57.946 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7533000000000003e-05 2024-09-11T06:49:57.946 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.947 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:57.947 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.947 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.947 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:141 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'99, mlcod=13'93)", 2024-09-11T06:49:57.947 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:49:57.947 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 51.029255947999999, 2024-09-11T06:49:57.947 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055394709999999998, 2024-09-11T06:49:57.947 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:57.947 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:57.947 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:57.947 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:57.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.948 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:57.948 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:49:57.948 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.948 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:57.948 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:49:57.948 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.948 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:57.948 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828602+0000", 2024-09-11T06:49:57.949 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.46e-06 2024-09-11T06:49:57.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.949 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:57.949 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828618+0000", 2024-09-11T06:49:57.949 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6592999999999999e-05 2024-09-11T06:49:57.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.949 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:57.949 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828646+0000", 2024-09-11T06:49:57.949 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7552999999999999e-05 2024-09-11T06:49:57.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.949 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:57.950 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828663+0000", 2024-09-11T06:49:57.950 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.719e-05 2024-09-11T06:49:57.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.950 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:57.950 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829023+0000", 2024-09-11T06:49:57.950 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036054799999999998 2024-09-11T06:49:57.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.950 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:57.950 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829062+0000", 2024-09-11T06:49:57.950 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8865999999999997e-05 2024-09-11T06:49:57.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.951 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:57.951 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834078+0000", 2024-09-11T06:49:57.951 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050157480000000004 2024-09-11T06:49:57.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.951 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:57.951 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834140+0000", 2024-09-11T06:49:57.951 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1513000000000005e-05 2024-09-11T06:49:57.951 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.951 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:57.951 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.951 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:144 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'102, mlcod=13'98)", 2024-09-11T06:49:57.952 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:49:57.952 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 51.020095335000001, 2024-09-11T06:49:57.952 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003210348, 2024-09-11T06:49:57.952 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:57.952 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:57.952 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:57.952 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:57.952 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.952 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:57.952 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:49:57.952 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.952 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.953 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:57.953 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:49:57.953 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.953 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:57.953 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837768+0000", 2024-09-11T06:49:57.953 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9330000000000004e-06 2024-09-11T06:49:57.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.953 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:57.953 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837858+0000", 2024-09-11T06:49:57.953 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0185999999999993e-05 2024-09-11T06:49:57.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.954 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.954 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:57.954 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837890+0000", 2024-09-11T06:49:57.954 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2437000000000001e-05 2024-09-11T06:49:57.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.954 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.954 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:57.954 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837905+0000", 2024-09-11T06:49:57.954 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:49:57.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.954 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.954 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:57.954 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838019+0000", 2024-09-11T06:49:57.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000114336 2024-09-11T06:49:57.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.955 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:57.955 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838052+0000", 2024-09-11T06:49:57.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3139000000000002e-05 2024-09-11T06:49:57.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.955 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:57.955 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840914+0000", 2024-09-11T06:49:57.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028612780000000001 2024-09-11T06:49:57.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.956 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:57.956 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840971+0000", 2024-09-11T06:49:57.956 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7402999999999999e-05 2024-09-11T06:49:57.956 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.956 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:57.956 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.956 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:145 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'103, mlcod=13'98)", 2024-09-11T06:49:57.956 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:49:57.956 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 51.019303450999999, 2024-09-11T06:49:57.956 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054455409999999999, 2024-09-11T06:49:57.957 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:57.957 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:57.957 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:57.957 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:57.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.957 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:57.957 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:49:57.957 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.957 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:57.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:49:57.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:57.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.958 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:57.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838554+0000", 2024-09-11T06:49:57.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3430000000000001e-06 2024-09-11T06:49:57.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.958 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:57.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838640+0000", 2024-09-11T06:49:57.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5569999999999996e-05 2024-09-11T06:49:57.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.959 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:57.959 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838660+0000", 2024-09-11T06:49:57.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0576000000000001e-05 2024-09-11T06:49:57.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.959 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.959 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:57.959 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838674+0000", 2024-09-11T06:49:57.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3529999999999999e-05 2024-09-11T06:49:57.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.959 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.959 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:57.959 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838838+0000", 2024-09-11T06:49:57.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016468799999999999 2024-09-11T06:49:57.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.960 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:57.960 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838885+0000", 2024-09-11T06:49:57.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6884e-05 2024-09-11T06:49:57.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.960 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:57.960 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843901+0000", 2024-09-11T06:49:57.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050162310000000003 2024-09-11T06:49:57.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:57.961 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:57.961 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:57.961 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843998+0000", 2024-09-11T06:49:57.961 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.6718999999999993e-05 2024-09-11T06:49:57.961 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.961 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:57.961 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.961 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:57.961 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:57.961 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:57.962 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:49:57.962 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:49:57.964 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:49:57.964 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:49:57.965 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:49:57.965 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T06:49:57.966 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:57.961+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:58.064 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:49:58.065+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:49:58.138 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:58.138 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:58.138 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:49:58.138 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:58.157 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:49:58.208 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:58.208+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:58.266 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:58.261+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:58.272 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:58.272 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:58.272 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:49:58.272 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:49:58.272 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:58.287 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:49:58.372 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:58.372+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:58.457 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:58.457 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:49:58.457 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:49:58.457 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:49:58.457 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.457 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:5 2.2 e17/15 2:5cea7035:::rbd_info:head v 17'1, mlcod=0'0)", 2024-09-11T06:49:58.457 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:49:58.457 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 46.220875407999998, 2024-09-11T06:49:58.458 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0019446719999999999, 2024-09-11T06:49:58.458 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:58.458 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:58.458 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:58.458 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:58.458 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.458 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:58.458 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:49:58.458 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:58.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.458 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.458 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:58.459 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:49:58.459 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:58.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.459 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.459 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:58.459 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236085+0000", 2024-09-11T06:49:58.459 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6000000000000003e-07 2024-09-11T06:49:58.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.459 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.459 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:58.459 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236108+0000", 2024-09-11T06:49:58.460 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3113e-05 2024-09-11T06:49:58.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.460 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.460 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:58.460 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236133+0000", 2024-09-11T06:49:58.460 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5723000000000001e-05 2024-09-11T06:49:58.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.460 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.460 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:58.460 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236154+0000", 2024-09-11T06:49:58.460 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0333999999999999e-05 2024-09-11T06:49:58.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.460 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.461 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:58.461 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236297+0000", 2024-09-11T06:49:58.461 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014278500000000001 2024-09-11T06:49:58.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.461 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:58.461 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236383+0000", 2024-09-11T06:49:58.461 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5927999999999995e-05 2024-09-11T06:49:58.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.461 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:58.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.237950+0000", 2024-09-11T06:49:58.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015673690000000001 2024-09-11T06:49:58.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:58.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.238029+0000", 2024-09-11T06:49:58.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8659999999999996e-05 2024-09-11T06:49:58.462 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:58.462 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:58.462 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:58.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.462 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:7 2.2 e18/15 2:5cea7035:::rbd_info:head v 18'2, mlcod=0'0)", 2024-09-11T06:49:58.462 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:49:58.463 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.206550200999999, 2024-09-11T06:49:58.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001907192, 2024-09-11T06:49:58.463 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:58.463 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:58.463 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:58.463 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:58.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:58.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:49:58.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:58.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:58.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:49:58.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:58.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:58.464 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250410+0000", 2024-09-11T06:49:58.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7700000000000004e-07 2024-09-11T06:49:58.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:58.464 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250445+0000", 2024-09-11T06:49:58.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4749000000000002e-05 2024-09-11T06:49:58.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.465 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:58.465 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250478+0000", 2024-09-11T06:49:58.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3269999999999998e-05 2024-09-11T06:49:58.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.465 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:58.465 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250501+0000", 2024-09-11T06:49:58.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3632999999999999e-05 2024-09-11T06:49:58.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.465 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:58.465 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250633+0000", 2024-09-11T06:49:58.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013143200000000001 2024-09-11T06:49:58.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.466 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:58.466 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250700+0000", 2024-09-11T06:49:58.466 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7002999999999995e-05 2024-09-11T06:49:58.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.466 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:58.466 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.252172+0000", 2024-09-11T06:49:58.466 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014719550000000001 2024-09-11T06:49:58.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.466 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:58.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.252316+0000", 2024-09-11T06:49:58.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144473 2024-09-11T06:49:58.467 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:58.467 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:58.467 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:58.467 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:58.467 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:58.467 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:58.468 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:58.468+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:58.472 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:49:58.569 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:49:58.568+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:49:58.641 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:58.641 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:58.641 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:49:58.641 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:58.654 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:49:58.670 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:58.665+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:58.770 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:58.769+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:58.819 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:58.820 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:49:58.820 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:49:58.820 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:49:58.820 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:58.834 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:49:58.880 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:58.880+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:58.951 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:49:58.952 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:49:58.952 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:49:58.952 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:49:58.952 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.952 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:1 2.3 e16/15 2:c4c92e5a:::rbd_trash:head v 16'1, mlcod=0'0)", 2024-09-11T06:49:58.952 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:49:58.952 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 47.715547846, 2024-09-11T06:49:58.952 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001234761, 2024-09-11T06:49:58.952 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:58.952 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:58.952 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:58.952 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:58.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.953 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:58.953 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:49:58.953 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:58.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.953 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:58.953 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:49:58.953 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:58.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.953 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:58.953 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:49:58.954 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6000000000000003e-07 2024-09-11T06:49:58.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.954 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.954 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:58.954 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236296+0000", 2024-09-11T06:49:58.954 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6393999999999999e-05 2024-09-11T06:49:58.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.954 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.954 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:58.954 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236317+0000", 2024-09-11T06:49:58.954 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1243e-05 2024-09-11T06:49:58.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.954 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.954 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:58.955 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236333+0000", 2024-09-11T06:49:58.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.558e-05 2024-09-11T06:49:58.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.955 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:58.955 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236435+0000", 2024-09-11T06:49:58.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000102803 2024-09-11T06:49:58.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.955 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:58.955 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236474+0000", 2024-09-11T06:49:58.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8383999999999997e-05 2024-09-11T06:49:58.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.956 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:49:58.956 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.237432+0000", 2024-09-11T06:49:58.956 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00095849300000000004 2024-09-11T06:49:58.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.956 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:58.956 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.237513+0000", 2024-09-11T06:49:58.956 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1204000000000004e-05 2024-09-11T06:49:58.956 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:58.956 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:58.956 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:58.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.956 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "MOSDPGPush(2.2 37/36 [PushOp(2:5cea7035:::rbd_info:head, version: 18'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@18'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:49:58.957 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:50.554243+0000", 2024-09-11T06:49:58.957 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.3975833850000008, 2024-09-11T06:49:58.957 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00051362700000000001, 2024-09-11T06:49:58.957 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:49:58.957 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:49:58.957 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:49:58.957 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:49:58.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.957 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:49:58.957 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:50.554243+0000", 2024-09-11T06:49:58.957 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:58.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.958 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:49:58.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:50.554243+0000", 2024-09-11T06:49:58.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:49:58.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.958 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:49:58.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:50.554245+0000", 2024-09-11T06:49:58.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6759999999999999e-06 2024-09-11T06:49:58.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.958 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:49:58.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:50.554270+0000", 2024-09-11T06:49:58.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5244000000000001e-05 2024-09-11T06:49:58.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.959 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.959 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:49:58.959 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:50.554316+0000", 2024-09-11T06:49:58.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6164e-05 2024-09-11T06:49:58.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.959 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.959 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:49:58.959 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:50.554361+0000", 2024-09-11T06:49:58.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5102000000000003e-05 2024-09-11T06:49:58.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.959 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.959 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:49:58.959 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:50.554463+0000", 2024-09-11T06:49:58.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000101968 2024-09-11T06:49:58.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.960 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:49:58.960 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:50.554472+0000", 2024-09-11T06:49:58.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.816e-06 2024-09-11T06:49:58.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:49:58.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:49:58.960 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:49:58.960 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:50.554757+0000", 2024-09-11T06:49:58.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00028465700000000001 2024-09-11T06:49:58.960 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:58.960 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:58.960 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:58.960 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:49:58.960 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:49:58.961 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:49:58.966 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:49:58.967 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is set 2024-09-11T06:49:59.011 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:59.009+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:59.073 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:59.072+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:59.096 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:59.096 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:59.096 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:49:59.096 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:59.110 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:49:59.174 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:49:59.172+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:49:59.275 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:49:59.273+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:49:59.278 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:59.278 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:59.278 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:49:59.278 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:49:59.278 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:59.292 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:49:59.379 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:59.377+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:59.464 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:59.464 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:49:59.464 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:49:59.464 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:49:59.464 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.464 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:59.464 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:49:59.464 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 53.291094977, 2024-09-11T06:49:59.464 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011420669, 2024-09-11T06:49:59.464 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:59.465 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:59.465 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:59.465 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:59.465 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:59.465 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:59.465 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 37 2024-09-11T06:49:59.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.465 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:59.465 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.465 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:59.465 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:49:59.466 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.466 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.466 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:59.466 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:49:59.466 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.466 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.466 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:59.466 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170509+0000", 2024-09-11T06:49:59.466 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.13e-06 2024-09-11T06:49:59.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.466 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.467 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:59.467 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170521+0000", 2024-09-11T06:49:59.467 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1776999999999999e-05 2024-09-11T06:49:59.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.467 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.467 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:59.467 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170538+0000", 2024-09-11T06:49:59.467 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7413000000000001e-05 2024-09-11T06:49:59.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.467 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.467 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:59.468 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170570+0000", 2024-09-11T06:49:59.468 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2313e-05 2024-09-11T06:49:59.468 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.468 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.468 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:59.468 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179768+0000", 2024-09-11T06:49:59.468 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0091978010000000002 2024-09-11T06:49:59.468 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.468 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.468 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:59.468 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179951+0000", 2024-09-11T06:49:59.468 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018227900000000001 2024-09-11T06:49:59.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.469 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.469 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:59.469 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.180175+0000", 2024-09-11T06:49:59.469 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022432800000000001 2024-09-11T06:49:59.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.469 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.469 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:59.469 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181112+0000", 2024-09-11T06:49:59.469 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00093712500000000002 2024-09-11T06:49:59.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.469 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.469 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:59.470 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181789+0000", 2024-09-11T06:49:59.470 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00067705100000000002 2024-09-11T06:49:59.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.470 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.470 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:59.470 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181871+0000", 2024-09-11T06:49:59.470 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1816000000000004e-05 2024-09-11T06:49:59.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.470 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.471 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:59.471 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181929+0000", 2024-09-11T06:49:59.471 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7636000000000002e-05 2024-09-11T06:49:59.471 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.471 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:59.471 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.472 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.472 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:59.472 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:49:59.472 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 53.189836016000001, 2024-09-11T06:49:59.473 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283336, 2024-09-11T06:49:59.473 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:59.473 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:59.473 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:59.473 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:59.473 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:59.473 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:59.473 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 87 2024-09-11T06:49:59.473 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.473 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:59.473 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.473 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:59.473 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:49:59.474 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.474 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.474 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:59.474 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:49:59.474 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.474 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.474 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:59.474 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271768+0000", 2024-09-11T06:49:59.474 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7599999999999996e-07 2024-09-11T06:49:59.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.474 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.475 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:59.475 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271860+0000", 2024-09-11T06:49:59.475 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2639999999999994e-05 2024-09-11T06:49:59.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.475 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:59.475 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271873+0000", 2024-09-11T06:49:59.475 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3139000000000001e-05 2024-09-11T06:49:59.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.475 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:59.476 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271882+0000", 2024-09-11T06:49:59.476 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2369999999999992e-06 2024-09-11T06:49:59.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.476 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:59.476 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277150+0000", 2024-09-11T06:49:59.476 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052684419999999999 2024-09-11T06:49:59.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.476 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:59.476 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277273+0000", 2024-09-11T06:49:59.476 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012278899999999999 2024-09-11T06:49:59.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.477 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:59.477 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277466+0000", 2024-09-11T06:49:59.477 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193578 2024-09-11T06:49:59.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.477 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:59.477 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.280808+0000", 2024-09-11T06:49:59.477 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003341971 2024-09-11T06:49:59.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.477 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:59.477 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284419+0000", 2024-09-11T06:49:59.477 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003610982 2024-09-11T06:49:59.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.478 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:59.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284531+0000", 2024-09-11T06:49:59.478 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011121299999999999 2024-09-11T06:49:59.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.478 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:59.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284600+0000", 2024-09-11T06:49:59.478 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9592999999999996e-05 2024-09-11T06:49:59.478 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.478 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:59.478 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.479 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:59.479 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:49:59.479 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 53.189705764000003, 2024-09-11T06:49:59.479 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283314, 2024-09-11T06:49:59.479 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:59.479 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:59.479 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:59.479 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:59.479 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:59.479 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:59.479 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 88 2024-09-11T06:49:59.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.479 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:59.479 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.480 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:59.480 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:49:59.480 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.480 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:59.480 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:49:59.480 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.480 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:59.480 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271898+0000", 2024-09-11T06:49:59.480 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2999999999999999e-07 2024-09-11T06:49:59.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.481 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:59.481 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271904+0000", 2024-09-11T06:49:59.481 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.767e-06 2024-09-11T06:49:59.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.481 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:59.481 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271913+0000", 2024-09-11T06:49:59.481 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7200000000000001e-06 2024-09-11T06:49:59.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.481 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:59.482 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271920+0000", 2024-09-11T06:49:59.482 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7100000000000001e-06 2024-09-11T06:49:59.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.482 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:59.482 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278684+0000", 2024-09-11T06:49:59.482 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0067642650000000002 2024-09-11T06:49:59.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.482 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:59.482 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278838+0000", 2024-09-11T06:49:59.482 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015371499999999999 2024-09-11T06:49:59.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.483 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:59.483 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.279040+0000", 2024-09-11T06:49:59.483 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020207499999999999 2024-09-11T06:49:59.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.483 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.483 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:59.483 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.281448+0000", 2024-09-11T06:49:59.483 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002407686 2024-09-11T06:49:59.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.483 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.483 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:59.483 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284615+0000", 2024-09-11T06:49:59.483 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031675499999999999 2024-09-11T06:49:59.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.484 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.484 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:59.484 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284682+0000", 2024-09-11T06:49:59.484 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7032000000000005e-05 2024-09-11T06:49:59.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.484 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.484 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:59.484 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284730+0000", 2024-09-11T06:49:59.484 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7790000000000002e-05 2024-09-11T06:49:59.484 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.484 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:59.484 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.485 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.485 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.485 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:59.485 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:49:59.485 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 53.021804541999998, 2024-09-11T06:49:59.485 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011114877, 2024-09-11T06:49:59.485 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:59.485 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:59.485 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:59.485 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:59.485 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:59.485 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:59.485 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 107 2024-09-11T06:49:59.485 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.486 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:59.486 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.486 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:59.486 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:49:59.486 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.486 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.486 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.486 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:59.486 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:49:59.486 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.486 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.486 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.486 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:59.487 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439799+0000", 2024-09-11T06:49:59.487 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.16e-07 2024-09-11T06:49:59.487 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.487 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.487 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:59.487 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439806+0000", 2024-09-11T06:49:59.487 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3640000000000004e-06 2024-09-11T06:49:59.487 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.487 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.487 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:59.487 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439816+0000", 2024-09-11T06:49:59.487 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0485999999999999e-05 2024-09-11T06:49:59.487 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.487 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.488 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:59.488 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439824+0000", 2024-09-11T06:49:59.488 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.6699999999999994e-06 2024-09-11T06:49:59.488 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.488 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.488 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:59.488 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443328+0000", 2024-09-11T06:49:59.488 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035044329999999999 2024-09-11T06:49:59.488 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.488 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.488 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:59.488 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443461+0000", 2024-09-11T06:49:59.488 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013272199999999999 2024-09-11T06:49:59.489 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.489 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.489 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:59.489 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443678+0000", 2024-09-11T06:49:59.489 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021674500000000001 2024-09-11T06:49:59.489 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.489 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.489 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:59.489 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449635+0000", 2024-09-11T06:49:59.489 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0059572119999999999 2024-09-11T06:49:59.489 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.489 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.489 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:59.489 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450750+0000", 2024-09-11T06:49:59.490 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001115654 2024-09-11T06:49:59.490 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.490 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.490 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:59.490 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450832+0000", 2024-09-11T06:49:59.490 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1188999999999998e-05 2024-09-11T06:49:59.490 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.490 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.490 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:59.490 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450913+0000", 2024-09-11T06:49:59.490 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1485999999999999e-05 2024-09-11T06:49:59.491 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.491 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:59.491 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.491 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.491 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.491 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:59.491 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:49:59.491 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 53.021213566999997, 2024-09-11T06:49:59.491 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011236556, 2024-09-11T06:49:59.491 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:59.491 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:59.491 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:59.491 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:59.492 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:59.492 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:59.492 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 109 2024-09-11T06:49:59.492 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.492 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:59.492 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.492 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:59.492 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:49:59.492 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.492 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.492 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.492 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:59.492 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:49:59.492 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.493 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.493 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:59.493 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440391+0000", 2024-09-11T06:49:59.493 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8360000000000001e-06 2024-09-11T06:49:59.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.493 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.493 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:59.493 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440560+0000", 2024-09-11T06:49:59.493 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000169085 2024-09-11T06:49:59.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.493 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.493 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:59.493 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440585+0000", 2024-09-11T06:49:59.494 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4530000000000001e-05 2024-09-11T06:49:59.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.494 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.494 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:59.494 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440606+0000", 2024-09-11T06:49:59.494 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1277e-05 2024-09-11T06:49:59.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.494 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.494 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:59.494 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446479+0000", 2024-09-11T06:49:59.494 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0058734989999999999 2024-09-11T06:49:59.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.494 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.494 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:59.495 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446649+0000", 2024-09-11T06:49:59.495 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016948199999999999 2024-09-11T06:49:59.495 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.495 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.495 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:59.495 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446854+0000", 2024-09-11T06:49:59.495 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020491500000000001 2024-09-11T06:49:59.495 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.495 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.495 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:59.495 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451380+0000", 2024-09-11T06:49:59.495 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0045264110000000001 2024-09-11T06:49:59.495 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.495 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.496 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:59.496 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451479+0000", 2024-09-11T06:49:59.496 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.8975999999999996e-05 2024-09-11T06:49:59.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.496 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.496 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:59.496 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451562+0000", 2024-09-11T06:49:59.496 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3213000000000004e-05 2024-09-11T06:49:59.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.496 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.496 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:59.496 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451626+0000", 2024-09-11T06:49:59.496 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3331999999999996e-05 2024-09-11T06:49:59.497 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.497 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:59.497 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.497 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.497 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:59.497 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:49:59.497 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 53.020964438999997, 2024-09-11T06:49:59.497 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011252866, 2024-09-11T06:49:59.497 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:59.497 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:59.497 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:59.497 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:59.497 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:59.498 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:59.498 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 110 2024-09-11T06:49:59.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.498 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:59.498 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.498 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:59.498 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:49:59.498 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.498 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.498 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:59.498 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:49:59.498 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.499 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.499 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:59.499 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440640+0000", 2024-09-11T06:49:59.499 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5430000000000001e-06 2024-09-11T06:49:59.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.499 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.499 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:59.499 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440651+0000", 2024-09-11T06:49:59.499 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0672999999999999e-05 2024-09-11T06:49:59.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.499 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.499 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:59.499 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440670+0000", 2024-09-11T06:49:59.499 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9726999999999999e-05 2024-09-11T06:49:59.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.500 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.500 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:59.500 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440691+0000", 2024-09-11T06:49:59.500 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0526e-05 2024-09-11T06:49:59.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.500 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.500 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:59.500 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448191+0000", 2024-09-11T06:49:59.500 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0075005990000000002 2024-09-11T06:49:59.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.500 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.500 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:59.501 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448390+0000", 2024-09-11T06:49:59.501 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000198815 2024-09-11T06:49:59.501 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.501 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.501 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:59.501 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448735+0000", 2024-09-11T06:49:59.501 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034424999999999999 2024-09-11T06:49:59.501 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.501 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.501 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:59.501 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451742+0000", 2024-09-11T06:49:59.501 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030077440000000001 2024-09-11T06:49:59.501 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.501 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.502 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:59.502 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451761+0000", 2024-09-11T06:49:59.502 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9196999999999999e-05 2024-09-11T06:49:59.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.502 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.502 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:59.502 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451834+0000", 2024-09-11T06:49:59.502 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2071999999999997e-05 2024-09-11T06:49:59.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.502 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.502 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:59.502 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451891+0000", 2024-09-11T06:49:59.502 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7720000000000003e-05 2024-09-11T06:49:59.502 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.503 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:59.503 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.503 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:59.503 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:49:59.503 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 53.020856563999999, 2024-09-11T06:49:59.503 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014944941, 2024-09-11T06:49:59.503 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:59.503 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:59.503 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:59.503 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:59.503 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:59.503 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:59.504 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 111 2024-09-11T06:49:59.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.504 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:59.504 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.504 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:59.504 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:49:59.504 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.504 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.504 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:59.504 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:49:59.504 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.505 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:59.505 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440748+0000", 2024-09-11T06:49:59.505 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:49:59.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.505 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:59.505 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440760+0000", 2024-09-11T06:49:59.505 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1897e-05 2024-09-11T06:49:59.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.505 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:59.505 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440782+0000", 2024-09-11T06:49:59.505 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2552999999999999e-05 2024-09-11T06:49:59.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.506 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.506 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:59.506 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440793+0000", 2024-09-11T06:49:59.506 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0643e-05 2024-09-11T06:49:59.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.506 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.506 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:59.506 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449755+0000", 2024-09-11T06:49:59.506 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089622620000000004 2024-09-11T06:49:59.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.506 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.506 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:59.506 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449918+0000", 2024-09-11T06:49:59.506 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163086 2024-09-11T06:49:59.507 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.507 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.507 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:59.507 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450150+0000", 2024-09-11T06:49:59.507 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023204500000000001 2024-09-11T06:49:59.507 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.507 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.507 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:59.507 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454533+0000", 2024-09-11T06:49:59.507 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0043831349999999998 2024-09-11T06:49:59.507 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.507 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.507 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:59.507 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455511+0000", 2024-09-11T06:49:59.508 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0009776050000000001 2024-09-11T06:49:59.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.508 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.508 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:59.508 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455597+0000", 2024-09-11T06:49:59.508 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6446000000000005e-05 2024-09-11T06:49:59.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.508 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.508 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:59.508 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455691+0000", 2024-09-11T06:49:59.508 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3832000000000006e-05 2024-09-11T06:49:59.508 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.508 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:59.509 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.509 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.509 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.509 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:59.509 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:49:59.509 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 53.020786801, 2024-09-11T06:49:59.509 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018885411000000001, 2024-09-11T06:49:59.509 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:59.509 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:59.509 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:59.509 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:59.509 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:59.509 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:59.510 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 112 2024-09-11T06:49:59.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.510 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:59.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.510 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:59.510 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:49:59.510 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.510 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:59.510 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:49:59.510 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.511 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:59.511 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440817+0000", 2024-09-11T06:49:59.511 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:49:59.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.511 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.511 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:59.511 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441003+0000", 2024-09-11T06:49:59.511 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018661200000000001 2024-09-11T06:49:59.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.511 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.511 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:59.511 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441027+0000", 2024-09-11T06:49:59.511 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3920000000000001e-05 2024-09-11T06:49:59.512 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.512 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.512 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:59.512 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441062+0000", 2024-09-11T06:49:59.512 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4663000000000003e-05 2024-09-11T06:49:59.512 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.512 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.512 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:59.512 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451981+0000", 2024-09-11T06:49:59.512 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010918926000000001 2024-09-11T06:49:59.512 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.512 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.512 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:59.513 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452111+0000", 2024-09-11T06:49:59.513 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129529 2024-09-11T06:49:59.513 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.513 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.513 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:59.513 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452311+0000", 2024-09-11T06:49:59.513 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020013799999999999 2024-09-11T06:49:59.513 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.513 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.513 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:59.513 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455809+0000", 2024-09-11T06:49:59.513 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003498443 2024-09-11T06:49:59.513 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.513 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.514 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:59.514 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459566+0000", 2024-09-11T06:49:59.514 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003757271 2024-09-11T06:49:59.514 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.514 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.514 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:59.514 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459640+0000", 2024-09-11T06:49:59.514 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3949000000000006e-05 2024-09-11T06:49:59.514 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.514 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.514 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:59.514 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459701+0000", 2024-09-11T06:49:59.514 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1117000000000005e-05 2024-09-11T06:49:59.514 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.515 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:59.515 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.515 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.515 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:59.515 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:49:59.515 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 53.020507950000002, 2024-09-11T06:49:59.515 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018760915, 2024-09-11T06:49:59.515 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:59.515 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:59.515 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:59.515 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:59.515 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:59.515 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:59.516 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 113 2024-09-11T06:49:59.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.516 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:59.516 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.516 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:59.516 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:49:59.516 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.516 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.516 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:59.516 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:49:59.516 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.517 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.517 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:59.517 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441096+0000", 2024-09-11T06:49:59.517 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4130000000000001e-06 2024-09-11T06:49:59.517 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.517 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.517 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:59.517 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441110+0000", 2024-09-11T06:49:59.517 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3900000000000001e-05 2024-09-11T06:49:59.517 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.517 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.517 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:59.517 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441131+0000", 2024-09-11T06:49:59.518 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0636999999999999e-05 2024-09-11T06:49:59.518 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.518 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.518 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:59.518 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441147+0000", 2024-09-11T06:49:59.518 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6512999999999999e-05 2024-09-11T06:49:59.518 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.518 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.518 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:59.518 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453274+0000", 2024-09-11T06:49:59.518 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012126764999999999 2024-09-11T06:49:59.518 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.518 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.518 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:59.519 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453498+0000", 2024-09-11T06:49:59.519 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000223568 2024-09-11T06:49:59.519 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.519 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.519 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:59.519 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453872+0000", 2024-09-11T06:49:59.519 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037467099999999998 2024-09-11T06:49:59.519 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.519 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.519 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:59.519 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458709+0000", 2024-09-11T06:49:59.519 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0048365680000000003 2024-09-11T06:49:59.519 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.519 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.519 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:59.520 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459716+0000", 2024-09-11T06:49:59.520 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010075240000000001 2024-09-11T06:49:59.520 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.520 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.520 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:59.520 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459779+0000", 2024-09-11T06:49:59.520 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2336999999999996e-05 2024-09-11T06:49:59.520 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.520 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.520 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:59.520 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459856+0000", 2024-09-11T06:49:59.520 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7019000000000002e-05 2024-09-11T06:49:59.520 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.520 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:59.521 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.521 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.521 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.521 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:59.521 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:59.521 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 53.020431211000002, 2024-09-11T06:49:59.521 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018963114, 2024-09-11T06:49:59.521 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:59.521 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:59.521 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:59.521 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:59.521 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:59.521 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:59.522 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 114 2024-09-11T06:49:59.522 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.522 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:59.522 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.522 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:59.522 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:59.522 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.522 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.522 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.522 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:59.523 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:59.523 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.523 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.523 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.523 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:59.523 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:49:59.523 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9299999999999996e-07 2024-09-11T06:49:59.523 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.523 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.523 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:59.524 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441180+0000", 2024-09-11T06:49:59.524 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7200000000000006e-06 2024-09-11T06:49:59.524 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.524 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.524 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:59.524 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441265+0000", 2024-09-11T06:49:59.524 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4753000000000007e-05 2024-09-11T06:49:59.524 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.524 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.524 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:59.524 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441288+0000", 2024-09-11T06:49:59.524 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2860000000000001e-05 2024-09-11T06:49:59.524 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.525 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.525 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:59.525 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454585+0000", 2024-09-11T06:49:59.525 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013297286 2024-09-11T06:49:59.525 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.525 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.525 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:59.525 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454714+0000", 2024-09-11T06:49:59.525 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000128465 2024-09-11T06:49:59.525 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.525 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.525 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:59.525 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454909+0000", 2024-09-11T06:49:59.525 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019522499999999999 2024-09-11T06:49:59.526 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.526 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.526 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:59.526 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459868+0000", 2024-09-11T06:49:59.526 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0049591239999999997 2024-09-11T06:49:59.526 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.526 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.526 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:59.526 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459988+0000", 2024-09-11T06:49:59.526 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000120532 2024-09-11T06:49:59.526 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.526 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.526 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:59.526 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460071+0000", 2024-09-11T06:49:59.527 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2256000000000007e-05 2024-09-11T06:49:59.527 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.527 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.527 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:59.527 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460135+0000", 2024-09-11T06:49:59.527 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3999999999999997e-05 2024-09-11T06:49:59.527 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.527 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:59.527 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.527 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.527 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.528 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:59.528 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:49:59.528 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 53.020273582000002, 2024-09-11T06:49:59.528 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.019036849000000002, 2024-09-11T06:49:59.528 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:59.528 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:59.528 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:59.528 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:59.528 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:59.528 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:59.528 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 115 2024-09-11T06:49:59.528 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.528 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:59.529 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.529 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:59.529 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:49:59.529 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.529 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.529 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.529 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:59.529 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:49:59.529 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.529 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.529 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.529 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:59.530 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441330+0000", 2024-09-11T06:49:59.530 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1930000000000001e-06 2024-09-11T06:49:59.530 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.530 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.530 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:59.530 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441544+0000", 2024-09-11T06:49:59.530 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021405799999999999 2024-09-11T06:49:59.530 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.530 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.530 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:59.530 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441572+0000", 2024-09-11T06:49:59.530 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7317e-05 2024-09-11T06:49:59.530 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.530 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.531 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:59.531 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441593+0000", 2024-09-11T06:49:59.531 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1702999999999999e-05 2024-09-11T06:49:59.531 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.531 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.531 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:59.531 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455891+0000", 2024-09-11T06:49:59.531 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01429708 2024-09-11T06:49:59.531 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.531 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.531 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:59.531 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456046+0000", 2024-09-11T06:49:59.531 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000155682 2024-09-11T06:49:59.532 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.532 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.532 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:59.532 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456242+0000", 2024-09-11T06:49:59.532 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000195692 2024-09-11T06:49:59.532 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.532 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.532 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:59.532 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459876+0000", 2024-09-11T06:49:59.532 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036336480000000002 2024-09-11T06:49:59.532 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.532 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.532 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:59.533 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460245+0000", 2024-09-11T06:49:59.533 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036901700000000002 2024-09-11T06:49:59.533 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.533 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.533 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:59.533 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460316+0000", 2024-09-11T06:49:59.533 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1846000000000003e-05 2024-09-11T06:49:59.533 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.533 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.533 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:59.533 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460366+0000", 2024-09-11T06:49:59.533 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9613000000000001e-05 2024-09-11T06:49:59.533 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.533 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:49:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 53.019951501999998, 2024-09-11T06:49:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01904432, 2024-09-11T06:49:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 116 2024-09-11T06:49:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:49:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:49:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441653+0000", 2024-09-11T06:49:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:49:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.536 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.536 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:59.536 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441665+0000", 2024-09-11T06:49:59.536 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1653e-05 2024-09-11T06:49:59.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.536 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.536 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:59.536 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441684+0000", 2024-09-11T06:49:59.536 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9490000000000001e-05 2024-09-11T06:49:59.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.536 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.536 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:59.536 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441699+0000", 2024-09-11T06:49:59.536 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5044e-05 2024-09-11T06:49:59.537 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.537 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.537 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:59.537 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457297+0000", 2024-09-11T06:49:59.537 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015598241000000001 2024-09-11T06:49:59.537 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.537 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.537 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:59.537 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457577+0000", 2024-09-11T06:49:59.537 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027930799999999998 2024-09-11T06:49:59.537 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.537 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.537 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:59.538 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457957+0000", 2024-09-11T06:49:59.538 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00038088 2024-09-11T06:49:59.538 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.538 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.538 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:59.538 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460432+0000", 2024-09-11T06:49:59.538 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024743349999999998 2024-09-11T06:49:59.538 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.538 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.538 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:59.538 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460584+0000", 2024-09-11T06:49:59.538 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015232900000000001 2024-09-11T06:49:59.538 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.539 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.539 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:59.539 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460646+0000", 2024-09-11T06:49:59.539 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1883000000000004e-05 2024-09-11T06:49:59.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.539 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.539 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:59.539 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460696+0000", 2024-09-11T06:49:59.539 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9589999999999998e-05 2024-09-11T06:49:59.539 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.539 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:59.539 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.539 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.540 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:59.540 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:49:59.540 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 53.019879435999997, 2024-09-11T06:49:59.540 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.021834473, 2024-09-11T06:49:59.540 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:59.540 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:59.540 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:59.540 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:59.540 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:59.540 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:59.540 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 117 2024-09-11T06:49:59.540 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.540 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:59.541 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.541 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:59.541 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:49:59.541 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.541 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.541 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.541 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:59.541 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:49:59.541 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.541 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.541 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.541 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:59.541 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441725+0000", 2024-09-11T06:49:59.541 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5069999999999999e-06 2024-09-11T06:49:59.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.542 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.542 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:59.542 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441738+0000", 2024-09-11T06:49:59.542 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2813000000000001e-05 2024-09-11T06:49:59.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.542 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.542 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:59.542 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441759+0000", 2024-09-11T06:49:59.542 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1246999999999998e-05 2024-09-11T06:49:59.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.542 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.542 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:59.542 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441779+0000", 2024-09-11T06:49:59.543 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0392999999999999e-05 2024-09-11T06:49:59.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.543 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.543 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:59.543 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458804+0000", 2024-09-11T06:49:59.543 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.017024182999999998 2024-09-11T06:49:59.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.543 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.543 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:59.543 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458977+0000", 2024-09-11T06:49:59.543 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000173926 2024-09-11T06:49:59.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.543 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.544 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:59.544 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459170+0000", 2024-09-11T06:49:59.544 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000192184 2024-09-11T06:49:59.544 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.544 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.544 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:59.544 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.462300+0000", 2024-09-11T06:49:59.544 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031304969999999999 2024-09-11T06:49:59.544 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.544 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.544 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:59.544 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463408+0000", 2024-09-11T06:49:59.544 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011077839999999999 2024-09-11T06:49:59.544 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.545 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.545 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:59.545 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463497+0000", 2024-09-11T06:49:59.545 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8828999999999999e-05 2024-09-11T06:49:59.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.545 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.545 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:59.545 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463558+0000", 2024-09-11T06:49:59.545 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1110000000000003e-05 2024-09-11T06:49:59.545 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.545 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:59.545 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.545 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:49:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 52.643371748, 2024-09-11T06:49:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012216898, 2024-09-11T06:49:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 137 2024-09-11T06:49:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:59.547 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:49:59.547 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.547 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.547 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.547 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:59.547 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:49:59.547 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.547 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.547 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.547 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:59.547 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818232+0000", 2024-09-11T06:49:59.547 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2839999999999999e-06 2024-09-11T06:49:59.547 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.547 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.547 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:59.548 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818326+0000", 2024-09-11T06:49:59.548 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3531999999999998e-05 2024-09-11T06:49:59.548 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.548 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.548 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:59.548 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818337+0000", 2024-09-11T06:49:59.548 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0655999999999999e-05 2024-09-11T06:49:59.548 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.548 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.548 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:59.548 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818342+0000", 2024-09-11T06:49:59.548 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4600000000000002e-06 2024-09-11T06:49:59.548 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.548 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.548 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:59.549 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823890+0000", 2024-09-11T06:49:59.549 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0055480700000000004 2024-09-11T06:49:59.549 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.549 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.549 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:59.549 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823953+0000", 2024-09-11T06:49:59.549 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2848999999999994e-05 2024-09-11T06:49:59.549 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.549 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.549 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:59.549 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824074+0000", 2024-09-11T06:49:59.549 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121309 2024-09-11T06:49:59.549 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.549 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828043+0000", 2024-09-11T06:49:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0039685240000000002 2024-09-11T06:49:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830293+0000", 2024-09-11T06:49:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0022503179999999998 2024-09-11T06:49:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830366+0000", 2024-09-11T06:49:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2539999999999993e-05 2024-09-11T06:49:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.551 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:59.551 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830448+0000", 2024-09-11T06:49:59.551 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2355999999999996e-05 2024-09-11T06:49:59.551 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.551 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:59.551 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.551 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.551 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:59.551 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:59.551 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 52.643237933999998, 2024-09-11T06:49:59.551 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013136297, 2024-09-11T06:49:59.551 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:59.551 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:59.551 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 138 2024-09-11T06:49:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.553 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:59.553 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:49:59.553 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.89e-07 2024-09-11T06:49:59.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.553 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:59.553 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818369+0000", 2024-09-11T06:49:59.553 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4170000000000001e-06 2024-09-11T06:49:59.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.553 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:59.553 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818376+0000", 2024-09-11T06:49:59.554 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4270000000000002e-06 2024-09-11T06:49:59.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.554 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.554 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:59.554 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818383+0000", 2024-09-11T06:49:59.554 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3060000000000003e-06 2024-09-11T06:49:59.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.554 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.554 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:59.554 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824783+0000", 2024-09-11T06:49:59.554 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064003020000000001 2024-09-11T06:49:59.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.554 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825013+0000", 2024-09-11T06:49:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023039499999999999 2024-09-11T06:49:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825484+0000", 2024-09-11T06:49:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000470562 2024-09-11T06:49:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828633+0000", 2024-09-11T06:49:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031496689999999999 2024-09-11T06:49:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.556 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.556 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:59.556 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831413+0000", 2024-09-11T06:49:59.556 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027793219999999999 2024-09-11T06:49:59.556 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.556 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.556 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:59.556 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831450+0000", 2024-09-11T06:49:59.556 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7370999999999998e-05 2024-09-11T06:49:59.556 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.556 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.556 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:59.556 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831501+0000", 2024-09-11T06:49:59.556 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1036999999999999e-05 2024-09-11T06:49:59.557 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.557 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:59.557 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.557 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.557 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.557 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:59.557 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:49:59.557 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 52.643213453999998, 2024-09-11T06:49:59.557 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013223011, 2024-09-11T06:49:59.557 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:59.557 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:59.557 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:59.557 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:59.558 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:59.558 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:59.558 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 139 2024-09-11T06:49:59.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.558 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:59.558 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.558 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:59.558 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:49:59.558 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.558 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.558 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:59.558 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:49:59.558 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.559 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.559 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.559 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:59.559 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818390+0000", 2024-09-11T06:49:59.559 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5600000000000001e-07 2024-09-11T06:49:59.559 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.559 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.559 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:59.559 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818394+0000", 2024-09-11T06:49:59.559 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5929999999999999e-06 2024-09-11T06:49:59.559 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.559 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.559 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:59.559 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818399+0000", 2024-09-11T06:49:59.560 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7740000000000004e-06 2024-09-11T06:49:59.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.560 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:59.560 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818412+0000", 2024-09-11T06:49:59.560 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2625e-05 2024-09-11T06:49:59.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.560 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:59.560 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826205+0000", 2024-09-11T06:49:59.560 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0077930730000000002 2024-09-11T06:49:59.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826281+0000", 2024-09-11T06:49:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5876000000000003e-05 2024-09-11T06:49:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826384+0000", 2024-09-11T06:49:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103838 2024-09-11T06:49:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830642+0000", 2024-09-11T06:49:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004257083 2024-09-11T06:49:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831509+0000", 2024-09-11T06:49:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086730900000000003 2024-09-11T06:49:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831547+0000", 2024-09-11T06:49:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7793000000000002e-05 2024-09-11T06:49:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831612+0000", 2024-09-11T06:49:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5691000000000005e-05 2024-09-11T06:49:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.563 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:59.563 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.563 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.563 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:59.563 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:59.563 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 52.643184759999997, 2024-09-11T06:49:59.563 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016098024999999998, 2024-09-11T06:49:59.563 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:59.563 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:59.563 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:59.563 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:59.563 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:59.563 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:59.564 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 140 2024-09-11T06:49:59.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.564 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:59.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.564 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:59.564 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:59.564 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.564 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:59.564 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:59.564 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.565 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.565 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:59.565 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:49:59.565 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5199999999999998e-07 2024-09-11T06:49:59.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.565 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.565 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:59.565 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818479+0000", 2024-09-11T06:49:59.565 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0368000000000002e-05 2024-09-11T06:49:59.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.565 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.565 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:59.565 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818487+0000", 2024-09-11T06:49:59.566 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2339999999999994e-06 2024-09-11T06:49:59.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.566 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.566 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:59.566 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818491+0000", 2024-09-11T06:49:59.566 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4329999999999997e-06 2024-09-11T06:49:59.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.566 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.566 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:59.566 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828080+0000", 2024-09-11T06:49:59.566 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0095890449999999992 2024-09-11T06:49:59.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.566 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.567 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:59.567 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828130+0000", 2024-09-11T06:49:59.567 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9239000000000001e-05 2024-09-11T06:49:59.567 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.567 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.567 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:59.567 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828208+0000", 2024-09-11T06:49:59.567 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.852e-05 2024-09-11T06:49:59.567 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.567 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.567 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:59.567 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831625+0000", 2024-09-11T06:49:59.567 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034169410000000002 2024-09-11T06:49:59.567 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.568 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.568 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:59.568 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834411+0000", 2024-09-11T06:49:59.568 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002786111 2024-09-11T06:49:59.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.568 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.568 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:59.568 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834457+0000", 2024-09-11T06:49:59.568 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6041000000000002e-05 2024-09-11T06:49:59.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.568 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.568 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:59.568 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834516+0000", 2024-09-11T06:49:59.569 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8740999999999999e-05 2024-09-11T06:49:59.569 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.569 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:59.569 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.569 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:49:59.569 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:49:59.569 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 52.643103693, 2024-09-11T06:49:59.569 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016588772000000002, 2024-09-11T06:49:59.569 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:59.569 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:59.569 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:59.569 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:59.570 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:59.570 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:59.570 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 141 2024-09-11T06:49:59.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.570 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:59.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.570 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:59.570 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:49:59.570 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.570 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:59.570 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:49:59.570 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.571 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:59.571 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818500+0000", 2024-09-11T06:49:59.571 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0800000000000005e-07 2024-09-11T06:49:59.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.571 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:59.571 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818505+0000", 2024-09-11T06:49:59.571 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2429999999999996e-06 2024-09-11T06:49:59.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.571 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:59.571 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818510+0000", 2024-09-11T06:49:59.572 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9339999999999997e-06 2024-09-11T06:49:59.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.572 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:59.572 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818514+0000", 2024-09-11T06:49:59.572 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7270000000000001e-06 2024-09-11T06:49:59.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.572 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:59.572 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828711+0000", 2024-09-11T06:49:59.572 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010197031 2024-09-11T06:49:59.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.572 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:59.572 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828853+0000", 2024-09-11T06:49:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014201200000000001 2024-09-11T06:49:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.829115+0000", 2024-09-11T06:49:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026257500000000003 2024-09-11T06:49:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831629+0000", 2024-09-11T06:49:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025138169999999998 2024-09-11T06:49:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:59.574 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834991+0000", 2024-09-11T06:49:59.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033624459999999998 2024-09-11T06:49:59.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.574 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:59.574 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835032+0000", 2024-09-11T06:49:59.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1001000000000003e-05 2024-09-11T06:49:59.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.574 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:59.574 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835088+0000", 2024-09-11T06:49:59.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5478e-05 2024-09-11T06:49:59.574 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:49:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:49:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 52.019468789999998, 2024-09-11T06:49:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011161912, 2024-09-11T06:49:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 158 2024-09-11T06:49:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:49:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:49:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442135+0000", 2024-09-11T06:49:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3e-07 2024-09-11T06:49:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442259+0000", 2024-09-11T06:49:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000123903 2024-09-11T06:49:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442289+0000", 2024-09-11T06:49:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0833000000000001e-05 2024-09-11T06:49:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442301+0000", 2024-09-11T06:49:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1433e-05 2024-09-11T06:49:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449796+0000", 2024-09-11T06:49:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0074954959999999999 2024-09-11T06:49:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449917+0000", 2024-09-11T06:49:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121033 2024-09-11T06:49:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:59.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.450166+0000", 2024-09-11T06:49:59.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00024832400000000002 2024-09-11T06:49:59.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.579 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:59.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.452630+0000", 2024-09-11T06:49:59.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002464188 2024-09-11T06:49:59.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.579 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:59.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453149+0000", 2024-09-11T06:49:59.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00051860999999999997 2024-09-11T06:49:59.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453234+0000", 2024-09-11T06:49:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.5865000000000005e-05 2024-09-11T06:49:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453296+0000", 2024-09-11T06:49:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1496999999999998e-05 2024-09-11T06:49:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:49:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:49:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 52.019282687999997, 2024-09-11T06:49:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011561900999999999, 2024-09-11T06:49:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:49:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:49:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:49:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:49:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:49:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:49:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 159 2024-09-11T06:49:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:49:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:49:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:49:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:49:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:49:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:49:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:49:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442321+0000", 2024-09-11T06:49:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8400000000000003e-07 2024-09-11T06:49:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:49:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442328+0000", 2024-09-11T06:49:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5130000000000002e-06 2024-09-11T06:49:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:49:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442341+0000", 2024-09-11T06:49:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.363e-05 2024-09-11T06:49:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:49:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442349+0000", 2024-09-11T06:49:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4760000000000001e-06 2024-09-11T06:49:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:49:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451257+0000", 2024-09-11T06:49:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089088569999999992 2024-09-11T06:49:59.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.584 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:49:59.584 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451389+0000", 2024-09-11T06:49:59.584 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013110899999999999 2024-09-11T06:49:59.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.584 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:49:59.584 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451608+0000", 2024-09-11T06:49:59.584 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000219588 2024-09-11T06:49:59.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.584 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:49:59.584 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453517+0000", 2024-09-11T06:49:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0019088779999999999 2024-09-11T06:49:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:49:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453727+0000", 2024-09-11T06:49:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020955100000000001 2024-09-11T06:49:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:49:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453815+0000", 2024-09-11T06:49:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8858999999999997e-05 2024-09-11T06:49:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:49:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:49:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:49:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453882+0000", 2024-09-11T06:49:59.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.6556000000000004e-05 2024-09-11T06:49:59.586 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.586 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:59.586 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.586 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:49:59.586 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:49:59.586 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:59.587 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:49:59.588 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:59.584+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:59.688 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:59.688+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:59.760 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:59.760 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:59.760 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:49:59.760 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:59.774 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:49:59.823 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:49:59.821+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:49:59.890 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:49:59.888+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:49:59.898 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:49:59.898 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:49:59.898 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:49:59.898 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:49:59.898 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:49:59.912 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:49:59.959 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T06:49:59.959 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T06:49:59.960 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:49:59.966 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:50:00.008 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:00.008+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:00.072 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:00.073 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:00.073 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:00.073 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [] 2024-09-11T06:50:00.073 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:00.087 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:50:00.131 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:00.129+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:00.146 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:50:00.146 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:50:00.163 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:50:00.163 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:50:00.179 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:50:00.180 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:50:00.193 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:00.189+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:00.200 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:50:00.200 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:50:00.210 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:00.210 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:00.210 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:50:00.210 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:00.220 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:50:00.220 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:50:00.224 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:50:00.238 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:50:00.239 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:50:00.265 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:50:00.265 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:50:00.284 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-09-11T06:49:59.023716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":590406,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":139068,"kb_used_data":3896,"kb_used_omap":7,"kb_used_meta":135032,"kb_avail":471720132,"statfs":{"total":483183820800,"available":483041415168,"internally_reserved":0,"allocated":3989504,"data_stored":2555444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7971,"internal_metadata":138272989},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000889"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":63,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-11T06:49:52.934919+0000","last_change":"2024-09-11T06:49:13.393788+0000","last_active":"2024-09-11T06:49:52.934919+0000","last_peered":"2024-09-11T06:49:52.934919+0000","last_clean":"2024-09-11T06:49:52.934919+0000","last_became_active":"2024-09-11T06:49:11.433437+0000","last_became_peered":"2024-09-11T06:49:11.433437+0000","last_unstale":"2024-09-11T06:49:52.934919+0000","last_undegraded":"2024-09-11T06:49:52.934919+0000","last_fullsized":"2024-09-11T06:49:52.934919+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:20:04.339208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001403348,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-11T06:49:52.934850+0000","last_change":"2024-09-11T06:49:21.882377+0000","last_active":"2024-09-11T06:49:52.934850+0000","last_peered":"2024-09-11T06:49:52.934850+0000","last_clean":"2024-09-11T06:49:52.934850+0000","last_became_active":"2024-09-11T06:49:21.881858+0000","last_became_peered":"2024-09-11T06:49:21.881858+0000","last_unstale":"2024-09-11T06:49:52.934850+0000","last_undegraded":"2024-09-11T06:49:52.934850+0000","last_fullsized":"2024-09-11T06:49:52.934850+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:33:42.789131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":68,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-11T06:49:52.934919+0000","last_change":"2024-09-11T06:49:50.556978+0000","last_active":"2024-09-11T06:49:52.934919+0000","last_peered":"2024-09-11T06:49:52.934919+0000","last_clean":"2024-09-11T06:49:52.934919+0000","last_became_active":"2024-09-11T06:49:50.556016+0000","last_became_peered":"2024-09-11T06:49:50.556016+0000","last_unstale":"2024-09-11T06:49:52.934919+0000","last_undegraded":"2024-09-11T06:49:52.934919+0000","last_fullsized":"2024-09-11T06:49:52.934919+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:29:16.607923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095172900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-11T06:49:52.931025+0000","last_change":"2024-09-11T06:49:28.017449+0000","last_active":"2024-09-11T06:49:52.931025+0000","last_peered":"2024-09-11T06:49:52.931025+0000","last_clean":"2024-09-11T06:49:52.931025+0000","last_became_active":"2024-09-11T06:49:28.016860+0000","last_became_peered":"2024-09-11T06:49:28.016860+0000","last_unstale":"2024-09-11T06:49:52.931025+0000","last_undegraded":"2024-09-11T06:49:52.931025+0000","last_fullsized":"2024-09-11T06:49:52.931025+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:32:34.900467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":81,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-11T06:49:53.504605+0000","last_change":"2024-09-11T06:49:50.558793+0000","last_active":"2024-09-11T06:49:53.504605+0000","last_peered":"2024-09-11T06:49:53.504605+0000","last_clean":"2024-09-11T06:49:53.504605+0000","last_became_active":"2024-09-11T06:49:50.552853+0000","last_became_peered":"2024-09-11T06:49:50.552853+0000","last_unstale":"2024-09-11T06:49:53.504605+0000","last_undegraded":"2024-09-11T06:49:53.504605+0000","last_fullsized":"2024-09-11T06:49:53.504605+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:51.820084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001910244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":68,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-11T06:49:52.931358+0000","last_change":"2024-09-11T06:49:21.274656+0000","last_active":"2024-09-11T06:49:52.931358+0000","last_peered":"2024-09-11T06:49:52.931358+0000","last_clean":"2024-09-11T06:49:52.931358+0000","last_became_active":"2024-09-11T06:49:21.273719+0000","last_became_peered":"2024-09-11T06:49:21.273719+0000","last_unstale":"2024-09-11T06:49:52.931358+0000","last_undegraded":"2024-09-11T06:49:52.931358+0000","last_fullsized":"2024-09-11T06:49:52.931358+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:59.900889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010163780000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":73,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-11T06:49:52.934821+0000","last_change":"2024-09-11T06:49:50.555558+0000","last_active":"2024-09-11T06:49:52.934821+0000","last_peered":"2024-09-11T06:49:52.934821+0000","last_clean":"2024-09-11T06:49:52.934821+0000","last_became_active":"2024-09-11T06:49:50.554687+0000","last_became_peered":"2024-09-11T06:49:50.554687+0000","last_unstale":"2024-09-11T06:49:52.934821+0000","last_undegraded":"2024-09-11T06:49:52.934821+0000","last_fullsized":"2024-09-11T06:49:52.934821+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:59:03.284180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077615599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":62,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-11T06:49:53.504677+0000","last_change":"2024-09-11T06:49:13.251115+0000","last_active":"2024-09-11T06:49:53.504677+0000","last_peered":"2024-09-11T06:49:53.504677+0000","last_clean":"2024-09-11T06:49:53.504677+0000","last_became_active":"2024-09-11T06:49:11.235231+0000","last_became_peered":"2024-09-11T06:49:11.235231+0000","last_unstale":"2024-09-11T06:49:53.504677+0000","last_undegraded":"2024-09-11T06:49:53.504677+0000","last_fullsized":"2024-09-11T06:49:53.504677+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:01:21.028466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011212469999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":227,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-11T06:49:52.938602+0000","last_change":"2024-09-11T06:49:52.938602+0000","last_active":"2024-09-11T06:49:52.938602+0000","last_peered":"2024-09-11T06:49:52.938602+0000","last_clean":"2024-09-11T06:49:52.938602+0000","last_became_active":"2024-09-11T06:49:52.937483+0000","last_became_peered":"2024-09-11T06:49:52.937483+0000","last_unstale":"2024-09-11T06:49:52.938602+0000","last_undegraded":"2024-09-11T06:49:52.938602+0000","last_fullsized":"2024-09-11T06:49:52.938602+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_clean_scrub_stamp":"2024-09-11T06:49:05.088839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:34:06.580391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640268,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":446464,"data_stored":157795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640269,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":442368,"data_stored":157776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640269,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1036288,"data_stored":748144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640270,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1040384,"data_stored":748163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":47244640270,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640271,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1024000,"data_stored":743566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:50:00.315 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:00.313+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:00.391 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:00.391 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:00.395 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:00.392+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:00.406 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:50:00.496 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:00.493+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:00.564 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:00.564 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:00.578 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:50:00.619 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:00.617+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:00.698 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:00.696+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:00.710 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:00.710 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:00.724 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:50:00.799 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:00.801+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:00.892 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:00.892 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:00.900 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:00.896+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:00.906 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:50:00.989 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T06:50:01.040 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:01.040+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:01.076 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:01.077 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:01.092 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:50:01.101 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:01.101+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:01.202 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:01.200+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:01.251 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:01.251 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:01.265 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:50:01.308 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:01.308+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:01.404 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:01.405+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:01.436 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:01.437 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:01.451 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:50:01.506 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:01.501+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:01.577 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:01.577 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:01.591 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:50:01.606 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:01.601+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:01.708 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:01.704+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:01.708 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:01.708 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:01.723 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:50:01.809 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:01.808+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:01.885 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:01.886 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:01.900 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:50:01.941 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:01.940+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:01.988 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is unset 2024-09-11T06:50:02.010 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:02.008+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:02.061 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:02.061 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:02.076 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:50:02.117 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:02.116+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:02.207 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:02.207 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:02.212 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:02.208+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:02.220 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:50:02.256 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:50:02.312 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:02.309+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:02.338 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:02.338 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:02.352 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:50:02.409 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:50:02.410 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:50:02.413 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:02.412+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:02.426 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:50:02.427 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:50:02.443 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:50:02.444 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:50:02.464 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:50:02.464 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:50:02.483 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:50:02.483 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:50:02.503 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:50:02.503 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:50:02.514 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:02.509+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:02.524 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:02.524 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:02.524 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:02.524 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:50:02.524 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.524 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:02.524 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:50:02.524 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 56.351444833000002, 2024-09-11T06:50:02.525 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011420669, 2024-09-11T06:50:02.525 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:02.525 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:02.525 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:02.525 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:02.525 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:02.525 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:02.525 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 37 2024-09-11T06:50:02.525 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.525 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:02.525 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.525 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:02.525 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:50:02.526 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.526 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.526 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.526 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:02.526 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:50:02.526 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.526 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.526 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.526 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:02.526 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170509+0000", 2024-09-11T06:50:02.526 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.13e-06 2024-09-11T06:50:02.526 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.526 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.526 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:02.527 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170521+0000", 2024-09-11T06:50:02.527 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1776999999999999e-05 2024-09-11T06:50:02.527 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.527 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.527 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:02.527 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170538+0000", 2024-09-11T06:50:02.527 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7413000000000001e-05 2024-09-11T06:50:02.527 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.527 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.527 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:02.527 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170570+0000", 2024-09-11T06:50:02.527 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2313e-05 2024-09-11T06:50:02.528 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.528 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.528 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:02.528 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179768+0000", 2024-09-11T06:50:02.528 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0091978010000000002 2024-09-11T06:50:02.528 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.528 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.528 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:02.528 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179951+0000", 2024-09-11T06:50:02.528 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018227900000000001 2024-09-11T06:50:02.528 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.528 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.528 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:02.529 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.180175+0000", 2024-09-11T06:50:02.529 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022432800000000001 2024-09-11T06:50:02.529 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.529 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.529 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:02.529 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181112+0000", 2024-09-11T06:50:02.529 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00093712500000000002 2024-09-11T06:50:02.529 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.529 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.529 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:02.529 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181789+0000", 2024-09-11T06:50:02.529 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00067705100000000002 2024-09-11T06:50:02.529 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.529 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.530 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:02.530 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181871+0000", 2024-09-11T06:50:02.530 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1816000000000004e-05 2024-09-11T06:50:02.530 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.530 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.530 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:02.530 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181929+0000", 2024-09-11T06:50:02.530 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7636000000000002e-05 2024-09-11T06:50:02.530 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.530 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:02.530 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.530 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.530 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.531 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:02.531 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:02.531 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 56.250185872000003, 2024-09-11T06:50:02.531 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283336, 2024-09-11T06:50:02.531 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:02.531 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:02.531 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:02.531 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:02.531 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:02.531 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:02.531 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 87 2024-09-11T06:50:02.531 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.531 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:02.532 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.532 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:02.532 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:02.532 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.532 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.532 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.532 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:02.532 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:02.532 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.532 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.532 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.533 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:02.533 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271768+0000", 2024-09-11T06:50:02.533 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7599999999999996e-07 2024-09-11T06:50:02.533 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.533 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.533 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:02.533 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271860+0000", 2024-09-11T06:50:02.533 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2639999999999994e-05 2024-09-11T06:50:02.533 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.533 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.533 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:02.533 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271873+0000", 2024-09-11T06:50:02.533 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3139000000000001e-05 2024-09-11T06:50:02.534 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.534 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.534 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:02.534 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271882+0000", 2024-09-11T06:50:02.534 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2369999999999992e-06 2024-09-11T06:50:02.534 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.534 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.534 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:02.534 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277150+0000", 2024-09-11T06:50:02.534 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052684419999999999 2024-09-11T06:50:02.534 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.534 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.534 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:02.534 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277273+0000", 2024-09-11T06:50:02.534 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012278899999999999 2024-09-11T06:50:02.535 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.535 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.535 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:02.535 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277466+0000", 2024-09-11T06:50:02.535 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193578 2024-09-11T06:50:02.535 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.535 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.535 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:02.535 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.280808+0000", 2024-09-11T06:50:02.535 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003341971 2024-09-11T06:50:02.535 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.535 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.535 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:02.535 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284419+0000", 2024-09-11T06:50:02.536 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003610982 2024-09-11T06:50:02.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.536 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.536 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:02.536 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284531+0000", 2024-09-11T06:50:02.536 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011121299999999999 2024-09-11T06:50:02.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.536 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.536 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:02.536 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284600+0000", 2024-09-11T06:50:02.536 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9592999999999996e-05 2024-09-11T06:50:02.536 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.536 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:02.536 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.537 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.537 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:02.537 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:02.537 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 56.250055619999998, 2024-09-11T06:50:02.537 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283314, 2024-09-11T06:50:02.537 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:02.537 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:02.537 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:02.537 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:02.537 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:02.537 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:02.537 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 88 2024-09-11T06:50:02.537 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.537 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:02.537 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.538 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:02.538 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:02.538 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.538 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.538 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.538 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:02.538 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:02.538 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.538 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.538 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.538 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:02.538 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271898+0000", 2024-09-11T06:50:02.538 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2999999999999999e-07 2024-09-11T06:50:02.538 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.539 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.539 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:02.539 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271904+0000", 2024-09-11T06:50:02.539 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.767e-06 2024-09-11T06:50:02.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.539 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.539 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:02.539 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271913+0000", 2024-09-11T06:50:02.539 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7200000000000001e-06 2024-09-11T06:50:02.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.539 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.539 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:02.539 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271920+0000", 2024-09-11T06:50:02.539 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7100000000000001e-06 2024-09-11T06:50:02.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.540 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.540 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:02.540 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278684+0000", 2024-09-11T06:50:02.540 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0067642650000000002 2024-09-11T06:50:02.540 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.540 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.540 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:02.540 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278838+0000", 2024-09-11T06:50:02.540 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015371499999999999 2024-09-11T06:50:02.540 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.540 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.540 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:02.540 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.279040+0000", 2024-09-11T06:50:02.540 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020207499999999999 2024-09-11T06:50:02.541 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.541 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.541 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:02.541 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.281448+0000", 2024-09-11T06:50:02.541 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002407686 2024-09-11T06:50:02.541 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.541 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.541 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:02.541 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284615+0000", 2024-09-11T06:50:02.541 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031675499999999999 2024-09-11T06:50:02.541 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.541 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.541 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:02.542 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284682+0000", 2024-09-11T06:50:02.542 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7032000000000005e-05 2024-09-11T06:50:02.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.542 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.542 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:02.542 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284730+0000", 2024-09-11T06:50:02.542 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7790000000000002e-05 2024-09-11T06:50:02.542 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.542 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:02.542 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.542 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.543 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:02.543 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:50:02.543 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 56.082154398, 2024-09-11T06:50:02.543 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011114877, 2024-09-11T06:50:02.543 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:02.543 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:02.543 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:02.543 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:02.543 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:02.543 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:02.543 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 107 2024-09-11T06:50:02.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.543 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:02.543 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.544 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:02.544 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:50:02.544 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.544 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.544 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.544 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:02.544 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:50:02.544 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.544 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.544 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.544 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:02.544 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439799+0000", 2024-09-11T06:50:02.544 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.16e-07 2024-09-11T06:50:02.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.545 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.545 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:02.545 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439806+0000", 2024-09-11T06:50:02.545 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3640000000000004e-06 2024-09-11T06:50:02.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.545 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.545 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:02.545 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439816+0000", 2024-09-11T06:50:02.545 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0485999999999999e-05 2024-09-11T06:50:02.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.545 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.545 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:02.546 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439824+0000", 2024-09-11T06:50:02.546 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.6699999999999994e-06 2024-09-11T06:50:02.546 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.546 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.546 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:02.546 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443328+0000", 2024-09-11T06:50:02.546 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035044329999999999 2024-09-11T06:50:02.546 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.546 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.546 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:02.546 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443461+0000", 2024-09-11T06:50:02.546 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013272199999999999 2024-09-11T06:50:02.546 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.546 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.547 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:02.547 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443678+0000", 2024-09-11T06:50:02.547 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021674500000000001 2024-09-11T06:50:02.547 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.547 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.547 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:02.547 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449635+0000", 2024-09-11T06:50:02.547 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0059572119999999999 2024-09-11T06:50:02.547 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.547 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.547 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:02.547 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450750+0000", 2024-09-11T06:50:02.548 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001115654 2024-09-11T06:50:02.548 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.548 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.548 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:02.548 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450832+0000", 2024-09-11T06:50:02.548 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1188999999999998e-05 2024-09-11T06:50:02.548 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.548 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.548 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:02.548 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450913+0000", 2024-09-11T06:50:02.548 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1485999999999999e-05 2024-09-11T06:50:02.548 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.548 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:02.548 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.549 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.549 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.549 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:02.549 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:50:02.549 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 56.081563422999999, 2024-09-11T06:50:02.549 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011236556, 2024-09-11T06:50:02.549 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:02.549 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:02.549 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:02.549 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:02.549 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:02.549 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:02.549 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 109 2024-09-11T06:50:02.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.550 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:02.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.550 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:02.550 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:50:02.550 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.550 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:02.550 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:50:02.550 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.550 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:02.551 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440391+0000", 2024-09-11T06:50:02.551 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8360000000000001e-06 2024-09-11T06:50:02.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.551 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.551 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:02.551 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440560+0000", 2024-09-11T06:50:02.551 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000169085 2024-09-11T06:50:02.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.551 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.551 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:02.551 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440585+0000", 2024-09-11T06:50:02.551 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4530000000000001e-05 2024-09-11T06:50:02.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.551 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.552 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:02.552 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440606+0000", 2024-09-11T06:50:02.552 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1277e-05 2024-09-11T06:50:02.552 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.552 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.552 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:02.552 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446479+0000", 2024-09-11T06:50:02.552 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0058734989999999999 2024-09-11T06:50:02.552 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.552 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.552 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:02.552 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446649+0000", 2024-09-11T06:50:02.552 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016948199999999999 2024-09-11T06:50:02.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.553 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:02.553 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446854+0000", 2024-09-11T06:50:02.553 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020491500000000001 2024-09-11T06:50:02.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.553 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:02.553 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451380+0000", 2024-09-11T06:50:02.553 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0045264110000000001 2024-09-11T06:50:02.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.553 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:02.554 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451479+0000", 2024-09-11T06:50:02.554 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.8975999999999996e-05 2024-09-11T06:50:02.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.554 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.554 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:02.554 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451562+0000", 2024-09-11T06:50:02.554 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3213000000000004e-05 2024-09-11T06:50:02.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.554 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.554 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:02.554 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451626+0000", 2024-09-11T06:50:02.554 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3331999999999996e-05 2024-09-11T06:50:02.554 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.554 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:02.554 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.555 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.555 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:02.555 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:50:02.555 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 56.081314294999999, 2024-09-11T06:50:02.555 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011252866, 2024-09-11T06:50:02.555 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:02.555 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:02.555 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:02.555 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:02.555 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:02.555 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:02.555 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 110 2024-09-11T06:50:02.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.555 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:02.556 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.556 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:02.556 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:50:02.556 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.556 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.556 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.556 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:02.556 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:50:02.556 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.556 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.556 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.556 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:02.556 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440640+0000", 2024-09-11T06:50:02.557 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5430000000000001e-06 2024-09-11T06:50:02.557 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.557 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.557 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:02.557 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440651+0000", 2024-09-11T06:50:02.557 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0672999999999999e-05 2024-09-11T06:50:02.557 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.557 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.557 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:02.557 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440670+0000", 2024-09-11T06:50:02.557 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9726999999999999e-05 2024-09-11T06:50:02.557 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.557 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.558 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:02.558 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440691+0000", 2024-09-11T06:50:02.558 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0526e-05 2024-09-11T06:50:02.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.558 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.558 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:02.558 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448191+0000", 2024-09-11T06:50:02.558 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0075005990000000002 2024-09-11T06:50:02.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.558 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.558 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:02.558 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448390+0000", 2024-09-11T06:50:02.558 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000198815 2024-09-11T06:50:02.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.559 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.559 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:02.559 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448735+0000", 2024-09-11T06:50:02.559 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034424999999999999 2024-09-11T06:50:02.559 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.559 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.559 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:02.559 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451742+0000", 2024-09-11T06:50:02.559 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030077440000000001 2024-09-11T06:50:02.559 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.559 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.559 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:02.559 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451761+0000", 2024-09-11T06:50:02.559 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9196999999999999e-05 2024-09-11T06:50:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451834+0000", 2024-09-11T06:50:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2071999999999997e-05 2024-09-11T06:50:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451891+0000", 2024-09-11T06:50:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7720000000000003e-05 2024-09-11T06:50:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.561 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:02.561 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:02.561 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 56.081206420000001, 2024-09-11T06:50:02.561 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014944941, 2024-09-11T06:50:02.561 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:02.561 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:02.561 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:02.561 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:02.561 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:02.561 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:02.561 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 111 2024-09-11T06:50:02.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.561 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:02.561 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.562 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:02.562 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:02.562 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.562 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:02.562 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:02.562 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.562 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:02.562 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440748+0000", 2024-09-11T06:50:02.562 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:50:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440760+0000", 2024-09-11T06:50:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1897e-05 2024-09-11T06:50:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440782+0000", 2024-09-11T06:50:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2552999999999999e-05 2024-09-11T06:50:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440793+0000", 2024-09-11T06:50:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0643e-05 2024-09-11T06:50:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449755+0000", 2024-09-11T06:50:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089622620000000004 2024-09-11T06:50:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449918+0000", 2024-09-11T06:50:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163086 2024-09-11T06:50:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:02.565 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450150+0000", 2024-09-11T06:50:02.565 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023204500000000001 2024-09-11T06:50:02.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.565 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.565 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:02.565 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454533+0000", 2024-09-11T06:50:02.565 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0043831349999999998 2024-09-11T06:50:02.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.565 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.565 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:02.565 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455511+0000", 2024-09-11T06:50:02.565 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0009776050000000001 2024-09-11T06:50:02.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.565 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.566 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:02.566 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455597+0000", 2024-09-11T06:50:02.566 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6446000000000005e-05 2024-09-11T06:50:02.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.566 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.566 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:02.566 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455691+0000", 2024-09-11T06:50:02.566 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3832000000000006e-05 2024-09-11T06:50:02.566 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.566 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:02.566 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.566 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.567 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:02.567 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:02.567 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 56.081136657000002, 2024-09-11T06:50:02.567 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018885411000000001, 2024-09-11T06:50:02.567 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:02.567 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:02.567 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:02.567 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:02.567 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:02.567 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:02.567 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 112 2024-09-11T06:50:02.567 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.567 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:02.567 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.568 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:02.568 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:02.568 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.568 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.568 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:02.568 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:02.568 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.568 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.568 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:02.568 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440817+0000", 2024-09-11T06:50:02.569 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:50:02.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.569 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:02.569 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441003+0000", 2024-09-11T06:50:02.569 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018661200000000001 2024-09-11T06:50:02.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.569 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:02.569 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441027+0000", 2024-09-11T06:50:02.569 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3920000000000001e-05 2024-09-11T06:50:02.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.569 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:02.570 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441062+0000", 2024-09-11T06:50:02.570 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4663000000000003e-05 2024-09-11T06:50:02.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.570 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:02.570 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451981+0000", 2024-09-11T06:50:02.570 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010918926000000001 2024-09-11T06:50:02.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.570 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:02.570 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452111+0000", 2024-09-11T06:50:02.570 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129529 2024-09-11T06:50:02.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.571 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:02.571 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452311+0000", 2024-09-11T06:50:02.571 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020013799999999999 2024-09-11T06:50:02.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.571 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:02.571 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455809+0000", 2024-09-11T06:50:02.571 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003498443 2024-09-11T06:50:02.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.571 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:02.571 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459566+0000", 2024-09-11T06:50:02.572 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003757271 2024-09-11T06:50:02.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.572 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:02.572 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459640+0000", 2024-09-11T06:50:02.572 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3949000000000006e-05 2024-09-11T06:50:02.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.572 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:02.572 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459701+0000", 2024-09-11T06:50:02.572 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1117000000000005e-05 2024-09-11T06:50:02.572 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.573 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:02.573 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.573 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:02.573 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:02.573 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 56.080857805999997, 2024-09-11T06:50:02.573 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018760915, 2024-09-11T06:50:02.573 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:02.573 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:02.573 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:02.573 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:02.574 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:02.574 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:02.574 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 113 2024-09-11T06:50:02.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.574 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:02.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.574 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:02.574 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:02.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.574 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:02.574 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:02.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.575 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:02.575 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441096+0000", 2024-09-11T06:50:02.575 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4130000000000001e-06 2024-09-11T06:50:02.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.575 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:02.575 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441110+0000", 2024-09-11T06:50:02.575 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3900000000000001e-05 2024-09-11T06:50:02.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.575 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:02.576 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441131+0000", 2024-09-11T06:50:02.576 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0636999999999999e-05 2024-09-11T06:50:02.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.576 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:02.576 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441147+0000", 2024-09-11T06:50:02.576 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6512999999999999e-05 2024-09-11T06:50:02.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.576 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:02.576 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453274+0000", 2024-09-11T06:50:02.576 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012126764999999999 2024-09-11T06:50:02.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:02.577 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453498+0000", 2024-09-11T06:50:02.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000223568 2024-09-11T06:50:02.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:02.577 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453872+0000", 2024-09-11T06:50:02.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037467099999999998 2024-09-11T06:50:02.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:02.577 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458709+0000", 2024-09-11T06:50:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0048365680000000003 2024-09-11T06:50:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459716+0000", 2024-09-11T06:50:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010075240000000001 2024-09-11T06:50:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459779+0000", 2024-09-11T06:50:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2336999999999996e-05 2024-09-11T06:50:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:02.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459856+0000", 2024-09-11T06:50:02.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7019000000000002e-05 2024-09-11T06:50:02.579 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.579 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:02.579 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.579 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:02.579 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:02.579 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 56.080781066999997, 2024-09-11T06:50:02.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018963114, 2024-09-11T06:50:02.579 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:02.579 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:02.580 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:02.580 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:02.580 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:02.580 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:02.580 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 114 2024-09-11T06:50:02.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.580 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:02.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.580 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:02.580 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:02.580 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.580 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9299999999999996e-07 2024-09-11T06:50:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441180+0000", 2024-09-11T06:50:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7200000000000006e-06 2024-09-11T06:50:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441265+0000", 2024-09-11T06:50:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4753000000000007e-05 2024-09-11T06:50:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441288+0000", 2024-09-11T06:50:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2860000000000001e-05 2024-09-11T06:50:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454585+0000", 2024-09-11T06:50:02.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013297286 2024-09-11T06:50:02.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:02.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454714+0000", 2024-09-11T06:50:02.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000128465 2024-09-11T06:50:02.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:02.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454909+0000", 2024-09-11T06:50:02.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019522499999999999 2024-09-11T06:50:02.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459868+0000", 2024-09-11T06:50:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0049591239999999997 2024-09-11T06:50:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459988+0000", 2024-09-11T06:50:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000120532 2024-09-11T06:50:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460071+0000", 2024-09-11T06:50:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2256000000000007e-05 2024-09-11T06:50:02.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.585 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:02.585 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460135+0000", 2024-09-11T06:50:02.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3999999999999997e-05 2024-09-11T06:50:02.585 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.585 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:02.585 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.585 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:02.585 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:02.585 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 56.080623438000003, 2024-09-11T06:50:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.019036849000000002, 2024-09-11T06:50:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 115 2024-09-11T06:50:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:02.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.587 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:02.587 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:02.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.587 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:02.587 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441330+0000", 2024-09-11T06:50:02.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1930000000000001e-06 2024-09-11T06:50:02.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.588 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:02.588 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441544+0000", 2024-09-11T06:50:02.588 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021405799999999999 2024-09-11T06:50:02.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.588 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:02.588 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441572+0000", 2024-09-11T06:50:02.588 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7317e-05 2024-09-11T06:50:02.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.588 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:02.588 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441593+0000", 2024-09-11T06:50:02.588 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1702999999999999e-05 2024-09-11T06:50:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455891+0000", 2024-09-11T06:50:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01429708 2024-09-11T06:50:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456046+0000", 2024-09-11T06:50:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000155682 2024-09-11T06:50:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456242+0000", 2024-09-11T06:50:02.590 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000195692 2024-09-11T06:50:02.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:02.590 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459876+0000", 2024-09-11T06:50:02.590 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036336480000000002 2024-09-11T06:50:02.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:02.590 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460245+0000", 2024-09-11T06:50:02.590 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036901700000000002 2024-09-11T06:50:02.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:02.591 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460316+0000", 2024-09-11T06:50:02.591 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1846000000000003e-05 2024-09-11T06:50:02.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.592 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:02.592 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460366+0000", 2024-09-11T06:50:02.592 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9613000000000001e-05 2024-09-11T06:50:02.592 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.592 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:02.592 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.592 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:02.592 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:02.593 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 56.080301358, 2024-09-11T06:50:02.593 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01904432, 2024-09-11T06:50:02.593 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:02.593 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:02.593 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:02.593 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:02.593 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:02.593 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:02.593 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 116 2024-09-11T06:50:02.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.593 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:02.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.594 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:02.594 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:02.594 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.594 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:02.594 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:02.594 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.594 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:02.594 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441653+0000", 2024-09-11T06:50:02.594 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:50:02.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.595 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:02.595 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441665+0000", 2024-09-11T06:50:02.595 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1653e-05 2024-09-11T06:50:02.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.595 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:02.595 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441684+0000", 2024-09-11T06:50:02.595 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9490000000000001e-05 2024-09-11T06:50:02.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.595 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:02.596 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441699+0000", 2024-09-11T06:50:02.596 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5044e-05 2024-09-11T06:50:02.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.596 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:02.596 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457297+0000", 2024-09-11T06:50:02.596 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015598241000000001 2024-09-11T06:50:02.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.596 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:02.596 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457577+0000", 2024-09-11T06:50:02.596 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027930799999999998 2024-09-11T06:50:02.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.597 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:02.597 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457957+0000", 2024-09-11T06:50:02.597 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00038088 2024-09-11T06:50:02.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.597 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:02.597 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460432+0000", 2024-09-11T06:50:02.597 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024743349999999998 2024-09-11T06:50:02.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.597 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:02.597 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460584+0000", 2024-09-11T06:50:02.598 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015232900000000001 2024-09-11T06:50:02.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.598 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:02.598 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460646+0000", 2024-09-11T06:50:02.598 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1883000000000004e-05 2024-09-11T06:50:02.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.598 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:02.598 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460696+0000", 2024-09-11T06:50:02.598 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9589999999999998e-05 2024-09-11T06:50:02.598 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.598 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:02.599 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.599 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:02.599 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:02.599 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 56.080229291999999, 2024-09-11T06:50:02.599 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.021834473, 2024-09-11T06:50:02.599 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:02.599 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:02.599 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:02.599 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:02.599 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:02.599 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:02.600 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 117 2024-09-11T06:50:02.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.600 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:02.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.600 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:02.600 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:02.600 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.600 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:02.600 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:02.600 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.601 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:02.601 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441725+0000", 2024-09-11T06:50:02.601 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5069999999999999e-06 2024-09-11T06:50:02.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.601 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:02.601 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441738+0000", 2024-09-11T06:50:02.601 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2813000000000001e-05 2024-09-11T06:50:02.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.601 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:02.602 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441759+0000", 2024-09-11T06:50:02.602 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1246999999999998e-05 2024-09-11T06:50:02.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.602 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:02.602 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441779+0000", 2024-09-11T06:50:02.602 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0392999999999999e-05 2024-09-11T06:50:02.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.602 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:02.602 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458804+0000", 2024-09-11T06:50:02.602 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.017024182999999998 2024-09-11T06:50:02.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458977+0000", 2024-09-11T06:50:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000173926 2024-09-11T06:50:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459170+0000", 2024-09-11T06:50:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000192184 2024-09-11T06:50:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.462300+0000", 2024-09-11T06:50:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031304969999999999 2024-09-11T06:50:02.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.604 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:02.604 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463408+0000", 2024-09-11T06:50:02.604 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011077839999999999 2024-09-11T06:50:02.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.604 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:02.604 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463497+0000", 2024-09-11T06:50:02.604 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8828999999999999e-05 2024-09-11T06:50:02.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.605 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:02.605 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463558+0000", 2024-09-11T06:50:02.605 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1110000000000003e-05 2024-09-11T06:50:02.605 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.605 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:02.605 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.605 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:02.605 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:50:02.605 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 55.703721604000002, 2024-09-11T06:50:02.605 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012216898, 2024-09-11T06:50:02.606 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:02.606 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:02.606 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:02.606 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:02.606 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:02.606 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:02.606 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 137 2024-09-11T06:50:02.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.606 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:02.606 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.606 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:02.606 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:50:02.606 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:50:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818232+0000", 2024-09-11T06:50:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2839999999999999e-06 2024-09-11T06:50:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818326+0000", 2024-09-11T06:50:02.608 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3531999999999998e-05 2024-09-11T06:50:02.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.608 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:02.608 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818337+0000", 2024-09-11T06:50:02.608 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0655999999999999e-05 2024-09-11T06:50:02.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.608 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:02.608 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818342+0000", 2024-09-11T06:50:02.608 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4600000000000002e-06 2024-09-11T06:50:02.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823890+0000", 2024-09-11T06:50:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0055480700000000004 2024-09-11T06:50:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823953+0000", 2024-09-11T06:50:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2848999999999994e-05 2024-09-11T06:50:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824074+0000", 2024-09-11T06:50:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121309 2024-09-11T06:50:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.610 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:02.610 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828043+0000", 2024-09-11T06:50:02.610 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0039685240000000002 2024-09-11T06:50:02.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.610 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:02.610 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830293+0000", 2024-09-11T06:50:02.610 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0022503179999999998 2024-09-11T06:50:02.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.610 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:02.610 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830366+0000", 2024-09-11T06:50:02.611 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2539999999999993e-05 2024-09-11T06:50:02.611 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.611 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.611 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:02.611 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830448+0000", 2024-09-11T06:50:02.611 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2355999999999996e-05 2024-09-11T06:50:02.611 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.611 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:02.611 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.611 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.611 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.611 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:02.611 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:02.612 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 55.70358779, 2024-09-11T06:50:02.612 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013136297, 2024-09-11T06:50:02.612 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:02.612 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:02.612 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:02.612 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:02.612 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:02.612 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:02.612 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 138 2024-09-11T06:50:02.612 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.612 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:02.612 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.612 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.89e-07 2024-09-11T06:50:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:02.614 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818369+0000", 2024-09-11T06:50:02.614 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4170000000000001e-06 2024-09-11T06:50:02.614 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.614 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.614 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:02.614 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818376+0000", 2024-09-11T06:50:02.614 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4270000000000002e-06 2024-09-11T06:50:02.614 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.614 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.614 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:02.614 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818383+0000", 2024-09-11T06:50:02.614 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3060000000000003e-06 2024-09-11T06:50:02.614 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.615 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.615 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:02.615 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824783+0000", 2024-09-11T06:50:02.615 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064003020000000001 2024-09-11T06:50:02.615 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.615 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.615 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:02.615 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825013+0000", 2024-09-11T06:50:02.615 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023039499999999999 2024-09-11T06:50:02.615 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.615 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.615 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:02.615 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825484+0000", 2024-09-11T06:50:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000470562 2024-09-11T06:50:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828633+0000", 2024-09-11T06:50:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031496689999999999 2024-09-11T06:50:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831413+0000", 2024-09-11T06:50:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027793219999999999 2024-09-11T06:50:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:02.617 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831450+0000", 2024-09-11T06:50:02.617 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7370999999999998e-05 2024-09-11T06:50:02.617 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.617 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.617 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:02.617 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831501+0000", 2024-09-11T06:50:02.617 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1036999999999999e-05 2024-09-11T06:50:02.617 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.617 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:02.617 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.617 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.617 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.617 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:02.617 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 55.70356331, 2024-09-11T06:50:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013223011, 2024-09-11T06:50:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 139 2024-09-11T06:50:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.619 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.619 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.619 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:02.619 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:02.619 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.619 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.619 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.619 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:02.619 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818390+0000", 2024-09-11T06:50:02.619 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5600000000000001e-07 2024-09-11T06:50:02.619 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.619 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.619 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:02.620 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818394+0000", 2024-09-11T06:50:02.620 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5929999999999999e-06 2024-09-11T06:50:02.620 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.620 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.620 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:02.620 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818399+0000", 2024-09-11T06:50:02.620 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7740000000000004e-06 2024-09-11T06:50:02.620 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.620 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.620 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:02.620 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818412+0000", 2024-09-11T06:50:02.620 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2625e-05 2024-09-11T06:50:02.620 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.621 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.621 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:02.621 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826205+0000", 2024-09-11T06:50:02.621 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0077930730000000002 2024-09-11T06:50:02.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.621 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.621 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:02.621 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826281+0000", 2024-09-11T06:50:02.621 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5876000000000003e-05 2024-09-11T06:50:02.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.621 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.621 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:02.621 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826384+0000", 2024-09-11T06:50:02.621 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103838 2024-09-11T06:50:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830642+0000", 2024-09-11T06:50:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004257083 2024-09-11T06:50:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831509+0000", 2024-09-11T06:50:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086730900000000003 2024-09-11T06:50:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831547+0000", 2024-09-11T06:50:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7793000000000002e-05 2024-09-11T06:50:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831612+0000", 2024-09-11T06:50:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5691000000000005e-05 2024-09-11T06:50:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 55.703534615999999, 2024-09-11T06:50:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016098024999999998, 2024-09-11T06:50:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 140 2024-09-11T06:50:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5199999999999998e-07 2024-09-11T06:50:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818479+0000", 2024-09-11T06:50:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0368000000000002e-05 2024-09-11T06:50:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818487+0000", 2024-09-11T06:50:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2339999999999994e-06 2024-09-11T06:50:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818491+0000", 2024-09-11T06:50:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4329999999999997e-06 2024-09-11T06:50:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828080+0000", 2024-09-11T06:50:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0095890449999999992 2024-09-11T06:50:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828130+0000", 2024-09-11T06:50:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9239000000000001e-05 2024-09-11T06:50:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828208+0000", 2024-09-11T06:50:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.852e-05 2024-09-11T06:50:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831625+0000", 2024-09-11T06:50:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034169410000000002 2024-09-11T06:50:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834411+0000", 2024-09-11T06:50:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002786111 2024-09-11T06:50:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834457+0000", 2024-09-11T06:50:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6041000000000002e-05 2024-09-11T06:50:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834516+0000", 2024-09-11T06:50:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8740999999999999e-05 2024-09-11T06:50:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 55.703453549000002, 2024-09-11T06:50:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016588772000000002, 2024-09-11T06:50:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 141 2024-09-11T06:50:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818500+0000", 2024-09-11T06:50:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0800000000000005e-07 2024-09-11T06:50:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818505+0000", 2024-09-11T06:50:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2429999999999996e-06 2024-09-11T06:50:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818510+0000", 2024-09-11T06:50:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9339999999999997e-06 2024-09-11T06:50:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818514+0000", 2024-09-11T06:50:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7270000000000001e-06 2024-09-11T06:50:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828711+0000", 2024-09-11T06:50:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010197031 2024-09-11T06:50:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828853+0000", 2024-09-11T06:50:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014201200000000001 2024-09-11T06:50:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.829115+0000", 2024-09-11T06:50:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026257500000000003 2024-09-11T06:50:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831629+0000", 2024-09-11T06:50:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025138169999999998 2024-09-11T06:50:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834991+0000", 2024-09-11T06:50:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033624459999999998 2024-09-11T06:50:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835032+0000", 2024-09-11T06:50:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1001000000000003e-05 2024-09-11T06:50:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835088+0000", 2024-09-11T06:50:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5478e-05 2024-09-11T06:50:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.636 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:50:02.636 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:50:02.636 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 55.079818646, 2024-09-11T06:50:02.636 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011161912, 2024-09-11T06:50:02.636 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:02.636 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:02.636 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:02.636 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:02.636 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:02.636 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:02.636 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 158 2024-09-11T06:50:02.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.636 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:50:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:50:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442135+0000", 2024-09-11T06:50:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3e-07 2024-09-11T06:50:02.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.638 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:02.638 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442259+0000", 2024-09-11T06:50:02.638 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000123903 2024-09-11T06:50:02.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.638 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:02.638 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442289+0000", 2024-09-11T06:50:02.638 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0833000000000001e-05 2024-09-11T06:50:02.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.638 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:02.639 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442301+0000", 2024-09-11T06:50:02.639 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1433e-05 2024-09-11T06:50:02.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.639 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:02.639 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449796+0000", 2024-09-11T06:50:02.639 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0074954959999999999 2024-09-11T06:50:02.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.639 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:02.639 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449917+0000", 2024-09-11T06:50:02.639 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121033 2024-09-11T06:50:02.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.640 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.640 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:02.640 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.450166+0000", 2024-09-11T06:50:02.640 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00024832400000000002 2024-09-11T06:50:02.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.640 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.640 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:02.640 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.452630+0000", 2024-09-11T06:50:02.640 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002464188 2024-09-11T06:50:02.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.640 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.640 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:02.640 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453149+0000", 2024-09-11T06:50:02.640 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00051860999999999997 2024-09-11T06:50:02.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.641 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:02.641 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453234+0000", 2024-09-11T06:50:02.641 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.5865000000000005e-05 2024-09-11T06:50:02.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.641 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:02.641 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453296+0000", 2024-09-11T06:50:02.641 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1496999999999998e-05 2024-09-11T06:50:02.641 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.641 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:02.641 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.642 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:50:02.642 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:50:02.642 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 55.079632543999999, 2024-09-11T06:50:02.642 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011561900999999999, 2024-09-11T06:50:02.642 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:02.642 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:02.642 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:02.642 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:02.642 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:02.642 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:02.642 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 159 2024-09-11T06:50:02.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.643 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:02.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.643 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:02.643 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:50:02.643 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.643 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:02.643 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:50:02.643 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:02.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.644 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:02.644 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442321+0000", 2024-09-11T06:50:02.644 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8400000000000003e-07 2024-09-11T06:50:02.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.644 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:02.644 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442328+0000", 2024-09-11T06:50:02.644 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5130000000000002e-06 2024-09-11T06:50:02.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.644 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:02.644 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442341+0000", 2024-09-11T06:50:02.644 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.363e-05 2024-09-11T06:50:02.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:02.645 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442349+0000", 2024-09-11T06:50:02.645 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4760000000000001e-06 2024-09-11T06:50:02.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:02.645 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451257+0000", 2024-09-11T06:50:02.645 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089088569999999992 2024-09-11T06:50:02.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:02.645 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451389+0000", 2024-09-11T06:50:02.645 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013110899999999999 2024-09-11T06:50:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451608+0000", 2024-09-11T06:50:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000219588 2024-09-11T06:50:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453517+0000", 2024-09-11T06:50:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0019088779999999999 2024-09-11T06:50:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453727+0000", 2024-09-11T06:50:02.647 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020955100000000001 2024-09-11T06:50:02.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.647 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:02.647 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453815+0000", 2024-09-11T06:50:02.647 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8858999999999997e-05 2024-09-11T06:50:02.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:02.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:02.647 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:02.647 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453882+0000", 2024-09-11T06:50:02.647 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.6556000000000004e-05 2024-09-11T06:50:02.647 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.647 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:02.647 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.648 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:02.648 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:02.648 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:02.649 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:50:02.650 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:02.649+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:02.750 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:02.745+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:02.817 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:02.817 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:02.818 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:50:02.818 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:02.832 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:50:02.875 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:02.873+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:02.952 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:02.952+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:02.997 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:02.997 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:02.997 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:50:02.997 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:50:02.997 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:03.012 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:50:03.059 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:03.057+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:03.132 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:03.132 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:03.132 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:03.132 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [] 2024-09-11T06:50:03.132 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:03.146 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:50:03.187 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:03.185+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:03.254 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:03.249+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:03.308 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:03.308 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:03.308 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:50:03.308 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:03.322 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:50:03.363 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:03.361+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:03.456 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:03.453+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:03.486 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:03.486 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:03.486 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:50:03.487 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:50:03.487 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:03.501 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:50:03.587 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:03.585+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:03.658 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:03.656+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:03.668 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:03.668 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:03.668 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:03.668 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:50:03.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.668 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:50:03.668 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:50:03.668 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 52.438317292000001, 2024-09-11T06:50:03.668 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010553425999999999, 2024-09-11T06:50:03.669 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:03.669 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:03.669 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:03.669 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:03.669 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:03.669 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:03.669 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:50:03.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.669 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:03.669 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.669 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:03.669 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:50:03.669 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:03.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.670 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:03.670 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:50:03.670 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:03.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.670 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:03.670 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230274+0000", 2024-09-11T06:50:03.670 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.533e-06 2024-09-11T06:50:03.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.670 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:03.670 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230285+0000", 2024-09-11T06:50:03.671 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0540000000000001e-05 2024-09-11T06:50:03.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.671 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.671 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:03.671 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230309+0000", 2024-09-11T06:50:03.671 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4223000000000002e-05 2024-09-11T06:50:03.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.671 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.671 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:03.671 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230327+0000", 2024-09-11T06:50:03.671 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:50:03.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.671 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.671 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:03.672 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230478+0000", 2024-09-11T06:50:03.672 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000151459 2024-09-11T06:50:03.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.672 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.672 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:03.672 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230490+0000", 2024-09-11T06:50:03.672 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.147e-05 2024-09-11T06:50:03.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.672 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.672 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:03.672 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231803+0000", 2024-09-11T06:50:03.672 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001313158 2024-09-11T06:50:03.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.673 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:50:03.673 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231817+0000", 2024-09-11T06:50:03.673 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4197e-05 2024-09-11T06:50:03.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.673 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:03.673 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.235945+0000", 2024-09-11T06:50:03.673 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0041282480000000002 2024-09-11T06:50:03.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.673 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:03.674 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236312+0000", 2024-09-11T06:50:03.674 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036652700000000001 2024-09-11T06:50:03.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.674 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.674 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:50:03.674 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236689+0000", 2024-09-11T06:50:03.674 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000377117 2024-09-11T06:50:03.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.674 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.674 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:03.674 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238460+0000", 2024-09-11T06:50:03.674 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001770918 2024-09-11T06:50:03.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.674 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.675 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:03.675 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240627+0000", 2024-09-11T06:50:03.675 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021670779999999998 2024-09-11T06:50:03.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.675 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.675 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:03.675 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240751+0000", 2024-09-11T06:50:03.675 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012384900000000001 2024-09-11T06:50:03.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.676 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:03.676 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240821+0000", 2024-09-11T06:50:03.676 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0516e-05 2024-09-11T06:50:03.676 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:03.676 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:03.676 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:03.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.676 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop_reply(client.4230.0:1 2.3 e16/15 ondisk, result = 0)", 2024-09-11T06:50:03.676 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.238282+0000", 2024-09-11T06:50:03.676 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 52.430302967000003, 2024-09-11T06:50:03.676 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018970799999999999, 2024-09-11T06:50:03.677 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:03.677 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:03.677 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:03.677 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:03.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.677 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:03.677 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238282+0000", 2024-09-11T06:50:03.677 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:03.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.677 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:03.677 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238282+0000", 2024-09-11T06:50:03.677 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:03.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.678 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:03.678 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238284+0000", 2024-09-11T06:50:03.678 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:50:03.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.678 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.678 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:03.678 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238292+0000", 2024-09-11T06:50:03.678 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2390000000000002e-06 2024-09-11T06:50:03.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.678 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.678 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:03.678 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238314+0000", 2024-09-11T06:50:03.678 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1776999999999998e-05 2024-09-11T06:50:03.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.678 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.679 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:03.679 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238339+0000", 2024-09-11T06:50:03.679 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5119000000000001e-05 2024-09-11T06:50:03.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.679 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:03.679 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238438+0000", 2024-09-11T06:50:03.679 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9553e-05 2024-09-11T06:50:03.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.679 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:03.679 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238451+0000", 2024-09-11T06:50:03.679 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2286000000000001e-05 2024-09-11T06:50:03.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.680 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:03.680 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238472+0000", 2024-09-11T06:50:03.680 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1166999999999999e-05 2024-09-11T06:50:03.680 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:03.680 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:03.680 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:03.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.680 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:50:03.680 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.244627+0000", 2024-09-11T06:50:03.680 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 52.423958269000003, 2024-09-11T06:50:03.680 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00046141999999999998, 2024-09-11T06:50:03.681 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:03.681 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:03.681 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:03.681 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:03.681 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:03.681 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:03.681 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 2 2024-09-11T06:50:03.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.681 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:03.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.681 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:03.681 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244627+0000", 2024-09-11T06:50:03.682 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:03.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.682 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:03.682 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244627+0000", 2024-09-11T06:50:03.682 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:03.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.682 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:03.682 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244629+0000", 2024-09-11T06:50:03.682 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.95e-06 2024-09-11T06:50:03.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.682 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244640+0000", 2024-09-11T06:50:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1494e-05 2024-09-11T06:50:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244664+0000", 2024-09-11T06:50:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4006e-05 2024-09-11T06:50:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244684+0000", 2024-09-11T06:50:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.961e-05 2024-09-11T06:50:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:03.684 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244862+0000", 2024-09-11T06:50:03.684 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017863500000000001 2024-09-11T06:50:03.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.684 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:03.684 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245088+0000", 2024-09-11T06:50:03.684 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000225725 2024-09-11T06:50:03.684 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:03.684 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:03.684 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:03.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.684 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:50:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.245492+0000", 2024-09-11T06:50:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 52.423093326, 2024-09-11T06:50:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00042092600000000001, 2024-09-11T06:50:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 3 2024-09-11T06:50:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245492+0000", 2024-09-11T06:50:03.686 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:03.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.686 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:03.686 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245492+0000", 2024-09-11T06:50:03.686 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:03.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.686 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:03.686 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245493+0000", 2024-09-11T06:50:03.686 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.657e-06 2024-09-11T06:50:03.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.686 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:03.686 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245501+0000", 2024-09-11T06:50:03.687 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2490000000000004e-06 2024-09-11T06:50:03.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.687 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:03.687 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245523+0000", 2024-09-11T06:50:03.687 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2016999999999999e-05 2024-09-11T06:50:03.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.687 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:03.687 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245548+0000", 2024-09-11T06:50:03.687 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5060000000000001e-05 2024-09-11T06:50:03.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.687 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:03.687 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245720+0000", 2024-09-11T06:50:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017190200000000001 2024-09-11T06:50:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245913+0000", 2024-09-11T06:50:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193041 2024-09-11T06:50:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-11T06:50:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:50:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 51.436146448999999, 2024-09-11T06:50:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069904770000000001, 2024-09-11T06:50:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 5 2024-09-11T06:50:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:50:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.690 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:03.690 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:50:03.690 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:03.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.690 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:03.690 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232442+0000", 2024-09-11T06:50:03.690 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.044e-06 2024-09-11T06:50:03.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.690 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:03.690 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232460+0000", 2024-09-11T06:50:03.690 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7863000000000002e-05 2024-09-11T06:50:03.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.691 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:03.691 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232492+0000", 2024-09-11T06:50:03.691 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2812999999999999e-05 2024-09-11T06:50:03.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.691 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:03.691 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232510+0000", 2024-09-11T06:50:03.691 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7552999999999999e-05 2024-09-11T06:50:03.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.691 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:03.691 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232640+0000", 2024-09-11T06:50:03.692 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013029900000000001 2024-09-11T06:50:03.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.692 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:03.692 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232650+0000", 2024-09-11T06:50:03.692 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0023999999999999e-05 2024-09-11T06:50:03.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.692 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:03.692 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235641+0000", 2024-09-11T06:50:03.692 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00299089 2024-09-11T06:50:03.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.692 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:03.693 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235980+0000", 2024-09-11T06:50:03.693 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000338907 2024-09-11T06:50:03.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.693 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:03.693 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.236351+0000", 2024-09-11T06:50:03.693 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037087699999999998 2024-09-11T06:50:03.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.693 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:03.694 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238074+0000", 2024-09-11T06:50:03.694 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001722859 2024-09-11T06:50:03.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.694 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:03.694 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239160+0000", 2024-09-11T06:50:03.694 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001086074 2024-09-11T06:50:03.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.694 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:03.694 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239299+0000", 2024-09-11T06:50:03.694 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013933500000000001 2024-09-11T06:50:03.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.695 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.695 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:03.695 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239429+0000", 2024-09-11T06:50:03.695 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129939 2024-09-11T06:50:03.695 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:03.695 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:03.695 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:03.695 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.695 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.695 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop_reply(client.4230.0:5 2.2 e16/15 ondisk, result = 0)", 2024-09-11T06:50:03.695 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:12.238911+0000", 2024-09-11T06:50:03.695 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 51.429674161000001, 2024-09-11T06:50:03.695 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00053624900000000004, 2024-09-11T06:50:03.695 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:03.696 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:03.696 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:03.696 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:03.696 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.696 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:03.696 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238911+0000", 2024-09-11T06:50:03.696 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:03.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.696 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.696 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:03.696 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238911+0000", 2024-09-11T06:50:03.696 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:03.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.696 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.697 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:03.697 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238913+0000", 2024-09-11T06:50:03.697 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2069999999999998e-06 2024-09-11T06:50:03.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.697 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.697 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:03.697 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238923+0000", 2024-09-11T06:50:03.697 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0360000000000001e-05 2024-09-11T06:50:03.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.697 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.697 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:03.697 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238953+0000", 2024-09-11T06:50:03.697 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9258999999999999e-05 2024-09-11T06:50:03.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.698 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.698 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:03.698 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238967+0000", 2024-09-11T06:50:03.698 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3797e-05 2024-09-11T06:50:03.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.698 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.698 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:03.698 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239117+0000", 2024-09-11T06:50:03.698 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000150765 2024-09-11T06:50:03.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.698 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.698 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:03.698 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239143+0000", 2024-09-11T06:50:03.698 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5514e-05 2024-09-11T06:50:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239447+0000", 2024-09-11T06:50:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00030434700000000001 2024-09-11T06:50:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-11T06:50:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:50:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 50.422058497000002, 2024-09-11T06:50:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069253250000000004, 2024-09-11T06:50:03.700 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:03.700 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:03.700 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:03.700 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:03.700 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:03.700 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:03.700 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 7 2024-09-11T06:50:03.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.700 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:03.700 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.700 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:03.700 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:50:03.700 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:03.701 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.701 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.701 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:03.701 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:50:03.701 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:03.701 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.701 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.701 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:03.701 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246530+0000", 2024-09-11T06:50:03.701 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9469999999999999e-06 2024-09-11T06:50:03.701 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.701 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.701 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:03.701 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246545+0000", 2024-09-11T06:50:03.702 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5973000000000001e-05 2024-09-11T06:50:03.702 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.702 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.702 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:03.702 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246578+0000", 2024-09-11T06:50:03.702 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2759999999999998e-05 2024-09-11T06:50:03.702 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.702 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.702 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:03.702 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246595+0000", 2024-09-11T06:50:03.702 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6939000000000002e-05 2024-09-11T06:50:03.702 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.702 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.702 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:03.703 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246770+0000", 2024-09-11T06:50:03.703 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000174723 2024-09-11T06:50:03.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.703 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.703 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:03.703 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246789+0000", 2024-09-11T06:50:03.703 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9159000000000001e-05 2024-09-11T06:50:03.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.703 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.703 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:03.703 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250337+0000", 2024-09-11T06:50:03.703 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035480759999999998 2024-09-11T06:50:03.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.703 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.704 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:03.704 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250549+0000", 2024-09-11T06:50:03.704 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021192199999999999 2024-09-11T06:50:03.704 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.704 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.704 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:03.704 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250750+0000", 2024-09-11T06:50:03.704 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020104199999999999 2024-09-11T06:50:03.704 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.704 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.704 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:03.704 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.252455+0000", 2024-09-11T06:50:03.705 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001705249 2024-09-11T06:50:03.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.705 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.705 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:03.705 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253305+0000", 2024-09-11T06:50:03.705 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00084941200000000002 2024-09-11T06:50:03.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.705 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.705 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:03.705 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253388+0000", 2024-09-11T06:50:03.705 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3642999999999999e-05 2024-09-11T06:50:03.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.705 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.706 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:03.706 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253452+0000", 2024-09-11T06:50:03.706 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3479999999999995e-05 2024-09-11T06:50:03.706 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:03.706 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:03.706 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:03.706 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.706 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.706 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop_reply(client.4230.0:7 2.2 e18/15 ondisk, result = 0)", 2024-09-11T06:50:03.706 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:13.253128+0000", 2024-09-11T06:50:03.706 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 50.415456923000001, 2024-09-11T06:50:03.706 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034023700000000002, 2024-09-11T06:50:03.706 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:03.706 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:03.707 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:03.707 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:03.707 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.707 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:03.707 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253128+0000", 2024-09-11T06:50:03.707 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:03.707 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.707 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.707 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:03.707 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253128+0000", 2024-09-11T06:50:03.707 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:03.707 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.707 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.708 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:03.708 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253135+0000", 2024-09-11T06:50:03.708 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.657e-06 2024-09-11T06:50:03.708 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.708 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.708 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:03.708 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253144+0000", 2024-09-11T06:50:03.708 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0869999999999996e-06 2024-09-11T06:50:03.708 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.708 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.708 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:03.708 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253167+0000", 2024-09-11T06:50:03.708 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3252e-05 2024-09-11T06:50:03.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.709 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.709 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:03.709 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253186+0000", 2024-09-11T06:50:03.709 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9239999999999999e-05 2024-09-11T06:50:03.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.709 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.709 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:03.709 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253282+0000", 2024-09-11T06:50:03.709 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.6093000000000003e-05 2024-09-11T06:50:03.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.709 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.709 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:03.710 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253294+0000", 2024-09-11T06:50:03.710 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1986e-05 2024-09-11T06:50:03.710 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.710 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.710 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:03.710 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253468+0000", 2024-09-11T06:50:03.710 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017392199999999999 2024-09-11T06:50:03.710 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:03.710 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:03.710 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:03.710 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.710 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.710 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPushReply(2.2 37/36 [PushReplyOp(2:5cea7035:::rbd_info:head)])", 2024-09-11T06:50:03.710 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:50.557624+0000", 2024-09-11T06:50:03.711 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 13.110960858, 2024-09-11T06:50:03.711 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00069280800000000005, 2024-09-11T06:50:03.711 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:03.711 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:03.711 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:03.711 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:03.711 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.711 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:03.711 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:50.557624+0000", 2024-09-11T06:50:03.711 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:03.711 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.711 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.711 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:03.711 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:50.557624+0000", 2024-09-11T06:50:03.712 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:03.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.712 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.712 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:03.712 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:50.557625+0000", 2024-09-11T06:50:03.712 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.37e-07 2024-09-11T06:50:03.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.712 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.712 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:03.712 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:50.557633+0000", 2024-09-11T06:50:03.712 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6270000000000006e-06 2024-09-11T06:50:03.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.712 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.712 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:03.713 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:50.557650+0000", 2024-09-11T06:50:03.713 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6350000000000001e-05 2024-09-11T06:50:03.713 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.713 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.713 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:03.713 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:50.557667+0000", 2024-09-11T06:50:03.713 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7668000000000001e-05 2024-09-11T06:50:03.713 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.713 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.713 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:03.713 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:50.557832+0000", 2024-09-11T06:50:03.713 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016408699999999999 2024-09-11T06:50:03.713 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:03.713 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:03.714 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:03.714 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:50.558317+0000", 2024-09-11T06:50:03.714 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000485439 2024-09-11T06:50:03.714 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:03.714 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:03.714 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:03.714 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:03.714 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:03.714 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:03.715 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:50:03.759 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:03.757+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:03.847 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:03.847 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:03.847 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:50:03.847 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:03.859 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:03.857+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:03.861 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:50:03.960 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:03.957+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:03.985 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:03.985 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:03.985 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:50:03.985 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:50:03.985 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:04.000 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:50:04.010 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T06:50:04.092 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:04.092+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:04.125 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:04.125 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:50:04.125 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:50:04.125 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:50:04.126 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.126 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:54 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'32, mlcod=13'29)", 2024-09-11T06:50:04.126 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:50:04.126 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 57.911856063999998, 2024-09-11T06:50:04.126 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034271670000000001, 2024-09-11T06:50:04.126 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:04.126 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:04.126 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:04.126 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:04.126 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.126 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:04.126 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:50:04.127 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.127 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.127 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.127 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:04.127 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:50:04.127 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.127 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.127 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.127 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:04.127 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210361+0000", 2024-09-11T06:50:04.127 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1260000000000001e-06 2024-09-11T06:50:04.127 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.127 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.128 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:04.128 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210370+0000", 2024-09-11T06:50:04.128 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8200000000000003e-06 2024-09-11T06:50:04.128 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.128 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.128 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:04.128 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210390+0000", 2024-09-11T06:50:04.128 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0126e-05 2024-09-11T06:50:04.128 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.128 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.128 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:04.128 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210404+0000", 2024-09-11T06:50:04.129 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4589999999999999e-05 2024-09-11T06:50:04.129 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.129 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.129 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:04.129 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210538+0000", 2024-09-11T06:50:04.129 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000133719 2024-09-11T06:50:04.129 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.129 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.129 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:04.129 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210593+0000", 2024-09-11T06:50:04.129 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5016999999999999e-05 2024-09-11T06:50:04.129 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.129 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.130 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:04.130 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213695+0000", 2024-09-11T06:50:04.130 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003101866 2024-09-11T06:50:04.130 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.130 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.130 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:04.130 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213787+0000", 2024-09-11T06:50:04.130 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1903000000000004e-05 2024-09-11T06:50:04.130 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.130 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:04.130 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.130 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.130 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.130 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:55 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'33, mlcod=13'29)", 2024-09-11T06:50:04.131 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:50:04.131 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 57.910836826000001, 2024-09-11T06:50:04.131 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.005086091, 2024-09-11T06:50:04.131 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:04.131 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:04.131 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:04.131 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:04.131 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.131 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:04.131 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:50:04.131 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.131 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.132 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.132 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:04.132 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:50:04.132 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.132 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.132 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.132 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:04.132 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211380+0000", 2024-09-11T06:50:04.132 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.097e-06 2024-09-11T06:50:04.132 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.132 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.132 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:04.132 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211537+0000", 2024-09-11T06:50:04.132 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000157369 2024-09-11T06:50:04.133 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.133 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.133 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:04.133 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211559+0000", 2024-09-11T06:50:04.133 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000001e-05 2024-09-11T06:50:04.133 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.133 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.133 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:04.133 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211573+0000", 2024-09-11T06:50:04.133 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4946000000000001e-05 2024-09-11T06:50:04.133 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.133 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.133 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:04.133 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211676+0000", 2024-09-11T06:50:04.134 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0001026 2024-09-11T06:50:04.134 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.134 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.134 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:04.134 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211709+0000", 2024-09-11T06:50:04.134 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3071999999999998e-05 2024-09-11T06:50:04.134 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.134 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.134 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:04.134 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216379+0000", 2024-09-11T06:50:04.134 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0046693780000000001 2024-09-11T06:50:04.134 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.134 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.134 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:04.135 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216465+0000", 2024-09-11T06:50:04.135 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6469000000000002e-05 2024-09-11T06:50:04.135 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.135 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:04.135 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.135 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.135 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.135 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:56 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'34, mlcod=13'29)", 2024-09-11T06:50:04.135 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:50:04.135 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 57.909402966999998, 2024-09-11T06:50:04.135 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0060310629999999997, 2024-09-11T06:50:04.135 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:04.135 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:04.135 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:04.135 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:04.136 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.136 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:04.136 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:50:04.136 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.136 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.136 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.136 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:04.136 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:50:04.136 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.136 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.136 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.136 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:04.136 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212814+0000", 2024-09-11T06:50:04.136 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0869999999999999e-06 2024-09-11T06:50:04.136 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.137 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.137 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:04.137 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212822+0000", 2024-09-11T06:50:04.137 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0870000000000007e-06 2024-09-11T06:50:04.137 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.137 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.137 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:04.137 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212840+0000", 2024-09-11T06:50:04.137 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:50:04.137 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.137 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.137 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:04.137 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212854+0000", 2024-09-11T06:50:04.137 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4003e-05 2024-09-11T06:50:04.137 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.138 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.138 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:04.138 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213026+0000", 2024-09-11T06:50:04.138 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017207499999999999 2024-09-11T06:50:04.138 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.138 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.138 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:04.138 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213063+0000", 2024-09-11T06:50:04.138 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7187000000000002e-05 2024-09-11T06:50:04.138 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.138 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.138 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:04.138 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218762+0000", 2024-09-11T06:50:04.138 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0056993290000000004 2024-09-11T06:50:04.138 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.138 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.139 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:04.139 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218844+0000", 2024-09-11T06:50:04.139 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1701999999999998e-05 2024-09-11T06:50:04.139 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.139 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:04.139 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.139 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.139 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.139 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:60 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'38, mlcod=13'36)", 2024-09-11T06:50:04.139 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:50:04.139 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 57.895461617999999, 2024-09-11T06:50:04.139 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039254270000000004, 2024-09-11T06:50:04.139 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:04.140 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:04.140 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:04.140 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:04.140 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.140 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:04.140 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:50:04.140 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.140 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.140 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.140 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:04.140 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:50:04.140 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.140 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.141 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.141 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:04.141 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226756+0000", 2024-09-11T06:50:04.141 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-11T06:50:04.141 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.141 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.141 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:04.141 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226768+0000", 2024-09-11T06:50:04.141 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1919999999999999e-05 2024-09-11T06:50:04.141 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.141 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.141 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:04.141 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226797+0000", 2024-09-11T06:50:04.142 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9573e-05 2024-09-11T06:50:04.142 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.142 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.142 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:04.142 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226818+0000", 2024-09-11T06:50:04.142 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0432999999999999e-05 2024-09-11T06:50:04.142 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.142 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.142 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:04.142 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226909+0000", 2024-09-11T06:50:04.142 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0920000000000004e-05 2024-09-11T06:50:04.142 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.142 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.142 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:04.142 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226936+0000", 2024-09-11T06:50:04.143 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7472999999999999e-05 2024-09-11T06:50:04.143 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.143 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.143 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:04.143 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230596+0000", 2024-09-11T06:50:04.143 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0036602520000000001 2024-09-11T06:50:04.143 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.143 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.143 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:04.143 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230680+0000", 2024-09-11T06:50:04.143 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3106e-05 2024-09-11T06:50:04.143 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.143 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:04.143 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.143 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.144 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.144 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:107 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'69, mlcod=13'64)", 2024-09-11T06:50:04.144 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:50:04.144 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 57.679014160999998, 2024-09-11T06:50:04.144 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040728830000000002, 2024-09-11T06:50:04.144 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:04.144 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:04.144 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:04.144 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:04.144 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.144 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:04.144 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:50:04.144 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.145 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.145 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.145 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:04.145 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:50:04.145 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.145 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.145 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.145 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:04.145 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443203+0000", 2024-09-11T06:50:04.145 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1480000000000001e-06 2024-09-11T06:50:04.145 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.145 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.145 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:04.145 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443222+0000", 2024-09-11T06:50:04.145 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9040000000000001e-05 2024-09-11T06:50:04.146 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.146 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.146 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:04.146 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443246+0000", 2024-09-11T06:50:04.146 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4606000000000001e-05 2024-09-11T06:50:04.146 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.146 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.146 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:04.146 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443277+0000", 2024-09-11T06:50:04.146 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0207999999999999e-05 2024-09-11T06:50:04.146 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.146 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.146 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:04.146 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443340+0000", 2024-09-11T06:50:04.147 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3602000000000005e-05 2024-09-11T06:50:04.147 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.147 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.147 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:04.147 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443363+0000", 2024-09-11T06:50:04.147 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2331e-05 2024-09-11T06:50:04.147 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.147 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.147 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:04.147 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447216+0000", 2024-09-11T06:50:04.147 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0038539690000000001 2024-09-11T06:50:04.147 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.147 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.147 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:04.148 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447274+0000", 2024-09-11T06:50:04.148 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7979000000000002e-05 2024-09-11T06:50:04.148 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.148 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:04.148 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.148 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.148 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:111 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'73, mlcod=13'67)", 2024-09-11T06:50:04.148 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:50:04.148 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 57.672502530999999, 2024-09-11T06:50:04.148 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034372360000000002, 2024-09-11T06:50:04.148 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:04.148 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:04.148 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:04.148 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:04.149 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.149 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:04.149 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:50:04.149 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.149 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.149 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.149 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:04.149 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:50:04.149 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.149 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.149 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.149 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:04.149 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449714+0000", 2024-09-11T06:50:04.150 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5000000000000001e-07 2024-09-11T06:50:04.150 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.150 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.150 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:04.150 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449721+0000", 2024-09-11T06:50:04.150 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8530000000000003e-06 2024-09-11T06:50:04.150 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.150 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.150 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:04.150 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449736+0000", 2024-09-11T06:50:04.150 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5114e-05 2024-09-11T06:50:04.150 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.150 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.151 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:04.151 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449743+0000", 2024-09-11T06:50:04.151 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0210000000000002e-06 2024-09-11T06:50:04.151 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.151 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.151 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:04.151 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449804+0000", 2024-09-11T06:50:04.151 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0479999999999997e-05 2024-09-11T06:50:04.151 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.151 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.151 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:04.151 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449826+0000", 2024-09-11T06:50:04.152 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1922000000000001e-05 2024-09-11T06:50:04.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.152 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.152 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:04.152 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453104+0000", 2024-09-11T06:50:04.152 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003278652 2024-09-11T06:50:04.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.152 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.152 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:04.152 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453150+0000", 2024-09-11T06:50:04.152 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6244e-05 2024-09-11T06:50:04.152 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.152 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:04.152 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.153 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.153 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:114 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'76, mlcod=13'71)", 2024-09-11T06:50:04.153 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:50:04.153 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 57.667824459999999, 2024-09-11T06:50:04.153 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0042576660000000002, 2024-09-11T06:50:04.153 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:04.153 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:04.153 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:04.153 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:04.153 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.153 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:04.153 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:50:04.153 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.154 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.154 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.154 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:04.154 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:50:04.154 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.154 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.154 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.154 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:04.154 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454392+0000", 2024-09-11T06:50:04.154 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5999999999999991e-07 2024-09-11T06:50:04.154 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.154 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.154 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:04.154 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454399+0000", 2024-09-11T06:50:04.155 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.878e-06 2024-09-11T06:50:04.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.155 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.155 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:04.155 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454415+0000", 2024-09-11T06:50:04.155 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5659999999999999e-05 2024-09-11T06:50:04.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.155 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.155 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:04.155 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454422+0000", 2024-09-11T06:50:04.155 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:50:04.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.155 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.156 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:04.156 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454474+0000", 2024-09-11T06:50:04.156 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1860999999999997e-05 2024-09-11T06:50:04.156 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.156 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.156 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:04.156 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454497+0000", 2024-09-11T06:50:04.156 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3686e-05 2024-09-11T06:50:04.156 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.156 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.156 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:04.156 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458552+0000", 2024-09-11T06:50:04.156 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040544550000000002 2024-09-11T06:50:04.156 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.157 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.157 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:04.157 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458649+0000", 2024-09-11T06:50:04.157 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7005999999999999e-05 2024-09-11T06:50:04.157 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.157 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:04.157 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.157 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.157 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:115 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'77, mlcod=13'72)", 2024-09-11T06:50:04.157 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:04.157 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 57.666483847999999, 2024-09-11T06:50:04.157 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003355584, 2024-09-11T06:50:04.158 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:04.158 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:04.158 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:04.158 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:04.158 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.158 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:04.158 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:04.158 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.159 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:04.159 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:04.159 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.159 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:04.159 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:04.159 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5999999999999999e-07 2024-09-11T06:50:04.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.159 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:04.160 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455803+0000", 2024-09-11T06:50:04.160 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0310999999999998e-05 2024-09-11T06:50:04.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.160 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.160 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:04.160 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455815+0000", 2024-09-11T06:50:04.160 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2787e-05 2024-09-11T06:50:04.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.160 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.160 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:04.160 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455823+0000", 2024-09-11T06:50:04.160 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5959999999999998e-06 2024-09-11T06:50:04.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.161 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.161 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:04.161 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455875+0000", 2024-09-11T06:50:04.161 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2054e-05 2024-09-11T06:50:04.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.161 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.161 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:04.161 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455896+0000", 2024-09-11T06:50:04.161 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0554999999999999e-05 2024-09-11T06:50:04.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.161 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.161 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:04.161 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459000+0000", 2024-09-11T06:50:04.162 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003104842 2024-09-11T06:50:04.162 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.162 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.162 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:04.162 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459087+0000", 2024-09-11T06:50:04.162 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6978999999999995e-05 2024-09-11T06:50:04.162 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.162 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:04.162 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.162 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.162 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.162 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:120 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'82, mlcod=13'78)", 2024-09-11T06:50:04.162 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:50:04.162 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 57.655806562999999, 2024-09-11T06:50:04.163 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0064612549999999999, 2024-09-11T06:50:04.163 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:04.163 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:04.163 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:04.163 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:04.163 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.163 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:04.163 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:50:04.163 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.163 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.163 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.163 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:04.163 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:50:04.164 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.164 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.164 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.164 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:04.164 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466410+0000", 2024-09-11T06:50:04.164 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0389999999999999e-06 2024-09-11T06:50:04.164 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.164 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.164 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:04.164 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466496+0000", 2024-09-11T06:50:04.164 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5519000000000006e-05 2024-09-11T06:50:04.164 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.164 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.165 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:04.165 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466510+0000", 2024-09-11T06:50:04.165 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3842999999999999e-05 2024-09-11T06:50:04.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.165 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.165 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:04.165 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466517+0000", 2024-09-11T06:50:04.165 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5229999999999998e-06 2024-09-11T06:50:04.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.165 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.165 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:04.165 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466595+0000", 2024-09-11T06:50:04.165 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8019e-05 2024-09-11T06:50:04.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.166 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.166 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:04.166 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466612+0000", 2024-09-11T06:50:04.166 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7013000000000001e-05 2024-09-11T06:50:04.166 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.166 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.166 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:04.166 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472787+0000", 2024-09-11T06:50:04.166 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0061750269999999996 2024-09-11T06:50:04.166 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.166 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.166 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:04.166 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472870+0000", 2024-09-11T06:50:04.166 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3271999999999998e-05 2024-09-11T06:50:04.166 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.167 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:04.167 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.167 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.167 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.167 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:121 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'83, mlcod=13'78)", 2024-09-11T06:50:04.167 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:50:04.167 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 57.654581767000003, 2024-09-11T06:50:04.167 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055613069999999997, 2024-09-11T06:50:04.167 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:04.167 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:04.167 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:04.167 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:04.167 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.167 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:04.167 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:50:04.168 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.168 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.168 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:04.168 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:50:04.168 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.168 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.168 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:04.168 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467635+0000", 2024-09-11T06:50:04.168 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6899999999999996e-07 2024-09-11T06:50:04.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.168 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.169 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:04.169 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467727+0000", 2024-09-11T06:50:04.169 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1800999999999997e-05 2024-09-11T06:50:04.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.169 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.169 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:04.169 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467740+0000", 2024-09-11T06:50:04.169 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3747999999999999e-05 2024-09-11T06:50:04.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.169 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.169 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:04.169 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467753+0000", 2024-09-11T06:50:04.169 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2807e-05 2024-09-11T06:50:04.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.170 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.170 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:04.170 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467812+0000", 2024-09-11T06:50:04.170 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8381999999999998e-05 2024-09-11T06:50:04.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.170 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.170 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:04.170 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467828+0000", 2024-09-11T06:50:04.170 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6728e-05 2024-09-11T06:50:04.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.170 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.170 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:04.170 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473112+0000", 2024-09-11T06:50:04.170 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052833619999999998 2024-09-11T06:50:04.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.171 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.171 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:04.171 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473195+0000", 2024-09-11T06:50:04.171 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3609999999999994e-05 2024-09-11T06:50:04.171 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.171 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:04.171 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.171 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.171 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:131 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'89, mlcod=13'85)", 2024-09-11T06:50:04.171 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:50:04.171 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 57.303176051999998, 2024-09-11T06:50:04.172 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052492260000000001, 2024-09-11T06:50:04.172 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:04.172 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:04.172 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:04.172 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:04.172 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.172 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:04.172 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:50:04.172 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.172 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.172 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.172 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:04.172 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:50:04.172 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.173 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.173 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.173 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:04.173 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819041+0000", 2024-09-11T06:50:04.173 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4360000000000001e-06 2024-09-11T06:50:04.173 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.173 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.173 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:04.173 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819100+0000", 2024-09-11T06:50:04.173 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9225999999999998e-05 2024-09-11T06:50:04.173 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.173 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.173 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:04.174 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819128+0000", 2024-09-11T06:50:04.174 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.779e-05 2024-09-11T06:50:04.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.174 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.174 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:04.174 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819142+0000", 2024-09-11T06:50:04.174 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3573e-05 2024-09-11T06:50:04.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.174 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.174 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:04.174 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819263+0000", 2024-09-11T06:50:04.174 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000121086 2024-09-11T06:50:04.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.174 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.175 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:04.175 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819292+0000", 2024-09-11T06:50:04.175 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9363e-05 2024-09-11T06:50:04.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.175 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.175 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:04.175 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824185+0000", 2024-09-11T06:50:04.175 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0048924620000000002 2024-09-11T06:50:04.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.175 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.175 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:04.175 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824289+0000", 2024-09-11T06:50:04.175 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010429000000000001 2024-09-11T06:50:04.175 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.176 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:04.176 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.176 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.176 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:132 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'90, mlcod=13'85)", 2024-09-11T06:50:04.176 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:50:04.176 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 57.302401361999998, 2024-09-11T06:50:04.176 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.004532342, 2024-09-11T06:50:04.176 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:04.176 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:04.176 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:04.176 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:04.176 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.176 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:04.177 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:50:04.177 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.177 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.177 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:04.177 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:50:04.177 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.177 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.177 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:04.177 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819816+0000", 2024-09-11T06:50:04.177 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.61e-06 2024-09-11T06:50:04.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.178 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:04.178 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819827+0000", 2024-09-11T06:50:04.178 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1e-05 2024-09-11T06:50:04.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.178 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:04.178 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819851+0000", 2024-09-11T06:50:04.178 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4267e-05 2024-09-11T06:50:04.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.178 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:04.178 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819864+0000", 2024-09-11T06:50:04.178 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2767e-05 2024-09-11T06:50:04.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.179 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.179 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:04.179 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820071+0000", 2024-09-11T06:50:04.179 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000207061 2024-09-11T06:50:04.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.179 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.179 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:04.179 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820103+0000", 2024-09-11T06:50:04.179 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2326999999999997e-05 2024-09-11T06:50:04.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.179 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.179 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:04.179 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824302+0000", 2024-09-11T06:50:04.179 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0041986000000000002 2024-09-11T06:50:04.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.180 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.180 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:04.180 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824347+0000", 2024-09-11T06:50:04.180 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4709999999999997e-05 2024-09-11T06:50:04.180 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.180 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:04.180 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.180 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.180 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:133 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'91, mlcod=13'86)", 2024-09-11T06:50:04.180 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:50:04.180 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 57.301769702999998, 2024-09-11T06:50:04.181 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039421459999999997, 2024-09-11T06:50:04.181 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:04.181 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:04.181 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:04.181 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:04.181 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.181 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:04.181 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:50:04.181 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.181 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.181 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.181 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:04.181 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:50:04.181 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.182 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:04.182 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820448+0000", 2024-09-11T06:50:04.182 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.84e-06 2024-09-11T06:50:04.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.182 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:04.182 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820463+0000", 2024-09-11T06:50:04.182 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5356e-05 2024-09-11T06:50:04.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.182 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:04.183 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820499+0000", 2024-09-11T06:50:04.183 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.625e-05 2024-09-11T06:50:04.183 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.183 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.183 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:04.183 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820546+0000", 2024-09-11T06:50:04.183 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7017000000000001e-05 2024-09-11T06:50:04.183 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.183 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.183 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:04.183 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820882+0000", 2024-09-11T06:50:04.183 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033512 2024-09-11T06:50:04.183 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.184 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.184 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:04.184 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820941+0000", 2024-09-11T06:50:04.184 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8922999999999998e-05 2024-09-11T06:50:04.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.184 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.184 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:04.184 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824351+0000", 2024-09-11T06:50:04.184 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034101940000000001 2024-09-11T06:50:04.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.184 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.184 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:04.185 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824388+0000", 2024-09-11T06:50:04.185 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7446e-05 2024-09-11T06:50:04.185 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.185 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:04.185 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.185 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.185 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:134 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'92, mlcod=13'86)", 2024-09-11T06:50:04.185 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:50:04.185 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 57.301110585000004, 2024-09-11T06:50:04.185 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033248179999999998, 2024-09-11T06:50:04.185 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:04.185 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:04.186 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:04.186 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:04.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.186 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:04.186 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:50:04.186 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.186 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:04.186 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:50:04.186 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.186 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:04.187 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821107+0000", 2024-09-11T06:50:04.187 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7430000000000001e-06 2024-09-11T06:50:04.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.187 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.187 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:04.187 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821205+0000", 2024-09-11T06:50:04.187 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7666999999999998e-05 2024-09-11T06:50:04.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.187 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.187 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:04.187 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821234+0000", 2024-09-11T06:50:04.187 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9756000000000001e-05 2024-09-11T06:50:04.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.188 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.188 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:04.188 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821254+0000", 2024-09-11T06:50:04.188 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9446e-05 2024-09-11T06:50:04.188 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.188 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.188 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:04.188 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821846+0000", 2024-09-11T06:50:04.188 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00059187599999999997 2024-09-11T06:50:04.188 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.188 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.188 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:04.188 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821894+0000", 2024-09-11T06:50:04.189 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8766e-05 2024-09-11T06:50:04.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.189 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.189 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:04.189 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824392+0000", 2024-09-11T06:50:04.189 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0024977509999999999 2024-09-11T06:50:04.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.189 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.189 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:04.189 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824430+0000", 2024-09-11T06:50:04.189 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7812999999999999e-05 2024-09-11T06:50:04.189 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.189 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:04.189 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.190 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.190 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:137 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'95, mlcod=13'86)", 2024-09-11T06:50:04.190 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:50:04.190 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 57.298648444000001, 2024-09-11T06:50:04.190 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003084299, 2024-09-11T06:50:04.190 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:04.190 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:04.190 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:04.190 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:04.190 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.190 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:04.190 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:50:04.191 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.191 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:04.191 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:50:04.191 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.191 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:04.191 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823568+0000", 2024-09-11T06:50:04.191 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1030000000000001e-06 2024-09-11T06:50:04.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.191 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:04.192 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823662+0000", 2024-09-11T06:50:04.192 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3529999999999994e-05 2024-09-11T06:50:04.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.192 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:04.192 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823686+0000", 2024-09-11T06:50:04.192 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4358999999999998e-05 2024-09-11T06:50:04.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.192 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:04.192 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823701+0000", 2024-09-11T06:50:04.192 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4847e-05 2024-09-11T06:50:04.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.193 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.193 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:04.193 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824518+0000", 2024-09-11T06:50:04.193 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00081716700000000005 2024-09-11T06:50:04.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.193 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.193 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:04.193 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824546+0000", 2024-09-11T06:50:04.193 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7860000000000001e-05 2024-09-11T06:50:04.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.193 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.193 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:04.193 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826558+0000", 2024-09-11T06:50:04.194 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0020120009999999998 2024-09-11T06:50:04.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.194 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.194 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:04.194 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826652+0000", 2024-09-11T06:50:04.194 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3431999999999996e-05 2024-09-11T06:50:04.194 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.194 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:04.194 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.194 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.194 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:139 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'97, mlcod=13'86)", 2024-09-11T06:50:04.194 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:50:04.194 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 57.296359725000002, 2024-09-11T06:50:04.195 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003732221, 2024-09-11T06:50:04.195 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:04.195 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:04.195 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:04.195 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:04.195 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.195 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:04.195 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:50:04.195 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.195 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.195 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.195 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:04.195 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:50:04.196 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.196 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.196 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:04.196 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825858+0000", 2024-09-11T06:50:04.196 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5829999999999999e-06 2024-09-11T06:50:04.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.196 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.196 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:04.196 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825874+0000", 2024-09-11T06:50:04.196 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5967000000000001e-05 2024-09-11T06:50:04.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.197 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:04.197 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825909+0000", 2024-09-11T06:50:04.197 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5692999999999999e-05 2024-09-11T06:50:04.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.197 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:04.197 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825923+0000", 2024-09-11T06:50:04.197 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4080000000000001e-05 2024-09-11T06:50:04.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.197 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:04.197 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826010+0000", 2024-09-11T06:50:04.197 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6248999999999994e-05 2024-09-11T06:50:04.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:04.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826038+0000", 2024-09-11T06:50:04.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8010000000000001e-05 2024-09-11T06:50:04.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:04.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829499+0000", 2024-09-11T06:50:04.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003461783 2024-09-11T06:50:04.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:04.199 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829588+0000", 2024-09-11T06:50:04.199 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8856000000000004e-05 2024-09-11T06:50:04.199 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.199 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:04.199 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.199 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:140 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'98, mlcod=13'93)", 2024-09-11T06:50:04.199 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:50:04.199 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 57.294508995999998, 2024-09-11T06:50:04.199 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0057995929999999996, 2024-09-11T06:50:04.199 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:04.199 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:04.199 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:04.200 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:04.200 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.200 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:04.200 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:50:04.200 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.200 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.200 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:04.200 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:50:04.200 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.200 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.200 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:04.201 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827708+0000", 2024-09-11T06:50:04.201 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.547e-06 2024-09-11T06:50:04.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.201 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.201 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:04.201 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827765+0000", 2024-09-11T06:50:04.201 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6266e-05 2024-09-11T06:50:04.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.201 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.201 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:04.201 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827791+0000", 2024-09-11T06:50:04.201 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6098999999999999e-05 2024-09-11T06:50:04.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.201 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.202 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:04.202 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827806+0000", 2024-09-11T06:50:04.202 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5537e-05 2024-09-11T06:50:04.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.202 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.202 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:04.202 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827928+0000", 2024-09-11T06:50:04.202 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000122012 2024-09-11T06:50:04.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.202 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.202 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:04.202 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827961+0000", 2024-09-11T06:50:04.202 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2663999999999998e-05 2024-09-11T06:50:04.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.203 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:04.203 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833409+0000", 2024-09-11T06:50:04.203 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054479350000000001 2024-09-11T06:50:04.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.203 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:04.203 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833506+0000", 2024-09-11T06:50:04.203 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7533000000000003e-05 2024-09-11T06:50:04.203 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.203 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:04.203 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.204 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:141 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'99, mlcod=13'93)", 2024-09-11T06:50:04.204 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:50:04.204 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 57.293615613, 2024-09-11T06:50:04.204 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055394709999999998, 2024-09-11T06:50:04.204 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:04.204 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:04.204 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:04.204 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:04.204 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.204 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:04.204 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:50:04.204 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.205 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:04.205 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:50:04.205 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.205 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:04.205 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828602+0000", 2024-09-11T06:50:04.205 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.46e-06 2024-09-11T06:50:04.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.205 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:04.205 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828618+0000", 2024-09-11T06:50:04.205 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6592999999999999e-05 2024-09-11T06:50:04.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.206 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.206 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:04.206 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828646+0000", 2024-09-11T06:50:04.206 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7552999999999999e-05 2024-09-11T06:50:04.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.206 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.206 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:04.206 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828663+0000", 2024-09-11T06:50:04.206 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.719e-05 2024-09-11T06:50:04.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.206 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.206 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:04.206 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829023+0000", 2024-09-11T06:50:04.207 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036054799999999998 2024-09-11T06:50:04.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.207 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:04.207 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829062+0000", 2024-09-11T06:50:04.207 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8865999999999997e-05 2024-09-11T06:50:04.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.207 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:04.207 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834078+0000", 2024-09-11T06:50:04.207 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050157480000000004 2024-09-11T06:50:04.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.207 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:04.208 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834140+0000", 2024-09-11T06:50:04.208 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1513000000000005e-05 2024-09-11T06:50:04.208 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.208 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:04.208 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.208 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.208 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:144 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'102, mlcod=13'98)", 2024-09-11T06:50:04.208 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:50:04.208 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 57.284455000000001, 2024-09-11T06:50:04.208 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003210348, 2024-09-11T06:50:04.208 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:04.208 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:04.208 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:04.209 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:04.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:04.209 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:50:04.209 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:04.209 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:50:04.209 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:04.210 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837768+0000", 2024-09-11T06:50:04.210 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9330000000000004e-06 2024-09-11T06:50:04.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.210 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.210 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:04.210 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837858+0000", 2024-09-11T06:50:04.210 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0185999999999993e-05 2024-09-11T06:50:04.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.210 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.210 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:04.210 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837890+0000", 2024-09-11T06:50:04.210 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2437000000000001e-05 2024-09-11T06:50:04.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.210 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.211 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:04.211 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837905+0000", 2024-09-11T06:50:04.211 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:50:04.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.211 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.211 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:04.211 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838019+0000", 2024-09-11T06:50:04.211 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000114336 2024-09-11T06:50:04.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.211 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.211 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:04.211 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838052+0000", 2024-09-11T06:50:04.211 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3139000000000002e-05 2024-09-11T06:50:04.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.212 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:04.212 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840914+0000", 2024-09-11T06:50:04.212 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028612780000000001 2024-09-11T06:50:04.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.212 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:04.212 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840971+0000", 2024-09-11T06:50:04.212 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7402999999999999e-05 2024-09-11T06:50:04.212 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.212 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:04.212 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.213 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:145 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'103, mlcod=13'98)", 2024-09-11T06:50:04.213 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:50:04.213 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 57.283663116, 2024-09-11T06:50:04.213 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054455409999999999, 2024-09-11T06:50:04.213 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:04.213 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:04.213 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:04.213 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:04.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.213 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:04.213 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:50:04.213 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.214 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:04.214 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:50:04.214 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:04.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.214 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.214 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:04.214 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838554+0000", 2024-09-11T06:50:04.214 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3430000000000001e-06 2024-09-11T06:50:04.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.214 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.214 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:04.214 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838640+0000", 2024-09-11T06:50:04.214 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5569999999999996e-05 2024-09-11T06:50:04.215 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.215 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.215 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:04.215 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838660+0000", 2024-09-11T06:50:04.215 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0576000000000001e-05 2024-09-11T06:50:04.215 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.215 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.215 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:04.215 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838674+0000", 2024-09-11T06:50:04.215 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3529999999999999e-05 2024-09-11T06:50:04.215 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.215 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.215 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:04.215 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838838+0000", 2024-09-11T06:50:04.216 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016468799999999999 2024-09-11T06:50:04.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.216 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:04.216 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838885+0000", 2024-09-11T06:50:04.216 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6884e-05 2024-09-11T06:50:04.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.216 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:04.216 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843901+0000", 2024-09-11T06:50:04.216 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050162310000000003 2024-09-11T06:50:04.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:04.217 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:04.217 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:04.217 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843998+0000", 2024-09-11T06:50:04.217 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.6718999999999993e-05 2024-09-11T06:50:04.217 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.217 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:04.217 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.217 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:04.217 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:04.217 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:04.219 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:50:04.219 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:04.217+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:04.320 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:04.316+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:04.378 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:04.378 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:04.378 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:50:04.378 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:04.393 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:50:04.437 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:04.436+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:04.521 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:50:04.522 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:04.521+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:04.557 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:04.557 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:04.557 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:50:04.557 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:50:04.557 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:04.571 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:50:04.622 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:04.621+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:04.653 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:50:04.653 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:50:04.673 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:50:04.673 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:50:04.687 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:50:04.687 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:50:04.708 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:50:04.708 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:50:04.724 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:04.725+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:04.728 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:50:04.729 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:50:04.730 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:04.730 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:04.744 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:50:04.749 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:50:04.749 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:50:04.824 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:04.824+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:04.869 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:04.869 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:04.883 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:50:04.926 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:04.924+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:05.005 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is unset 2024-09-11T06:50:05.027 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:05.025+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:05.059 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:05.059 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:05.073 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:50:05.128 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:05.129+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:05.229 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:05.229+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:05.235 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:05.235 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:05.249 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:50:05.285 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T06:50:05.286 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-11T06:50:05.329 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:05.329+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:05.383 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:05.383 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:05.397 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:50:05.439 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:05.441+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:05.531 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:05.533+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:05.561 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:05.561 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:05.575 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:50:05.663 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:05.665+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:05.733 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:05.732+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:05.744 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:05.744 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:05.759 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:50:05.855 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:05.857+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:05.932 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:05.932 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:05.935 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:05.932+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:05.946 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:50:06.021 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.0 to 1 (10000) 2024-09-11T06:50:06.035 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:06.032+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:06.045 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-11T06:50:06.107 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:06.107 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:06.121 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:50:06.167 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:06.168+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:06.238 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:06.236+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:06.244 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:06.245 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:06.259 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:50:06.339 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:06.336+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:06.423 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:06.424 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:06.437 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:50:06.439 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:06.440+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:06.541 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:06.540+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:06.559 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:06.560 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:06.574 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:50:06.659 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:06.660+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:06.743 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:06.740+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:06.748 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:06.748 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:06.762 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:50:06.769 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:50:06.843 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:06.844+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:06.896 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:06.896 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:06.911 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:50:06.923 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.2 to 1 (10000) 2024-09-11T06:50:06.945 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:06.944+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:06.948 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-11T06:50:06.953 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:50:06.953 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:50:06.971 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:50:06.972 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:50:06.990 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:50:06.990 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:50:07.010 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:50:07.010 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:50:07.031 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:50:07.032 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:50:07.032 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T06:50:07.034 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:07.035 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:07.035 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:50:07.035 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:50:07.035 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:07.051 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:50:07.052 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:07.048+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:07.053 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:50:07.054 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:50:07.146 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:07.144+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:07.173 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:07.173 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:50:07.174 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:50:07.174 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:50:07.174 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.174 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:54 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'32, mlcod=13'29)", 2024-09-11T06:50:07.174 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:50:07.174 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 60.961898669, 2024-09-11T06:50:07.174 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034271670000000001, 2024-09-11T06:50:07.174 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:07.174 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:07.174 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:07.174 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:07.174 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.175 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:07.175 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:50:07.175 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.175 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.175 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:07.175 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:50:07.175 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.175 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.175 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:07.175 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210361+0000", 2024-09-11T06:50:07.175 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1260000000000001e-06 2024-09-11T06:50:07.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.176 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.176 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:07.176 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210370+0000", 2024-09-11T06:50:07.176 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8200000000000003e-06 2024-09-11T06:50:07.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.176 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.176 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:07.176 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210390+0000", 2024-09-11T06:50:07.176 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0126e-05 2024-09-11T06:50:07.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.176 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.177 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:07.177 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210404+0000", 2024-09-11T06:50:07.177 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4589999999999999e-05 2024-09-11T06:50:07.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.177 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.177 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:07.177 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210538+0000", 2024-09-11T06:50:07.177 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000133719 2024-09-11T06:50:07.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.177 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.177 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:07.177 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210593+0000", 2024-09-11T06:50:07.177 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5016999999999999e-05 2024-09-11T06:50:07.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.178 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:07.178 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213695+0000", 2024-09-11T06:50:07.178 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003101866 2024-09-11T06:50:07.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.178 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:07.178 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213787+0000", 2024-09-11T06:50:07.178 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1903000000000004e-05 2024-09-11T06:50:07.178 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.178 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:07.178 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.179 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.179 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:55 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'33, mlcod=13'29)", 2024-09-11T06:50:07.179 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:50:07.179 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 60.960879431000002, 2024-09-11T06:50:07.179 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.005086091, 2024-09-11T06:50:07.179 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:07.179 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:07.179 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:07.179 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:07.179 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.179 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:07.179 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:50:07.179 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.180 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.180 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:07.180 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:50:07.180 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.180 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.180 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:07.180 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211380+0000", 2024-09-11T06:50:07.180 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.097e-06 2024-09-11T06:50:07.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.180 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.180 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:07.180 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211537+0000", 2024-09-11T06:50:07.181 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000157369 2024-09-11T06:50:07.181 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.181 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.181 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:07.181 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211559+0000", 2024-09-11T06:50:07.181 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000001e-05 2024-09-11T06:50:07.181 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.181 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.181 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:07.181 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211573+0000", 2024-09-11T06:50:07.181 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4946000000000001e-05 2024-09-11T06:50:07.181 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.181 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.182 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:07.182 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211676+0000", 2024-09-11T06:50:07.182 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0001026 2024-09-11T06:50:07.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.182 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:07.182 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211709+0000", 2024-09-11T06:50:07.182 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3071999999999998e-05 2024-09-11T06:50:07.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.182 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:07.182 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216379+0000", 2024-09-11T06:50:07.182 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0046693780000000001 2024-09-11T06:50:07.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.183 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.183 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:07.183 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216465+0000", 2024-09-11T06:50:07.183 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6469000000000002e-05 2024-09-11T06:50:07.183 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.183 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:07.183 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.183 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.183 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.183 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:56 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'34, mlcod=13'29)", 2024-09-11T06:50:07.183 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:50:07.183 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 60.959445572, 2024-09-11T06:50:07.183 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0060310629999999997, 2024-09-11T06:50:07.183 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:07.183 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:07.184 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:07.184 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:07.184 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.184 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:07.184 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:50:07.184 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.184 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.184 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:07.184 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:50:07.184 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.184 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.184 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:07.185 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212814+0000", 2024-09-11T06:50:07.185 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0869999999999999e-06 2024-09-11T06:50:07.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.185 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.185 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:07.185 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212822+0000", 2024-09-11T06:50:07.185 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0870000000000007e-06 2024-09-11T06:50:07.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.185 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.185 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:07.185 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212840+0000", 2024-09-11T06:50:07.185 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:50:07.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.185 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.185 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:07.186 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212854+0000", 2024-09-11T06:50:07.186 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4003e-05 2024-09-11T06:50:07.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.186 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:07.186 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213026+0000", 2024-09-11T06:50:07.186 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017207499999999999 2024-09-11T06:50:07.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.186 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:07.186 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213063+0000", 2024-09-11T06:50:07.186 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7187000000000002e-05 2024-09-11T06:50:07.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.187 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:07.187 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218762+0000", 2024-09-11T06:50:07.187 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0056993290000000004 2024-09-11T06:50:07.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.187 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.187 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:07.187 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218844+0000", 2024-09-11T06:50:07.187 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1701999999999998e-05 2024-09-11T06:50:07.187 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.187 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:07.187 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.187 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.187 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:60 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'38, mlcod=13'36)", 2024-09-11T06:50:07.188 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:50:07.188 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 60.945504223, 2024-09-11T06:50:07.188 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039254270000000004, 2024-09-11T06:50:07.188 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:07.188 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:07.188 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:07.188 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:07.188 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.188 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:07.188 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:50:07.188 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.188 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.188 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.189 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:07.189 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:50:07.189 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.189 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.189 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:07.189 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226756+0000", 2024-09-11T06:50:07.189 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-11T06:50:07.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.189 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.189 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:07.189 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226768+0000", 2024-09-11T06:50:07.189 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1919999999999999e-05 2024-09-11T06:50:07.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.189 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.190 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:07.190 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226797+0000", 2024-09-11T06:50:07.190 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9573e-05 2024-09-11T06:50:07.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.190 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.190 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:07.190 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226818+0000", 2024-09-11T06:50:07.190 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0432999999999999e-05 2024-09-11T06:50:07.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.190 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.190 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:07.190 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226909+0000", 2024-09-11T06:50:07.190 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0920000000000004e-05 2024-09-11T06:50:07.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.191 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:07.191 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226936+0000", 2024-09-11T06:50:07.191 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7472999999999999e-05 2024-09-11T06:50:07.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.191 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:07.191 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230596+0000", 2024-09-11T06:50:07.191 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0036602520000000001 2024-09-11T06:50:07.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.191 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:07.192 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230680+0000", 2024-09-11T06:50:07.192 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3106e-05 2024-09-11T06:50:07.192 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.192 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:07.192 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.192 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:107 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'69, mlcod=13'64)", 2024-09-11T06:50:07.192 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:50:07.192 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 60.729056765999999, 2024-09-11T06:50:07.192 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040728830000000002, 2024-09-11T06:50:07.192 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:07.192 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:07.192 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:07.193 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:07.193 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.193 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:07.193 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:50:07.193 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.193 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.193 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:07.193 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:50:07.193 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.193 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.193 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:07.194 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443203+0000", 2024-09-11T06:50:07.194 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1480000000000001e-06 2024-09-11T06:50:07.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.194 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.194 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:07.194 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443222+0000", 2024-09-11T06:50:07.194 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9040000000000001e-05 2024-09-11T06:50:07.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.194 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.194 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:07.194 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443246+0000", 2024-09-11T06:50:07.194 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4606000000000001e-05 2024-09-11T06:50:07.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.194 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.195 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:07.195 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443277+0000", 2024-09-11T06:50:07.195 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0207999999999999e-05 2024-09-11T06:50:07.195 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.195 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.195 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:07.195 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443340+0000", 2024-09-11T06:50:07.195 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3602000000000005e-05 2024-09-11T06:50:07.195 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.195 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.195 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:07.195 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443363+0000", 2024-09-11T06:50:07.195 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2331e-05 2024-09-11T06:50:07.195 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.195 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.196 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:07.196 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447216+0000", 2024-09-11T06:50:07.196 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0038539690000000001 2024-09-11T06:50:07.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.196 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.196 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:07.196 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447274+0000", 2024-09-11T06:50:07.196 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7979000000000002e-05 2024-09-11T06:50:07.196 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.196 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:07.196 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.196 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.196 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:111 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'73, mlcod=13'67)", 2024-09-11T06:50:07.197 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:50:07.197 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 60.722545136000001, 2024-09-11T06:50:07.197 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034372360000000002, 2024-09-11T06:50:07.197 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:07.197 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:07.197 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:07.197 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:07.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.197 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:07.197 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:50:07.197 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.197 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:07.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:50:07.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:07.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449714+0000", 2024-09-11T06:50:07.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5000000000000001e-07 2024-09-11T06:50:07.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:07.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449721+0000", 2024-09-11T06:50:07.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8530000000000003e-06 2024-09-11T06:50:07.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:07.199 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449736+0000", 2024-09-11T06:50:07.199 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5114e-05 2024-09-11T06:50:07.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.199 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:07.199 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449743+0000", 2024-09-11T06:50:07.199 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0210000000000002e-06 2024-09-11T06:50:07.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.199 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:07.199 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449804+0000", 2024-09-11T06:50:07.199 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0479999999999997e-05 2024-09-11T06:50:07.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.200 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:07.200 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449826+0000", 2024-09-11T06:50:07.200 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1922000000000001e-05 2024-09-11T06:50:07.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.200 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.200 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:07.200 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453104+0000", 2024-09-11T06:50:07.200 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003278652 2024-09-11T06:50:07.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.200 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.200 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:07.200 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453150+0000", 2024-09-11T06:50:07.200 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6244e-05 2024-09-11T06:50:07.200 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.201 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:07.201 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.201 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.201 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:114 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'76, mlcod=13'71)", 2024-09-11T06:50:07.201 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:50:07.201 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 60.717867065, 2024-09-11T06:50:07.201 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0042576660000000002, 2024-09-11T06:50:07.201 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:07.201 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:07.201 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:07.201 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:07.201 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.201 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:07.202 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:50:07.202 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.202 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.202 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:07.202 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:50:07.202 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.202 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.202 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:07.202 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454392+0000", 2024-09-11T06:50:07.202 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5999999999999991e-07 2024-09-11T06:50:07.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.203 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:07.203 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454399+0000", 2024-09-11T06:50:07.203 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.878e-06 2024-09-11T06:50:07.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.203 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:07.203 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454415+0000", 2024-09-11T06:50:07.203 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5659999999999999e-05 2024-09-11T06:50:07.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.203 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:07.203 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454422+0000", 2024-09-11T06:50:07.203 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:50:07.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.204 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.204 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:07.204 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454474+0000", 2024-09-11T06:50:07.204 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1860999999999997e-05 2024-09-11T06:50:07.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.204 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.204 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:07.204 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454497+0000", 2024-09-11T06:50:07.204 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3686e-05 2024-09-11T06:50:07.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.204 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.204 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:07.204 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458552+0000", 2024-09-11T06:50:07.205 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040544550000000002 2024-09-11T06:50:07.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.205 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:07.205 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458649+0000", 2024-09-11T06:50:07.205 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7005999999999999e-05 2024-09-11T06:50:07.205 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.205 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:07.205 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.205 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:115 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'77, mlcod=13'72)", 2024-09-11T06:50:07.205 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:07.205 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 60.716526453, 2024-09-11T06:50:07.206 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003355584, 2024-09-11T06:50:07.206 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:07.206 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:07.206 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:07.206 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:07.206 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.206 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:07.206 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:07.206 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.206 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.206 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:07.206 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:07.206 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.207 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:07.207 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:07.207 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5999999999999999e-07 2024-09-11T06:50:07.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.207 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:07.207 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455803+0000", 2024-09-11T06:50:07.207 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0310999999999998e-05 2024-09-11T06:50:07.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.207 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:07.207 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455815+0000", 2024-09-11T06:50:07.208 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2787e-05 2024-09-11T06:50:07.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.208 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.208 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:07.208 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455823+0000", 2024-09-11T06:50:07.208 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5959999999999998e-06 2024-09-11T06:50:07.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.208 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.208 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:07.208 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455875+0000", 2024-09-11T06:50:07.208 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2054e-05 2024-09-11T06:50:07.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.208 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.208 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:07.209 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455896+0000", 2024-09-11T06:50:07.209 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0554999999999999e-05 2024-09-11T06:50:07.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:07.209 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459000+0000", 2024-09-11T06:50:07.209 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003104842 2024-09-11T06:50:07.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:07.209 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459087+0000", 2024-09-11T06:50:07.209 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6978999999999995e-05 2024-09-11T06:50:07.209 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.209 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:07.210 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.210 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.210 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:120 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'82, mlcod=13'78)", 2024-09-11T06:50:07.210 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:50:07.210 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 60.705849168, 2024-09-11T06:50:07.210 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0064612549999999999, 2024-09-11T06:50:07.210 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:07.210 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:07.210 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:07.210 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:07.210 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.210 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:07.210 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:50:07.211 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.211 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.211 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:07.211 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:50:07.211 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.211 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.211 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:07.211 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466410+0000", 2024-09-11T06:50:07.211 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0389999999999999e-06 2024-09-11T06:50:07.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.211 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.211 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:07.212 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466496+0000", 2024-09-11T06:50:07.212 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5519000000000006e-05 2024-09-11T06:50:07.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.212 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:07.212 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466510+0000", 2024-09-11T06:50:07.212 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3842999999999999e-05 2024-09-11T06:50:07.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.212 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:07.212 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466517+0000", 2024-09-11T06:50:07.213 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5229999999999998e-06 2024-09-11T06:50:07.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.213 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:07.213 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466595+0000", 2024-09-11T06:50:07.213 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8019e-05 2024-09-11T06:50:07.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.213 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:07.213 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466612+0000", 2024-09-11T06:50:07.214 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7013000000000001e-05 2024-09-11T06:50:07.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.214 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.214 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:07.214 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472787+0000", 2024-09-11T06:50:07.214 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0061750269999999996 2024-09-11T06:50:07.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.214 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.214 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:07.214 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472870+0000", 2024-09-11T06:50:07.214 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3271999999999998e-05 2024-09-11T06:50:07.214 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.214 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:07.215 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.215 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.215 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.215 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:121 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'83, mlcod=13'78)", 2024-09-11T06:50:07.215 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:50:07.215 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 60.704624371999998, 2024-09-11T06:50:07.215 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055613069999999997, 2024-09-11T06:50:07.215 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:07.215 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:07.215 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:07.215 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:07.215 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.215 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:07.216 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:50:07.216 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.216 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:07.216 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:50:07.216 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.216 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:07.216 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467635+0000", 2024-09-11T06:50:07.216 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6899999999999996e-07 2024-09-11T06:50:07.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.217 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.217 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:07.217 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467727+0000", 2024-09-11T06:50:07.217 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1800999999999997e-05 2024-09-11T06:50:07.217 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.217 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.217 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:07.217 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467740+0000", 2024-09-11T06:50:07.217 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3747999999999999e-05 2024-09-11T06:50:07.217 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.217 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.217 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:07.217 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467753+0000", 2024-09-11T06:50:07.218 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2807e-05 2024-09-11T06:50:07.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.218 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:07.218 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467812+0000", 2024-09-11T06:50:07.218 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8381999999999998e-05 2024-09-11T06:50:07.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.218 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:07.218 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467828+0000", 2024-09-11T06:50:07.218 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6728e-05 2024-09-11T06:50:07.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.218 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:07.219 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473112+0000", 2024-09-11T06:50:07.219 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052833619999999998 2024-09-11T06:50:07.219 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.219 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.219 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:07.219 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473195+0000", 2024-09-11T06:50:07.219 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3609999999999994e-05 2024-09-11T06:50:07.219 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.219 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:07.219 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.219 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.219 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.219 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:131 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'89, mlcod=13'85)", 2024-09-11T06:50:07.219 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:50:07.220 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 60.353218656999999, 2024-09-11T06:50:07.220 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052492260000000001, 2024-09-11T06:50:07.220 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:07.220 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:07.220 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:07.220 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:07.220 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.220 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:07.220 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:50:07.220 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.220 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.220 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:07.220 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:50:07.221 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.221 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.221 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.221 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:07.221 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819041+0000", 2024-09-11T06:50:07.221 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4360000000000001e-06 2024-09-11T06:50:07.221 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.221 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.221 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:07.221 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819100+0000", 2024-09-11T06:50:07.221 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9225999999999998e-05 2024-09-11T06:50:07.221 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.221 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.221 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:07.222 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819128+0000", 2024-09-11T06:50:07.222 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.779e-05 2024-09-11T06:50:07.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.222 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:07.222 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819142+0000", 2024-09-11T06:50:07.222 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3573e-05 2024-09-11T06:50:07.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.222 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:07.222 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819263+0000", 2024-09-11T06:50:07.222 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000121086 2024-09-11T06:50:07.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.223 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:07.223 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819292+0000", 2024-09-11T06:50:07.223 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9363e-05 2024-09-11T06:50:07.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.223 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.223 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:07.223 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824185+0000", 2024-09-11T06:50:07.223 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0048924620000000002 2024-09-11T06:50:07.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.223 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.223 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:07.223 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824289+0000", 2024-09-11T06:50:07.223 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010429000000000001 2024-09-11T06:50:07.223 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.224 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:07.224 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.224 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.224 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:132 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'90, mlcod=13'85)", 2024-09-11T06:50:07.224 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:50:07.224 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 60.352443966999999, 2024-09-11T06:50:07.224 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.004532342, 2024-09-11T06:50:07.224 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:07.224 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:07.224 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:07.224 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:07.224 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.224 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:07.225 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:50:07.225 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.225 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:07.225 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:50:07.225 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.225 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:07.225 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819816+0000", 2024-09-11T06:50:07.225 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.61e-06 2024-09-11T06:50:07.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.226 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:07.226 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819827+0000", 2024-09-11T06:50:07.226 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1e-05 2024-09-11T06:50:07.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.226 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:07.226 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819851+0000", 2024-09-11T06:50:07.226 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4267e-05 2024-09-11T06:50:07.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.226 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:07.226 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819864+0000", 2024-09-11T06:50:07.226 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2767e-05 2024-09-11T06:50:07.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.227 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.227 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:07.227 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820071+0000", 2024-09-11T06:50:07.227 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000207061 2024-09-11T06:50:07.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.227 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.227 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:07.227 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820103+0000", 2024-09-11T06:50:07.227 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2326999999999997e-05 2024-09-11T06:50:07.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.227 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.227 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:07.227 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824302+0000", 2024-09-11T06:50:07.227 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0041986000000000002 2024-09-11T06:50:07.228 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.228 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.228 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:07.228 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824347+0000", 2024-09-11T06:50:07.228 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4709999999999997e-05 2024-09-11T06:50:07.228 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.228 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:07.228 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.228 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.228 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.228 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:133 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'91, mlcod=13'86)", 2024-09-11T06:50:07.228 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:50:07.229 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 60.351812308, 2024-09-11T06:50:07.229 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039421459999999997, 2024-09-11T06:50:07.229 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:07.229 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:07.229 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:07.229 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:07.229 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.229 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:07.229 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:50:07.229 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.229 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.229 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.229 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:07.229 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:50:07.229 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.230 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:07.230 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820448+0000", 2024-09-11T06:50:07.230 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.84e-06 2024-09-11T06:50:07.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.230 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:07.230 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820463+0000", 2024-09-11T06:50:07.230 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5356e-05 2024-09-11T06:50:07.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.230 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:07.230 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820499+0000", 2024-09-11T06:50:07.231 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.625e-05 2024-09-11T06:50:07.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.231 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.231 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:07.231 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820546+0000", 2024-09-11T06:50:07.231 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7017000000000001e-05 2024-09-11T06:50:07.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.231 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.231 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:07.231 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820882+0000", 2024-09-11T06:50:07.231 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033512 2024-09-11T06:50:07.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.231 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.232 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:07.232 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820941+0000", 2024-09-11T06:50:07.232 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8922999999999998e-05 2024-09-11T06:50:07.232 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.232 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.232 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:07.232 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824351+0000", 2024-09-11T06:50:07.232 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034101940000000001 2024-09-11T06:50:07.232 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.232 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.232 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:07.232 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824388+0000", 2024-09-11T06:50:07.232 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7446e-05 2024-09-11T06:50:07.232 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.233 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:07.233 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.233 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.233 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.233 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:134 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'92, mlcod=13'86)", 2024-09-11T06:50:07.233 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:50:07.233 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 60.351153189999998, 2024-09-11T06:50:07.233 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033248179999999998, 2024-09-11T06:50:07.233 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:07.233 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:07.233 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:07.233 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:07.233 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.234 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:07.234 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:50:07.234 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.234 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:07.234 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:50:07.234 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.234 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:07.234 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821107+0000", 2024-09-11T06:50:07.234 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7430000000000001e-06 2024-09-11T06:50:07.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.235 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:07.235 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821205+0000", 2024-09-11T06:50:07.235 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7666999999999998e-05 2024-09-11T06:50:07.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.235 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:07.235 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821234+0000", 2024-09-11T06:50:07.235 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9756000000000001e-05 2024-09-11T06:50:07.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.235 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:07.235 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821254+0000", 2024-09-11T06:50:07.235 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9446e-05 2024-09-11T06:50:07.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.236 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.236 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:07.236 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821846+0000", 2024-09-11T06:50:07.236 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00059187599999999997 2024-09-11T06:50:07.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.236 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.236 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:07.236 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821894+0000", 2024-09-11T06:50:07.236 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8766e-05 2024-09-11T06:50:07.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.236 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.236 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:07.236 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824392+0000", 2024-09-11T06:50:07.236 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0024977509999999999 2024-09-11T06:50:07.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.237 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.237 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:07.237 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824430+0000", 2024-09-11T06:50:07.237 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7812999999999999e-05 2024-09-11T06:50:07.237 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.237 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:07.237 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.237 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.237 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:137 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'95, mlcod=13'86)", 2024-09-11T06:50:07.237 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:50:07.237 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 60.348691049000003, 2024-09-11T06:50:07.237 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003084299, 2024-09-11T06:50:07.238 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:07.238 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:07.238 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:07.238 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:07.238 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.238 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:07.238 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:50:07.238 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.238 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.238 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:07.238 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:50:07.238 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.239 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.239 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:07.239 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823568+0000", 2024-09-11T06:50:07.239 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1030000000000001e-06 2024-09-11T06:50:07.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.239 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.239 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:07.239 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823662+0000", 2024-09-11T06:50:07.239 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3529999999999994e-05 2024-09-11T06:50:07.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.239 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.239 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:07.239 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823686+0000", 2024-09-11T06:50:07.239 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4358999999999998e-05 2024-09-11T06:50:07.240 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.240 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.240 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:07.240 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823701+0000", 2024-09-11T06:50:07.240 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4847e-05 2024-09-11T06:50:07.240 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.240 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.240 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:07.240 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824518+0000", 2024-09-11T06:50:07.240 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00081716700000000005 2024-09-11T06:50:07.240 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.241 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.241 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:07.241 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824546+0000", 2024-09-11T06:50:07.241 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7860000000000001e-05 2024-09-11T06:50:07.241 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.241 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.241 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:07.241 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826558+0000", 2024-09-11T06:50:07.241 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0020120009999999998 2024-09-11T06:50:07.241 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.242 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.242 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:07.242 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826652+0000", 2024-09-11T06:50:07.242 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3431999999999996e-05 2024-09-11T06:50:07.242 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.242 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:07.242 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.242 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.242 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.242 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:139 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'97, mlcod=13'86)", 2024-09-11T06:50:07.242 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:50:07.243 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 60.346402329999997, 2024-09-11T06:50:07.243 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003732221, 2024-09-11T06:50:07.243 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:07.243 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:07.243 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:07.243 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:07.243 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.243 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:07.243 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:50:07.243 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.243 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.244 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.244 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:07.244 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:50:07.244 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.244 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.244 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.244 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:07.244 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825858+0000", 2024-09-11T06:50:07.244 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5829999999999999e-06 2024-09-11T06:50:07.244 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.244 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.244 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:07.244 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825874+0000", 2024-09-11T06:50:07.245 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5967000000000001e-05 2024-09-11T06:50:07.245 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.245 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.245 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:07.245 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825909+0000", 2024-09-11T06:50:07.245 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5692999999999999e-05 2024-09-11T06:50:07.245 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.245 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.245 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:07.245 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825923+0000", 2024-09-11T06:50:07.245 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4080000000000001e-05 2024-09-11T06:50:07.246 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.246 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.246 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:07.246 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826010+0000", 2024-09-11T06:50:07.246 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6248999999999994e-05 2024-09-11T06:50:07.246 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.246 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.246 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:07.246 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826038+0000", 2024-09-11T06:50:07.246 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8010000000000001e-05 2024-09-11T06:50:07.246 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.246 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.246 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:07.247 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829499+0000", 2024-09-11T06:50:07.247 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003461783 2024-09-11T06:50:07.247 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.247 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.247 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:07.247 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829588+0000", 2024-09-11T06:50:07.247 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8856000000000004e-05 2024-09-11T06:50:07.247 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.247 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:07.247 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.247 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.247 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.247 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:140 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'98, mlcod=13'93)", 2024-09-11T06:50:07.248 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:50:07.248 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 60.344551600999999, 2024-09-11T06:50:07.248 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0057995929999999996, 2024-09-11T06:50:07.248 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:07.248 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:07.248 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:07.248 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:07.248 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.248 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:07.248 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:50:07.248 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.248 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.249 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.249 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:07.249 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:50:07.249 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.249 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.249 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.249 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:07.249 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827708+0000", 2024-09-11T06:50:07.249 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.547e-06 2024-09-11T06:50:07.249 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.249 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.249 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:07.249 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827765+0000", 2024-09-11T06:50:07.250 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6266e-05 2024-09-11T06:50:07.250 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.250 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.250 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:07.250 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827791+0000", 2024-09-11T06:50:07.250 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6098999999999999e-05 2024-09-11T06:50:07.250 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.250 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.250 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:07.250 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827806+0000", 2024-09-11T06:50:07.250 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5537e-05 2024-09-11T06:50:07.250 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.251 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.251 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:07.251 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827928+0000", 2024-09-11T06:50:07.251 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000122012 2024-09-11T06:50:07.251 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.251 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.251 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:07.251 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827961+0000", 2024-09-11T06:50:07.251 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2663999999999998e-05 2024-09-11T06:50:07.251 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.251 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.252 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:07.252 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833409+0000", 2024-09-11T06:50:07.252 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054479350000000001 2024-09-11T06:50:07.252 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.252 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.252 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:07.252 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833506+0000", 2024-09-11T06:50:07.252 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7533000000000003e-05 2024-09-11T06:50:07.252 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.252 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:07.252 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.252 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.253 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:141 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'99, mlcod=13'93)", 2024-09-11T06:50:07.253 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:50:07.253 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 60.343658218000002, 2024-09-11T06:50:07.253 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055394709999999998, 2024-09-11T06:50:07.253 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:07.253 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:07.253 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:07.253 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:07.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.253 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:07.253 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:50:07.253 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.254 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.254 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:07.254 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:50:07.254 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.254 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.254 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:07.254 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828602+0000", 2024-09-11T06:50:07.254 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.46e-06 2024-09-11T06:50:07.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.254 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.254 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:07.255 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828618+0000", 2024-09-11T06:50:07.255 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6592999999999999e-05 2024-09-11T06:50:07.255 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.255 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.255 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:07.255 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828646+0000", 2024-09-11T06:50:07.255 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7552999999999999e-05 2024-09-11T06:50:07.255 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.255 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.255 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:07.255 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828663+0000", 2024-09-11T06:50:07.255 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.719e-05 2024-09-11T06:50:07.255 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.256 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.256 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:07.256 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829023+0000", 2024-09-11T06:50:07.256 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036054799999999998 2024-09-11T06:50:07.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.256 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.256 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:07.256 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829062+0000", 2024-09-11T06:50:07.256 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8865999999999997e-05 2024-09-11T06:50:07.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.256 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.256 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:07.256 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834078+0000", 2024-09-11T06:50:07.257 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050157480000000004 2024-09-11T06:50:07.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.257 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.257 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:07.257 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834140+0000", 2024-09-11T06:50:07.257 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1513000000000005e-05 2024-09-11T06:50:07.257 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.257 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:07.257 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.257 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.257 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:144 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'102, mlcod=13'98)", 2024-09-11T06:50:07.257 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:50:07.257 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 60.334497605000003, 2024-09-11T06:50:07.258 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003210348, 2024-09-11T06:50:07.258 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:07.258 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:07.258 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:07.258 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:07.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.258 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:07.258 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:50:07.258 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.258 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:07.258 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:50:07.258 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.259 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.259 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:07.259 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837768+0000", 2024-09-11T06:50:07.259 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9330000000000004e-06 2024-09-11T06:50:07.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.259 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.259 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:07.259 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837858+0000", 2024-09-11T06:50:07.259 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0185999999999993e-05 2024-09-11T06:50:07.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.259 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.259 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:07.259 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837890+0000", 2024-09-11T06:50:07.259 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2437000000000001e-05 2024-09-11T06:50:07.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.260 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.260 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:07.260 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837905+0000", 2024-09-11T06:50:07.260 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:50:07.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.260 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.260 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:07.260 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838019+0000", 2024-09-11T06:50:07.260 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000114336 2024-09-11T06:50:07.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.260 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.260 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:07.260 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838052+0000", 2024-09-11T06:50:07.261 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3139000000000002e-05 2024-09-11T06:50:07.261 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.261 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.261 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:07.261 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840914+0000", 2024-09-11T06:50:07.261 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028612780000000001 2024-09-11T06:50:07.261 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.261 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.261 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:07.261 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840971+0000", 2024-09-11T06:50:07.261 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7402999999999999e-05 2024-09-11T06:50:07.261 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.261 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:07.262 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.262 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.262 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:145 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'103, mlcod=13'98)", 2024-09-11T06:50:07.262 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:50:07.262 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 60.333705721000001, 2024-09-11T06:50:07.262 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054455409999999999, 2024-09-11T06:50:07.262 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:07.262 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:07.262 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:07.262 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:07.262 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.262 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:07.262 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:50:07.263 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.263 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.263 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.263 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:07.263 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:50:07.263 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.263 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.263 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.263 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:07.263 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838554+0000", 2024-09-11T06:50:07.263 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3430000000000001e-06 2024-09-11T06:50:07.263 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.263 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.263 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:07.263 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838640+0000", 2024-09-11T06:50:07.264 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5569999999999996e-05 2024-09-11T06:50:07.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.264 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:07.264 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838660+0000", 2024-09-11T06:50:07.264 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0576000000000001e-05 2024-09-11T06:50:07.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.264 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:07.264 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838674+0000", 2024-09-11T06:50:07.264 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3529999999999999e-05 2024-09-11T06:50:07.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.265 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:07.265 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838838+0000", 2024-09-11T06:50:07.265 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016468799999999999 2024-09-11T06:50:07.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.265 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:07.265 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838885+0000", 2024-09-11T06:50:07.265 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6884e-05 2024-09-11T06:50:07.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.265 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:07.265 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843901+0000", 2024-09-11T06:50:07.265 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050162310000000003 2024-09-11T06:50:07.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.266 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.266 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:07.266 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843998+0000", 2024-09-11T06:50:07.266 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.6718999999999993e-05 2024-09-11T06:50:07.266 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.266 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:07.266 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.266 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.266 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:07.266 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:07.268 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:50:07.268 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:07.264+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:07.369 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:07.368+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:07.417 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:07.417 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:07.417 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:50:07.417 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:07.430 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:50:07.470 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:07.468+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:07.571 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:07.572+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:07.587 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:07.587 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:07.587 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:50:07.587 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:50:07.588 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:07.600 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:50:07.671 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:07.668+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:07.719 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:07.719 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:50:07.719 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:50:07.719 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:50:07.719 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.719 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:5 2.2 e17/15 2:5cea7035:::rbd_info:head v 17'1, mlcod=0'0)", 2024-09-11T06:50:07.720 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:50:07.720 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 55.482895581000001, 2024-09-11T06:50:07.720 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0019446719999999999, 2024-09-11T06:50:07.720 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:07.720 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:07.720 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:07.720 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:07.720 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.720 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:07.720 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:50:07.720 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.720 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.720 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.720 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:07.721 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:50:07.721 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.721 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.721 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.721 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:07.721 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236085+0000", 2024-09-11T06:50:07.721 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6000000000000003e-07 2024-09-11T06:50:07.721 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.721 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.721 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:07.721 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236108+0000", 2024-09-11T06:50:07.721 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3113e-05 2024-09-11T06:50:07.721 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.722 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.722 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:07.722 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236133+0000", 2024-09-11T06:50:07.722 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5723000000000001e-05 2024-09-11T06:50:07.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.722 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.722 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:07.722 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236154+0000", 2024-09-11T06:50:07.722 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0333999999999999e-05 2024-09-11T06:50:07.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.722 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.722 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:07.722 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236297+0000", 2024-09-11T06:50:07.723 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014278500000000001 2024-09-11T06:50:07.723 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.723 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.723 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:07.723 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236383+0000", 2024-09-11T06:50:07.723 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5927999999999995e-05 2024-09-11T06:50:07.723 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.723 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.723 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:07.723 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.237950+0000", 2024-09-11T06:50:07.723 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015673690000000001 2024-09-11T06:50:07.723 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.723 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.723 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:07.723 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.238029+0000", 2024-09-11T06:50:07.724 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8659999999999996e-05 2024-09-11T06:50:07.724 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.724 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:07.724 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.724 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.724 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.724 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:7 2.2 e18/15 2:5cea7035:::rbd_info:head v 18'2, mlcod=0'0)", 2024-09-11T06:50:07.724 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:50:07.724 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 54.468570374000002, 2024-09-11T06:50:07.724 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001907192, 2024-09-11T06:50:07.724 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:07.724 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:07.724 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:07.725 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:07.725 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.725 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:07.725 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:50:07.725 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.725 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.725 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.725 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:07.725 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:50:07.725 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:07.725 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.725 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.725 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:07.726 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250410+0000", 2024-09-11T06:50:07.726 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7700000000000004e-07 2024-09-11T06:50:07.726 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.726 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.726 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:07.726 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250445+0000", 2024-09-11T06:50:07.726 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4749000000000002e-05 2024-09-11T06:50:07.726 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.726 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.726 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:07.726 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250478+0000", 2024-09-11T06:50:07.726 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3269999999999998e-05 2024-09-11T06:50:07.726 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.726 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.727 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:07.727 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250501+0000", 2024-09-11T06:50:07.727 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3632999999999999e-05 2024-09-11T06:50:07.727 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.727 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.727 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:07.727 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250633+0000", 2024-09-11T06:50:07.727 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013143200000000001 2024-09-11T06:50:07.727 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.727 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.727 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:07.727 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250700+0000", 2024-09-11T06:50:07.728 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7002999999999995e-05 2024-09-11T06:50:07.728 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.728 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.728 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:07.728 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.252172+0000", 2024-09-11T06:50:07.728 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014719550000000001 2024-09-11T06:50:07.728 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:07.728 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:07.728 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:07.728 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.252316+0000", 2024-09-11T06:50:07.728 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144473 2024-09-11T06:50:07.728 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.728 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:07.728 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.729 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:07.729 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:07.729 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:07.733 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:50:07.772 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:07.772+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:07.874 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:07.872+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:07.891 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:07.891 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:07.891 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:50:07.891 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:07.904 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:50:07.930 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.3 to 1 (10000) 2024-09-11T06:50:07.931 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is set 2024-09-11T06:50:07.961 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-11T06:50:08.005 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:08.004+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:08.054 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:08.054 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:08.054 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:50:08.054 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:50:08.054 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:08.068 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:50:08.112 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:08.112+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:08.176 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:08.172+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:08.185 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:08.185 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:50:08.185 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:50:08.186 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:50:08.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:08.186 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:1 2.3 e16/15 2:c4c92e5a:::rbd_trash:head v 16'1, mlcod=0'0)", 2024-09-11T06:50:08.186 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:50:08.186 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 56.949436828000003, 2024-09-11T06:50:08.186 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001234761, 2024-09-11T06:50:08.186 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:08.186 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:08.186 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:08.186 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:08.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:08.186 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:08.186 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:50:08.187 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:08.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:08.187 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:08.187 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:08.187 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:50:08.187 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:08.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:08.187 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:08.187 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:08.187 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:50:08.187 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6000000000000003e-07 2024-09-11T06:50:08.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:08.187 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:08.187 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:08.187 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236296+0000", 2024-09-11T06:50:08.188 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6393999999999999e-05 2024-09-11T06:50:08.188 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:08.188 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:08.188 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:08.188 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236317+0000", 2024-09-11T06:50:08.188 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1243e-05 2024-09-11T06:50:08.188 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:08.188 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:08.188 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:08.188 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236333+0000", 2024-09-11T06:50:08.188 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.558e-05 2024-09-11T06:50:08.188 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:08.188 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:08.188 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:08.188 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236435+0000", 2024-09-11T06:50:08.189 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000102803 2024-09-11T06:50:08.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:08.189 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:08.189 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:08.189 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236474+0000", 2024-09-11T06:50:08.189 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8383999999999997e-05 2024-09-11T06:50:08.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:08.189 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:08.189 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:08.189 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.237432+0000", 2024-09-11T06:50:08.189 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00095849300000000004 2024-09-11T06:50:08.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:08.189 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:08.189 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:08.189 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.237513+0000", 2024-09-11T06:50:08.190 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1204000000000004e-05 2024-09-11T06:50:08.190 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:08.190 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:08.190 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:08.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:08.190 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:08.190 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "MOSDPGPush(2.2 37/36 [PushOp(2:5cea7035:::rbd_info:head, version: 18'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@18'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:50:08.190 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:50.554243+0000", 2024-09-11T06:50:08.190 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 17.631472367000001, 2024-09-11T06:50:08.190 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00051362700000000001, 2024-09-11T06:50:08.190 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:08.190 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:08.190 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:08.190 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:08.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:08.191 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:08.191 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:50.554243+0000", 2024-09-11T06:50:08.191 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:08.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:08.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:08.191 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:08.191 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:50.554243+0000", 2024-09-11T06:50:08.191 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:08.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:08.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:08.191 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:08.191 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:50.554245+0000", 2024-09-11T06:50:08.191 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6759999999999999e-06 2024-09-11T06:50:08.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:08.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:08.192 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:08.192 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:50.554270+0000", 2024-09-11T06:50:08.192 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5244000000000001e-05 2024-09-11T06:50:08.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:08.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:08.192 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:08.192 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:50.554316+0000", 2024-09-11T06:50:08.192 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6164e-05 2024-09-11T06:50:08.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:08.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:08.192 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:08.192 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:50.554361+0000", 2024-09-11T06:50:08.192 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5102000000000003e-05 2024-09-11T06:50:08.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:08.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:08.193 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:08.193 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:50.554463+0000", 2024-09-11T06:50:08.193 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000101968 2024-09-11T06:50:08.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:08.193 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:08.193 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:08.193 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:50.554472+0000", 2024-09-11T06:50:08.193 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.816e-06 2024-09-11T06:50:08.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:08.193 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:08.193 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:08.193 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:50.554757+0000", 2024-09-11T06:50:08.193 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00028465700000000001 2024-09-11T06:50:08.193 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:08.193 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:08.193 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:08.194 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:08.194 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:08.194 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:08.199 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:50:08.276 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:08.276+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:08.330 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:08.330 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:08.330 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:50:08.330 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:08.344 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:50:08.387 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:08.388+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:08.479 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:08.480+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:08.510 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:08.510 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:08.510 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:50:08.510 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:50:08.510 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:08.524 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:50:08.579 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:08.576+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:08.680 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:08.680+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:08.695 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:08.695 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:08.695 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:08.695 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:50:08.695 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.695 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:08.695 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:50:08.695 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 62.522318247999998, 2024-09-11T06:50:08.695 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011420669, 2024-09-11T06:50:08.696 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:08.696 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:08.696 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:08.696 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:08.696 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:08.696 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:08.696 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 37 2024-09-11T06:50:08.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.696 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:08.696 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.696 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:08.696 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:50:08.696 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.697 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.697 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:08.697 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:50:08.697 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.697 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.697 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:08.697 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170509+0000", 2024-09-11T06:50:08.697 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.13e-06 2024-09-11T06:50:08.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.697 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.697 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:08.697 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170521+0000", 2024-09-11T06:50:08.697 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1776999999999999e-05 2024-09-11T06:50:08.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.698 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.698 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:08.698 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170538+0000", 2024-09-11T06:50:08.698 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7413000000000001e-05 2024-09-11T06:50:08.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.698 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.698 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:08.698 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170570+0000", 2024-09-11T06:50:08.698 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2313e-05 2024-09-11T06:50:08.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.698 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.698 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:08.698 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179768+0000", 2024-09-11T06:50:08.698 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0091978010000000002 2024-09-11T06:50:08.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.699 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:08.699 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179951+0000", 2024-09-11T06:50:08.699 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018227900000000001 2024-09-11T06:50:08.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.699 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:08.699 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.180175+0000", 2024-09-11T06:50:08.699 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022432800000000001 2024-09-11T06:50:08.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.699 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:08.699 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181112+0000", 2024-09-11T06:50:08.699 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00093712500000000002 2024-09-11T06:50:08.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.700 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.700 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:08.700 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181789+0000", 2024-09-11T06:50:08.700 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00067705100000000002 2024-09-11T06:50:08.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.700 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.700 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:08.700 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181871+0000", 2024-09-11T06:50:08.700 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1816000000000004e-05 2024-09-11T06:50:08.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.700 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.700 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:08.700 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181929+0000", 2024-09-11T06:50:08.700 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7636000000000002e-05 2024-09-11T06:50:08.701 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.701 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:08.701 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.701 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.701 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.701 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:08.701 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:08.701 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 62.421059286999999, 2024-09-11T06:50:08.701 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283336, 2024-09-11T06:50:08.701 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:08.701 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:08.701 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:08.701 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:08.701 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:08.702 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:08.702 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 87 2024-09-11T06:50:08.702 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.702 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:08.702 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.702 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:08.702 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:08.702 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.702 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.702 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.702 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:08.702 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:08.702 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.702 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.702 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.703 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:08.703 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271768+0000", 2024-09-11T06:50:08.703 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7599999999999996e-07 2024-09-11T06:50:08.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.703 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.703 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:08.703 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271860+0000", 2024-09-11T06:50:08.703 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2639999999999994e-05 2024-09-11T06:50:08.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.703 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.703 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:08.703 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271873+0000", 2024-09-11T06:50:08.703 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3139000000000001e-05 2024-09-11T06:50:08.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.704 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.704 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:08.704 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271882+0000", 2024-09-11T06:50:08.704 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2369999999999992e-06 2024-09-11T06:50:08.704 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.704 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.704 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:08.704 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277150+0000", 2024-09-11T06:50:08.704 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052684419999999999 2024-09-11T06:50:08.704 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.704 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.704 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:08.704 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277273+0000", 2024-09-11T06:50:08.704 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012278899999999999 2024-09-11T06:50:08.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.705 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.705 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:08.705 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277466+0000", 2024-09-11T06:50:08.705 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193578 2024-09-11T06:50:08.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.705 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.705 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:08.705 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.280808+0000", 2024-09-11T06:50:08.705 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003341971 2024-09-11T06:50:08.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.705 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.705 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:08.705 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284419+0000", 2024-09-11T06:50:08.706 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003610982 2024-09-11T06:50:08.706 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.706 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.706 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:08.706 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284531+0000", 2024-09-11T06:50:08.706 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011121299999999999 2024-09-11T06:50:08.706 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.706 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.706 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:08.706 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284600+0000", 2024-09-11T06:50:08.706 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9592999999999996e-05 2024-09-11T06:50:08.706 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.706 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:08.706 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.706 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.706 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.707 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:08.707 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:08.707 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 62.420929035, 2024-09-11T06:50:08.707 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283314, 2024-09-11T06:50:08.707 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:08.707 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:08.707 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:08.707 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:08.707 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:08.707 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:08.707 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 88 2024-09-11T06:50:08.707 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.707 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:08.707 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.708 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:08.708 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:08.708 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.708 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.708 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.708 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:08.708 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:08.708 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.708 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.708 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.708 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:08.708 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271898+0000", 2024-09-11T06:50:08.708 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2999999999999999e-07 2024-09-11T06:50:08.708 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.709 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.709 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:08.709 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271904+0000", 2024-09-11T06:50:08.709 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.767e-06 2024-09-11T06:50:08.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.709 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.709 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:08.709 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271913+0000", 2024-09-11T06:50:08.709 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7200000000000001e-06 2024-09-11T06:50:08.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.709 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.709 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:08.709 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271920+0000", 2024-09-11T06:50:08.709 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7100000000000001e-06 2024-09-11T06:50:08.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.709 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.710 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:08.710 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278684+0000", 2024-09-11T06:50:08.710 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0067642650000000002 2024-09-11T06:50:08.710 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.710 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.710 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:08.710 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278838+0000", 2024-09-11T06:50:08.710 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015371499999999999 2024-09-11T06:50:08.710 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.710 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.710 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:08.710 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.279040+0000", 2024-09-11T06:50:08.710 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020207499999999999 2024-09-11T06:50:08.710 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.711 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.711 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:08.711 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.281448+0000", 2024-09-11T06:50:08.711 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002407686 2024-09-11T06:50:08.711 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.711 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.711 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:08.711 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284615+0000", 2024-09-11T06:50:08.711 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031675499999999999 2024-09-11T06:50:08.711 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.711 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.711 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:08.711 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284682+0000", 2024-09-11T06:50:08.711 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7032000000000005e-05 2024-09-11T06:50:08.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.712 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.712 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:08.712 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284730+0000", 2024-09-11T06:50:08.712 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7790000000000002e-05 2024-09-11T06:50:08.712 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.712 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:08.712 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.712 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.712 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:08.712 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:50:08.712 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 62.253027813000003, 2024-09-11T06:50:08.712 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011114877, 2024-09-11T06:50:08.712 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:08.713 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:08.713 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:08.713 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:08.713 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:08.713 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:08.713 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 107 2024-09-11T06:50:08.713 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.713 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:08.713 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.713 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:08.713 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:50:08.713 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.713 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.713 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.714 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:08.714 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:50:08.714 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.714 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.714 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.714 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:08.714 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439799+0000", 2024-09-11T06:50:08.714 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.16e-07 2024-09-11T06:50:08.714 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.714 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.714 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:08.714 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439806+0000", 2024-09-11T06:50:08.714 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3640000000000004e-06 2024-09-11T06:50:08.714 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.714 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.715 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:08.715 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439816+0000", 2024-09-11T06:50:08.715 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0485999999999999e-05 2024-09-11T06:50:08.715 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.715 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.715 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:08.715 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439824+0000", 2024-09-11T06:50:08.715 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.6699999999999994e-06 2024-09-11T06:50:08.715 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.715 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.715 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:08.715 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443328+0000", 2024-09-11T06:50:08.716 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035044329999999999 2024-09-11T06:50:08.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.716 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.716 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:08.716 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443461+0000", 2024-09-11T06:50:08.716 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013272199999999999 2024-09-11T06:50:08.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.716 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.716 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:08.716 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443678+0000", 2024-09-11T06:50:08.716 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021674500000000001 2024-09-11T06:50:08.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.716 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.716 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:08.717 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449635+0000", 2024-09-11T06:50:08.717 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0059572119999999999 2024-09-11T06:50:08.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.717 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.717 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:08.717 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450750+0000", 2024-09-11T06:50:08.717 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001115654 2024-09-11T06:50:08.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.717 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.717 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:08.717 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450832+0000", 2024-09-11T06:50:08.717 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1188999999999998e-05 2024-09-11T06:50:08.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.718 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.718 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:08.718 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450913+0000", 2024-09-11T06:50:08.718 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1485999999999999e-05 2024-09-11T06:50:08.718 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.718 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:08.718 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.718 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.718 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.718 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:08.718 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:50:08.718 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 62.252436838000001, 2024-09-11T06:50:08.718 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011236556, 2024-09-11T06:50:08.718 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:08.719 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:08.719 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:08.719 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:08.719 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:08.719 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:08.719 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 109 2024-09-11T06:50:08.719 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.719 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:08.719 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.719 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:08.719 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:50:08.719 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.719 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.719 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.719 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:08.719 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:50:08.720 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.720 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.720 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.720 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:08.720 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440391+0000", 2024-09-11T06:50:08.720 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8360000000000001e-06 2024-09-11T06:50:08.720 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.720 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.720 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:08.720 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440560+0000", 2024-09-11T06:50:08.720 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000169085 2024-09-11T06:50:08.720 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.720 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.720 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:08.721 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440585+0000", 2024-09-11T06:50:08.721 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4530000000000001e-05 2024-09-11T06:50:08.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.721 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.721 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:08.721 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440606+0000", 2024-09-11T06:50:08.721 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1277e-05 2024-09-11T06:50:08.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.721 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.721 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:08.721 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446479+0000", 2024-09-11T06:50:08.721 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0058734989999999999 2024-09-11T06:50:08.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.721 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.722 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:08.722 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446649+0000", 2024-09-11T06:50:08.722 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016948199999999999 2024-09-11T06:50:08.722 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.722 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.722 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:08.722 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446854+0000", 2024-09-11T06:50:08.722 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020491500000000001 2024-09-11T06:50:08.722 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.722 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.722 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:08.722 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451380+0000", 2024-09-11T06:50:08.722 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0045264110000000001 2024-09-11T06:50:08.722 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.722 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.723 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:08.723 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451479+0000", 2024-09-11T06:50:08.723 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.8975999999999996e-05 2024-09-11T06:50:08.723 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.723 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.723 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:08.723 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451562+0000", 2024-09-11T06:50:08.723 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3213000000000004e-05 2024-09-11T06:50:08.723 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.723 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.723 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:08.723 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451626+0000", 2024-09-11T06:50:08.723 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3331999999999996e-05 2024-09-11T06:50:08.723 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.723 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:08.723 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.724 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.724 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.724 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:08.724 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:50:08.724 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 62.252187710000001, 2024-09-11T06:50:08.724 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011252866, 2024-09-11T06:50:08.724 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:08.724 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:08.724 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:08.724 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:08.724 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:08.724 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:08.724 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 110 2024-09-11T06:50:08.724 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.724 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:08.725 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.725 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:08.725 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:50:08.725 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.725 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.725 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.725 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:08.725 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:50:08.725 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.725 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.725 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.725 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:08.725 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440640+0000", 2024-09-11T06:50:08.725 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5430000000000001e-06 2024-09-11T06:50:08.726 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.726 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.726 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:08.726 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440651+0000", 2024-09-11T06:50:08.726 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0672999999999999e-05 2024-09-11T06:50:08.726 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.726 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.726 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:08.726 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440670+0000", 2024-09-11T06:50:08.726 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9726999999999999e-05 2024-09-11T06:50:08.726 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.726 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.726 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:08.726 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440691+0000", 2024-09-11T06:50:08.727 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0526e-05 2024-09-11T06:50:08.727 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.727 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.727 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:08.727 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448191+0000", 2024-09-11T06:50:08.727 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0075005990000000002 2024-09-11T06:50:08.727 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.727 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.727 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:08.727 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448390+0000", 2024-09-11T06:50:08.727 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000198815 2024-09-11T06:50:08.727 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.727 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.727 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:08.728 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448735+0000", 2024-09-11T06:50:08.728 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034424999999999999 2024-09-11T06:50:08.728 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.728 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.728 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:08.728 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451742+0000", 2024-09-11T06:50:08.728 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030077440000000001 2024-09-11T06:50:08.728 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.728 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.728 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:08.728 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451761+0000", 2024-09-11T06:50:08.728 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9196999999999999e-05 2024-09-11T06:50:08.728 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.728 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.728 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:08.729 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451834+0000", 2024-09-11T06:50:08.729 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2071999999999997e-05 2024-09-11T06:50:08.729 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.729 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.729 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:08.729 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451891+0000", 2024-09-11T06:50:08.729 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7720000000000003e-05 2024-09-11T06:50:08.729 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.729 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:08.729 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.729 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.729 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.729 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:08.730 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:08.730 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 62.252079835000004, 2024-09-11T06:50:08.730 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014944941, 2024-09-11T06:50:08.730 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:08.730 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:08.730 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:08.730 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:08.730 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:08.732 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:08.732 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 111 2024-09-11T06:50:08.732 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.732 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:08.732 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.732 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:08.732 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:08.732 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.733 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.733 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.733 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:08.733 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:08.733 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.733 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.733 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.733 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:08.733 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440748+0000", 2024-09-11T06:50:08.733 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:50:08.733 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.733 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.733 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:08.733 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440760+0000", 2024-09-11T06:50:08.733 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1897e-05 2024-09-11T06:50:08.734 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.734 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.734 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:08.734 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440782+0000", 2024-09-11T06:50:08.734 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2552999999999999e-05 2024-09-11T06:50:08.734 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.734 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.734 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:08.734 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440793+0000", 2024-09-11T06:50:08.734 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0643e-05 2024-09-11T06:50:08.734 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.734 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.734 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:08.734 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449755+0000", 2024-09-11T06:50:08.735 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089622620000000004 2024-09-11T06:50:08.735 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.735 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.735 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:08.735 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449918+0000", 2024-09-11T06:50:08.735 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163086 2024-09-11T06:50:08.735 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.735 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.735 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:08.735 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450150+0000", 2024-09-11T06:50:08.735 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023204500000000001 2024-09-11T06:50:08.735 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.735 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.735 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:08.736 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454533+0000", 2024-09-11T06:50:08.736 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0043831349999999998 2024-09-11T06:50:08.736 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.736 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.736 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:08.736 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455511+0000", 2024-09-11T06:50:08.736 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0009776050000000001 2024-09-11T06:50:08.736 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.736 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.736 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:08.736 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455597+0000", 2024-09-11T06:50:08.736 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6446000000000005e-05 2024-09-11T06:50:08.736 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.737 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.737 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:08.737 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455691+0000", 2024-09-11T06:50:08.737 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3832000000000006e-05 2024-09-11T06:50:08.737 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.737 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:08.737 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.737 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.737 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.737 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:08.737 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:08.737 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 62.252010071999997, 2024-09-11T06:50:08.737 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018885411000000001, 2024-09-11T06:50:08.737 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:08.738 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:08.738 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:08.738 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:08.738 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:08.738 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:08.738 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 112 2024-09-11T06:50:08.738 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.738 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:08.738 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.738 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:08.738 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:08.738 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.738 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440817+0000", 2024-09-11T06:50:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:50:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441003+0000", 2024-09-11T06:50:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018661200000000001 2024-09-11T06:50:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.740 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:08.740 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441027+0000", 2024-09-11T06:50:08.740 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3920000000000001e-05 2024-09-11T06:50:08.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.740 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.740 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:08.740 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441062+0000", 2024-09-11T06:50:08.740 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4663000000000003e-05 2024-09-11T06:50:08.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.740 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.740 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:08.740 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451981+0000", 2024-09-11T06:50:08.740 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010918926000000001 2024-09-11T06:50:08.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.741 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.741 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:08.741 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452111+0000", 2024-09-11T06:50:08.741 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129529 2024-09-11T06:50:08.741 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.741 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.741 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:08.741 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452311+0000", 2024-09-11T06:50:08.741 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020013799999999999 2024-09-11T06:50:08.741 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.741 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.741 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:08.741 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455809+0000", 2024-09-11T06:50:08.741 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003498443 2024-09-11T06:50:08.741 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.742 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.742 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:08.742 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459566+0000", 2024-09-11T06:50:08.742 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003757271 2024-09-11T06:50:08.742 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.742 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.742 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:08.742 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459640+0000", 2024-09-11T06:50:08.742 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3949000000000006e-05 2024-09-11T06:50:08.742 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.742 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.742 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:08.742 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459701+0000", 2024-09-11T06:50:08.742 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1117000000000005e-05 2024-09-11T06:50:08.742 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 62.251731221, 2024-09-11T06:50:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018760915, 2024-09-11T06:50:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 113 2024-09-11T06:50:08.744 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.744 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:08.744 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.744 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:08.744 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:08.744 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.744 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.744 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.744 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:08.744 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:08.744 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.744 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.744 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.744 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:08.745 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441096+0000", 2024-09-11T06:50:08.745 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4130000000000001e-06 2024-09-11T06:50:08.745 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.745 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.745 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:08.745 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441110+0000", 2024-09-11T06:50:08.745 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3900000000000001e-05 2024-09-11T06:50:08.745 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.745 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.745 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:08.745 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441131+0000", 2024-09-11T06:50:08.745 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0636999999999999e-05 2024-09-11T06:50:08.745 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.745 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.746 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:08.746 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441147+0000", 2024-09-11T06:50:08.746 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6512999999999999e-05 2024-09-11T06:50:08.746 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.746 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.746 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:08.746 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453274+0000", 2024-09-11T06:50:08.746 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012126764999999999 2024-09-11T06:50:08.746 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.746 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.746 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:08.746 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453498+0000", 2024-09-11T06:50:08.746 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000223568 2024-09-11T06:50:08.746 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.747 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.747 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:08.747 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453872+0000", 2024-09-11T06:50:08.747 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037467099999999998 2024-09-11T06:50:08.747 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.747 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.747 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:08.747 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458709+0000", 2024-09-11T06:50:08.747 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0048365680000000003 2024-09-11T06:50:08.747 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.747 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.747 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:08.747 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459716+0000", 2024-09-11T06:50:08.748 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010075240000000001 2024-09-11T06:50:08.748 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.748 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.748 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:08.748 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459779+0000", 2024-09-11T06:50:08.748 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2336999999999996e-05 2024-09-11T06:50:08.748 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.748 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.748 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:08.748 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459856+0000", 2024-09-11T06:50:08.748 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7019000000000002e-05 2024-09-11T06:50:08.748 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.749 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:08.749 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.749 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.749 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.749 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:08.749 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:08.749 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 62.251654481999999, 2024-09-11T06:50:08.749 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018963114, 2024-09-11T06:50:08.749 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:08.749 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:08.749 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:08.749 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:08.749 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:08.750 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:08.750 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 114 2024-09-11T06:50:08.750 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.750 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:08.750 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.750 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:08.750 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:08.750 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.750 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.750 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.750 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:08.750 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:08.750 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.751 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.751 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.751 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:08.751 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:08.751 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9299999999999996e-07 2024-09-11T06:50:08.751 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.751 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.751 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:08.751 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441180+0000", 2024-09-11T06:50:08.751 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7200000000000006e-06 2024-09-11T06:50:08.751 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.751 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.751 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:08.751 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441265+0000", 2024-09-11T06:50:08.752 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4753000000000007e-05 2024-09-11T06:50:08.752 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.752 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.752 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:08.752 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441288+0000", 2024-09-11T06:50:08.752 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2860000000000001e-05 2024-09-11T06:50:08.752 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.752 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.752 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:08.752 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454585+0000", 2024-09-11T06:50:08.752 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013297286 2024-09-11T06:50:08.752 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.752 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.752 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:08.752 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454714+0000", 2024-09-11T06:50:08.753 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000128465 2024-09-11T06:50:08.753 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.753 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.753 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:08.753 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454909+0000", 2024-09-11T06:50:08.753 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019522499999999999 2024-09-11T06:50:08.753 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.753 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.753 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:08.753 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459868+0000", 2024-09-11T06:50:08.753 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0049591239999999997 2024-09-11T06:50:08.753 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.753 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.753 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:08.753 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459988+0000", 2024-09-11T06:50:08.754 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000120532 2024-09-11T06:50:08.754 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.754 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.754 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:08.754 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460071+0000", 2024-09-11T06:50:08.754 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2256000000000007e-05 2024-09-11T06:50:08.754 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.754 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.754 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:08.754 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460135+0000", 2024-09-11T06:50:08.754 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3999999999999997e-05 2024-09-11T06:50:08.754 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.754 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:08.754 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.755 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.755 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.755 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:08.755 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:08.755 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 62.251496852999999, 2024-09-11T06:50:08.755 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.019036849000000002, 2024-09-11T06:50:08.755 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:08.755 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:08.755 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:08.755 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:08.755 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:08.755 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:08.755 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 115 2024-09-11T06:50:08.755 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.756 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:08.756 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.756 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:08.756 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:08.756 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.756 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.756 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.756 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:08.756 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:08.756 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.756 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.756 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.756 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:08.757 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441330+0000", 2024-09-11T06:50:08.757 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1930000000000001e-06 2024-09-11T06:50:08.757 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.757 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.757 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:08.757 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441544+0000", 2024-09-11T06:50:08.757 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021405799999999999 2024-09-11T06:50:08.757 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.757 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.757 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:08.757 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441572+0000", 2024-09-11T06:50:08.757 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7317e-05 2024-09-11T06:50:08.757 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.757 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.758 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:08.758 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441593+0000", 2024-09-11T06:50:08.758 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1702999999999999e-05 2024-09-11T06:50:08.758 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.758 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.758 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:08.758 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455891+0000", 2024-09-11T06:50:08.758 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01429708 2024-09-11T06:50:08.758 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.758 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.758 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:08.758 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456046+0000", 2024-09-11T06:50:08.758 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000155682 2024-09-11T06:50:08.758 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.758 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.759 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:08.759 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456242+0000", 2024-09-11T06:50:08.759 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000195692 2024-09-11T06:50:08.759 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.759 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.759 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:08.759 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459876+0000", 2024-09-11T06:50:08.759 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036336480000000002 2024-09-11T06:50:08.759 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.759 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.759 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:08.759 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460245+0000", 2024-09-11T06:50:08.759 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036901700000000002 2024-09-11T06:50:08.759 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.760 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.760 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:08.760 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460316+0000", 2024-09-11T06:50:08.760 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1846000000000003e-05 2024-09-11T06:50:08.760 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.760 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.760 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:08.760 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460366+0000", 2024-09-11T06:50:08.760 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9613000000000001e-05 2024-09-11T06:50:08.760 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.760 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:08.760 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.760 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.760 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.760 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:08.761 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:08.761 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 62.251174773000002, 2024-09-11T06:50:08.761 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01904432, 2024-09-11T06:50:08.761 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:08.761 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:08.761 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:08.761 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:08.761 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:08.761 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:08.761 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 116 2024-09-11T06:50:08.761 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.761 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:08.761 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.761 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:08.761 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:08.762 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.762 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.762 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.762 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:08.762 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:08.762 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.762 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.762 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.762 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:08.762 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441653+0000", 2024-09-11T06:50:08.762 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:50:08.762 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.762 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.762 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:08.762 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441665+0000", 2024-09-11T06:50:08.763 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1653e-05 2024-09-11T06:50:08.763 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.763 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.763 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:08.763 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441684+0000", 2024-09-11T06:50:08.763 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9490000000000001e-05 2024-09-11T06:50:08.763 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.763 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.763 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:08.763 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441699+0000", 2024-09-11T06:50:08.763 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5044e-05 2024-09-11T06:50:08.763 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.763 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.763 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:08.763 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457297+0000", 2024-09-11T06:50:08.763 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015598241000000001 2024-09-11T06:50:08.764 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.764 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.764 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:08.764 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457577+0000", 2024-09-11T06:50:08.764 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027930799999999998 2024-09-11T06:50:08.764 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.764 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.764 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:08.764 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457957+0000", 2024-09-11T06:50:08.764 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00038088 2024-09-11T06:50:08.764 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.764 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.764 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:08.764 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460432+0000", 2024-09-11T06:50:08.764 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024743349999999998 2024-09-11T06:50:08.765 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.765 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.765 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:08.765 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460584+0000", 2024-09-11T06:50:08.765 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015232900000000001 2024-09-11T06:50:08.765 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.765 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.765 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:08.765 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460646+0000", 2024-09-11T06:50:08.765 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1883000000000004e-05 2024-09-11T06:50:08.765 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.765 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.765 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:08.765 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460696+0000", 2024-09-11T06:50:08.766 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9589999999999998e-05 2024-09-11T06:50:08.766 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.766 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:08.766 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.766 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.766 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.766 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:08.766 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:08.766 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 62.251102707000001, 2024-09-11T06:50:08.766 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.021834473, 2024-09-11T06:50:08.766 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:08.766 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:08.766 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:08.766 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:08.767 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:08.767 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:08.767 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 117 2024-09-11T06:50:08.767 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.767 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:08.767 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.767 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:08.767 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:08.767 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.767 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.767 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.767 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:08.767 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:08.767 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.768 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.768 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.768 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:08.768 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441725+0000", 2024-09-11T06:50:08.768 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5069999999999999e-06 2024-09-11T06:50:08.768 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.768 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.768 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:08.768 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441738+0000", 2024-09-11T06:50:08.768 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2813000000000001e-05 2024-09-11T06:50:08.768 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.768 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.768 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:08.769 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441759+0000", 2024-09-11T06:50:08.769 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1246999999999998e-05 2024-09-11T06:50:08.769 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.769 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.769 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:08.769 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441779+0000", 2024-09-11T06:50:08.769 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0392999999999999e-05 2024-09-11T06:50:08.769 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.769 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.769 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:08.769 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458804+0000", 2024-09-11T06:50:08.769 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.017024182999999998 2024-09-11T06:50:08.769 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.770 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.770 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:08.770 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458977+0000", 2024-09-11T06:50:08.770 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000173926 2024-09-11T06:50:08.770 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.770 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.770 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:08.770 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459170+0000", 2024-09-11T06:50:08.770 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000192184 2024-09-11T06:50:08.770 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.770 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.770 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:08.770 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.462300+0000", 2024-09-11T06:50:08.771 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031304969999999999 2024-09-11T06:50:08.771 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.771 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.771 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:08.771 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463408+0000", 2024-09-11T06:50:08.771 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011077839999999999 2024-09-11T06:50:08.771 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.771 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.771 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:08.771 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463497+0000", 2024-09-11T06:50:08.771 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8828999999999999e-05 2024-09-11T06:50:08.771 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.771 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.772 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:08.772 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463558+0000", 2024-09-11T06:50:08.772 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1110000000000003e-05 2024-09-11T06:50:08.772 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.772 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:08.772 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.772 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.772 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.772 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:08.772 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:50:08.772 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 61.874595018999997, 2024-09-11T06:50:08.772 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012216898, 2024-09-11T06:50:08.772 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:08.772 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:08.773 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:08.773 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:08.773 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:08.773 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:08.773 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 137 2024-09-11T06:50:08.773 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.773 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:08.773 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.773 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:08.773 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:50:08.773 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.773 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.773 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.773 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:08.774 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:50:08.774 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.774 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.774 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.774 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:08.774 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818232+0000", 2024-09-11T06:50:08.774 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2839999999999999e-06 2024-09-11T06:50:08.774 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.774 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.774 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:08.774 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818326+0000", 2024-09-11T06:50:08.774 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3531999999999998e-05 2024-09-11T06:50:08.774 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.774 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.774 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:08.774 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818337+0000", 2024-09-11T06:50:08.775 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0655999999999999e-05 2024-09-11T06:50:08.775 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.775 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.775 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:08.775 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818342+0000", 2024-09-11T06:50:08.775 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4600000000000002e-06 2024-09-11T06:50:08.775 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.775 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.775 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:08.775 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823890+0000", 2024-09-11T06:50:08.775 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0055480700000000004 2024-09-11T06:50:08.775 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.775 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.776 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:08.776 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823953+0000", 2024-09-11T06:50:08.776 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2848999999999994e-05 2024-09-11T06:50:08.776 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.776 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.776 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:08.776 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824074+0000", 2024-09-11T06:50:08.776 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121309 2024-09-11T06:50:08.776 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.776 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.776 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:08.776 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828043+0000", 2024-09-11T06:50:08.776 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0039685240000000002 2024-09-11T06:50:08.776 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.777 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.777 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:08.777 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830293+0000", 2024-09-11T06:50:08.777 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0022503179999999998 2024-09-11T06:50:08.777 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.777 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.777 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:08.777 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830366+0000", 2024-09-11T06:50:08.777 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2539999999999993e-05 2024-09-11T06:50:08.777 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.777 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.777 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:08.777 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830448+0000", 2024-09-11T06:50:08.777 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2355999999999996e-05 2024-09-11T06:50:08.778 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.778 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:08.778 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.778 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.778 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.778 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:08.778 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:08.778 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 61.874461205000003, 2024-09-11T06:50:08.778 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013136297, 2024-09-11T06:50:08.778 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:08.778 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:08.778 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:08.778 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:08.778 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:08.778 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:08.778 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 138 2024-09-11T06:50:08.779 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.779 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:08.779 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.779 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:08.779 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:08.779 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.779 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.779 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.779 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:08.779 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:08.779 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.779 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.779 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.779 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:08.780 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:08.780 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.89e-07 2024-09-11T06:50:08.780 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.780 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.780 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:08.780 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818369+0000", 2024-09-11T06:50:08.780 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4170000000000001e-06 2024-09-11T06:50:08.780 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.780 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.780 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:08.780 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818376+0000", 2024-09-11T06:50:08.780 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4270000000000002e-06 2024-09-11T06:50:08.780 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.780 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.781 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:08.781 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818383+0000", 2024-09-11T06:50:08.781 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3060000000000003e-06 2024-09-11T06:50:08.781 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.781 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.781 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:08.781 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824783+0000", 2024-09-11T06:50:08.781 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064003020000000001 2024-09-11T06:50:08.781 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.781 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.781 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:08.781 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825013+0000", 2024-09-11T06:50:08.782 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023039499999999999 2024-09-11T06:50:08.782 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.782 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.782 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:08.782 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825484+0000", 2024-09-11T06:50:08.782 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000470562 2024-09-11T06:50:08.782 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.782 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.782 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:08.782 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828633+0000", 2024-09-11T06:50:08.782 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031496689999999999 2024-09-11T06:50:08.782 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.782 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.782 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:08.783 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831413+0000", 2024-09-11T06:50:08.783 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027793219999999999 2024-09-11T06:50:08.783 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.783 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.783 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:08.783 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831450+0000", 2024-09-11T06:50:08.783 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7370999999999998e-05 2024-09-11T06:50:08.783 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.783 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.783 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:08.783 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831501+0000", 2024-09-11T06:50:08.783 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1036999999999999e-05 2024-09-11T06:50:08.783 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.783 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:08.783 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.784 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.784 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.784 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:08.784 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:08.784 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 61.874436725000002, 2024-09-11T06:50:08.784 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013223011, 2024-09-11T06:50:08.784 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:08.784 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:08.784 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:08.784 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:08.784 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:08.784 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:08.784 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 139 2024-09-11T06:50:08.784 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.785 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:08.785 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.785 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:08.785 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:08.785 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.785 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.785 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.785 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:08.785 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:08.785 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.785 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.785 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.785 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:08.786 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818390+0000", 2024-09-11T06:50:08.786 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5600000000000001e-07 2024-09-11T06:50:08.786 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.786 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.786 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:08.786 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818394+0000", 2024-09-11T06:50:08.786 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5929999999999999e-06 2024-09-11T06:50:08.786 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.786 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.786 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:08.786 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818399+0000", 2024-09-11T06:50:08.786 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7740000000000004e-06 2024-09-11T06:50:08.786 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.786 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.787 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:08.787 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818412+0000", 2024-09-11T06:50:08.787 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2625e-05 2024-09-11T06:50:08.787 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.787 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.787 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:08.787 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826205+0000", 2024-09-11T06:50:08.787 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0077930730000000002 2024-09-11T06:50:08.787 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.787 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.787 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:08.787 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826281+0000", 2024-09-11T06:50:08.787 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5876000000000003e-05 2024-09-11T06:50:08.787 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.787 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.787 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:08.788 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826384+0000", 2024-09-11T06:50:08.788 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103838 2024-09-11T06:50:08.788 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.788 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.788 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:08.788 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830642+0000", 2024-09-11T06:50:08.788 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004257083 2024-09-11T06:50:08.788 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.788 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.788 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:08.788 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831509+0000", 2024-09-11T06:50:08.788 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086730900000000003 2024-09-11T06:50:08.788 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.788 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.789 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:08.789 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831547+0000", 2024-09-11T06:50:08.789 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7793000000000002e-05 2024-09-11T06:50:08.789 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.789 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.789 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:08.789 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831612+0000", 2024-09-11T06:50:08.789 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5691000000000005e-05 2024-09-11T06:50:08.789 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.789 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:08.789 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.789 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.789 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.790 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:08.790 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:08.790 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 61.874408031000002, 2024-09-11T06:50:08.790 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016098024999999998, 2024-09-11T06:50:08.790 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:08.790 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:08.790 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:08.790 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:08.790 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:08.790 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:08.790 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 140 2024-09-11T06:50:08.790 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.790 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:08.790 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.791 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:08.791 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:08.791 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.791 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.791 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.791 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:08.791 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:08.791 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.791 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.791 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.791 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:08.791 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:08.791 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5199999999999998e-07 2024-09-11T06:50:08.792 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.792 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.792 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:08.792 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818479+0000", 2024-09-11T06:50:08.792 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0368000000000002e-05 2024-09-11T06:50:08.792 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.792 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.792 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:08.792 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818487+0000", 2024-09-11T06:50:08.792 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2339999999999994e-06 2024-09-11T06:50:08.792 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.792 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.792 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:08.792 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818491+0000", 2024-09-11T06:50:08.793 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4329999999999997e-06 2024-09-11T06:50:08.793 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.793 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.793 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:08.793 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828080+0000", 2024-09-11T06:50:08.793 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0095890449999999992 2024-09-11T06:50:08.793 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.793 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.793 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:08.793 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828130+0000", 2024-09-11T06:50:08.793 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9239000000000001e-05 2024-09-11T06:50:08.793 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.793 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.794 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:08.794 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828208+0000", 2024-09-11T06:50:08.794 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.852e-05 2024-09-11T06:50:08.794 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.794 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.794 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:08.794 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831625+0000", 2024-09-11T06:50:08.794 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034169410000000002 2024-09-11T06:50:08.794 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.794 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.794 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:08.794 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834411+0000", 2024-09-11T06:50:08.794 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002786111 2024-09-11T06:50:08.794 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.795 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.795 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:08.795 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834457+0000", 2024-09-11T06:50:08.795 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6041000000000002e-05 2024-09-11T06:50:08.795 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.795 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.795 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:08.795 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834516+0000", 2024-09-11T06:50:08.795 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8740999999999999e-05 2024-09-11T06:50:08.795 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.795 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:08.795 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.795 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.796 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.796 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:08.796 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:08.796 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 61.874326963999998, 2024-09-11T06:50:08.796 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016588772000000002, 2024-09-11T06:50:08.796 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:08.796 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:08.796 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:08.796 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:08.796 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:08.796 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:08.796 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 141 2024-09-11T06:50:08.796 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.796 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:08.797 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.797 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:08.797 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:08.797 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.797 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.797 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.797 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:08.797 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:08.797 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.797 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.797 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.797 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:08.797 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818500+0000", 2024-09-11T06:50:08.797 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0800000000000005e-07 2024-09-11T06:50:08.798 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.798 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.798 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:08.798 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818505+0000", 2024-09-11T06:50:08.798 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2429999999999996e-06 2024-09-11T06:50:08.798 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.798 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.798 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:08.798 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818510+0000", 2024-09-11T06:50:08.798 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9339999999999997e-06 2024-09-11T06:50:08.798 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.798 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.798 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:08.798 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818514+0000", 2024-09-11T06:50:08.799 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7270000000000001e-06 2024-09-11T06:50:08.799 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.799 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.799 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:08.799 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828711+0000", 2024-09-11T06:50:08.799 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010197031 2024-09-11T06:50:08.799 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.799 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.799 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:08.799 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828853+0000", 2024-09-11T06:50:08.799 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014201200000000001 2024-09-11T06:50:08.799 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.799 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.799 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:08.800 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.829115+0000", 2024-09-11T06:50:08.800 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026257500000000003 2024-09-11T06:50:08.800 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.800 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.800 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:08.800 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831629+0000", 2024-09-11T06:50:08.800 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025138169999999998 2024-09-11T06:50:08.800 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.800 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.800 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:08.800 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834991+0000", 2024-09-11T06:50:08.800 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033624459999999998 2024-09-11T06:50:08.800 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.800 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.801 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:08.801 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835032+0000", 2024-09-11T06:50:08.801 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1001000000000003e-05 2024-09-11T06:50:08.801 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.801 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.801 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:08.801 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835088+0000", 2024-09-11T06:50:08.801 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5478e-05 2024-09-11T06:50:08.801 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.801 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:08.801 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.801 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.802 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.802 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:50:08.802 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:50:08.802 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 61.250692061000002, 2024-09-11T06:50:08.802 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011161912, 2024-09-11T06:50:08.802 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:08.802 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:08.802 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:08.802 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:08.802 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:08.802 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:08.802 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 158 2024-09-11T06:50:08.802 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.802 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:08.802 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.803 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:08.803 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:50:08.803 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.803 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.803 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.803 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:08.803 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:50:08.803 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.803 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.803 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.803 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:08.803 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442135+0000", 2024-09-11T06:50:08.803 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3e-07 2024-09-11T06:50:08.803 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.803 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.803 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:08.804 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442259+0000", 2024-09-11T06:50:08.804 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000123903 2024-09-11T06:50:08.804 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.804 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.804 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:08.804 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442289+0000", 2024-09-11T06:50:08.804 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0833000000000001e-05 2024-09-11T06:50:08.804 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.804 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.804 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:08.804 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442301+0000", 2024-09-11T06:50:08.804 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1433e-05 2024-09-11T06:50:08.804 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.804 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.804 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:08.805 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449796+0000", 2024-09-11T06:50:08.805 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0074954959999999999 2024-09-11T06:50:08.805 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.805 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.805 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:08.805 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449917+0000", 2024-09-11T06:50:08.805 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121033 2024-09-11T06:50:08.805 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.805 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.805 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:08.805 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.450166+0000", 2024-09-11T06:50:08.805 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00024832400000000002 2024-09-11T06:50:08.805 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.806 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.806 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:08.806 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.452630+0000", 2024-09-11T06:50:08.806 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002464188 2024-09-11T06:50:08.806 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.806 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.806 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:08.806 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453149+0000", 2024-09-11T06:50:08.806 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00051860999999999997 2024-09-11T06:50:08.806 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.806 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.806 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:08.806 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453234+0000", 2024-09-11T06:50:08.807 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.5865000000000005e-05 2024-09-11T06:50:08.807 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.807 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.807 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:08.807 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453296+0000", 2024-09-11T06:50:08.807 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1496999999999998e-05 2024-09-11T06:50:08.807 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.807 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:08.807 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.807 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.807 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.807 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:50:08.807 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:50:08.807 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 61.250505959000002, 2024-09-11T06:50:08.808 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011561900999999999, 2024-09-11T06:50:08.808 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:08.808 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:08.808 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:08.808 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:08.808 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:08.808 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:08.808 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 159 2024-09-11T06:50:08.808 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.808 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:08.808 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.808 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:08.808 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:50:08.809 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.809 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.809 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.809 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:08.809 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:50:08.809 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:08.809 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.809 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.809 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:08.809 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442321+0000", 2024-09-11T06:50:08.809 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8400000000000003e-07 2024-09-11T06:50:08.809 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.809 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.810 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:08.810 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442328+0000", 2024-09-11T06:50:08.810 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5130000000000002e-06 2024-09-11T06:50:08.810 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.810 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.810 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:08.810 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442341+0000", 2024-09-11T06:50:08.810 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.363e-05 2024-09-11T06:50:08.810 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.810 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.810 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:08.810 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442349+0000", 2024-09-11T06:50:08.810 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4760000000000001e-06 2024-09-11T06:50:08.810 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.811 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.811 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:08.811 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451257+0000", 2024-09-11T06:50:08.811 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089088569999999992 2024-09-11T06:50:08.811 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.811 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.811 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:08.811 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451389+0000", 2024-09-11T06:50:08.811 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013110899999999999 2024-09-11T06:50:08.811 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.811 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.811 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:08.812 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451608+0000", 2024-09-11T06:50:08.812 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000219588 2024-09-11T06:50:08.812 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.812 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.812 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:08.812 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453517+0000", 2024-09-11T06:50:08.812 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0019088779999999999 2024-09-11T06:50:08.812 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.812 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.812 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:08.812 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453727+0000", 2024-09-11T06:50:08.812 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020955100000000001 2024-09-11T06:50:08.813 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.813 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.813 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:08.813 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453815+0000", 2024-09-11T06:50:08.813 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8858999999999997e-05 2024-09-11T06:50:08.813 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:08.813 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:08.813 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:08.813 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453882+0000", 2024-09-11T06:50:08.813 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.6556000000000004e-05 2024-09-11T06:50:08.813 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.813 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:08.814 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.814 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:08.814 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:08.814 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:08.816 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:50:08.816 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:08.816+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:08.916 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:08.916+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:08.928 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.4 to 1 (10000) 2024-09-11T06:50:08.953 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-11T06:50:08.986 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:08.986 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:08.987 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:50:08.987 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:09.000 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:50:09.045 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:09.044+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:09.074 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:50:09.118 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:09.116+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:09.118 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:09.118 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:09.118 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:50:09.119 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:50:09.119 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:09.133 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:50:09.212 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:50:09.212 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:50:09.218 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:09.216+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:09.229 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:50:09.229 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:50:09.247 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:50:09.247 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:50:09.266 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:50:09.266 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:50:09.286 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:50:09.286 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:50:09.295 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:09.295 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:09.307 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:50:09.307 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:50:09.309 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:50:09.319 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:09.316+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:09.419 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:09.420+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:09.434 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:09.434 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:09.449 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:50:09.535 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:09.536+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:09.615 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:09.616 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:09.622 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:09.620+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:09.630 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:50:09.723 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:09.720+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:09.791 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:09.791 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:09.806 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:50:09.853 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:09.852+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:09.925 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:09.924+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:09.931 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.5 to 1 (10000) 2024-09-11T06:50:09.939 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:09.939 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:09.956 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:50:09.959 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-11T06:50:09.960 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T06:50:10.053 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:10.052+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:10.114 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:10.115 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:10.127 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:10.124+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:10.131 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:50:10.228 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:10.224+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:10.285 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:10.286 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:10.298 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:50:10.341 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:10.340+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:10.429 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:10.428+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:10.456 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:10.456 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:10.469 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:50:10.530 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:10.528+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:10.577 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:10.577 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:10.589 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:50:10.631 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:10.632+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:10.707 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:10.707 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:10.721 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:50:10.732 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:10.732+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:10.833 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:10.832+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:10.838 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:10.838 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:10.852 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:50:10.934 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:10.932+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:10.974 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:10.974 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:10.988 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:50:11.034 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:11.032+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:11.036 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.1 to 1 (10000) 2024-09-11T06:50:11.044 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is set 2024-09-11T06:50:11.061 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-11T06:50:11.061 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:50:11.110 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:11.110 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:11.124 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:50:11.171 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:11.172+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:11.236 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:11.236+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:11.254 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:11.255 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:11.269 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:50:11.323 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:50:11.369 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:11.368+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:11.382 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:50:11.383 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:50:11.400 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-09-11T06:50:11.039568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1771161,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":167256,"kb_used_data":5060,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566063784,"statfs":{"total":579820584960,"available":579649314816,"internally_reserved":0,"allocated":5181440,"data_stored":2994956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9565,"internal_metadata":165927587},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.013858"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T06:50:08.932993+0000","last_change":"2024-09-11T06:49:13.393788+0000","last_active":"2024-09-11T06:50:08.932993+0000","last_peered":"2024-09-11T06:50:08.932993+0000","last_clean":"2024-09-11T06:50:08.932993+0000","last_became_active":"2024-09-11T06:49:11.433437+0000","last_became_peered":"2024-09-11T06:49:11.433437+0000","last_unstale":"2024-09-11T06:50:08.932993+0000","last_undegraded":"2024-09-11T06:50:08.932993+0000","last_fullsized":"2024-09-11T06:50:08.932993+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:20:04.339208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001403348,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":28,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T06:50:08.934792+0000","last_change":"2024-09-11T06:49:55.948281+0000","last_active":"2024-09-11T06:50:08.934792+0000","last_peered":"2024-09-11T06:50:08.934792+0000","last_clean":"2024-09-11T06:50:08.934792+0000","last_became_active":"2024-09-11T06:49:55.948006+0000","last_became_peered":"2024-09-11T06:49:55.948006+0000","last_unstale":"2024-09-11T06:50:08.934792+0000","last_undegraded":"2024-09-11T06:50:08.934792+0000","last_fullsized":"2024-09-11T06:50:08.934792+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:10:41.665226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":95,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T06:50:08.932956+0000","last_change":"2024-09-11T06:50:06.933758+0000","last_active":"2024-09-11T06:50:08.932956+0000","last_peered":"2024-09-11T06:50:08.932956+0000","last_clean":"2024-09-11T06:50:08.932956+0000","last_became_active":"2024-09-11T06:50:06.932797+0000","last_became_peered":"2024-09-11T06:50:06.932797+0000","last_unstale":"2024-09-11T06:50:08.932956+0000","last_undegraded":"2024-09-11T06:50:08.932956+0000","last_fullsized":"2024-09-11T06:50:08.932956+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:22:15.086364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095172900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T06:50:08.934667+0000","last_change":"2024-09-11T06:49:56.122331+0000","last_active":"2024-09-11T06:50:08.934667+0000","last_peered":"2024-09-11T06:50:08.934667+0000","last_clean":"2024-09-11T06:50:08.934667+0000","last_became_active":"2024-09-11T06:49:56.121998+0000","last_became_peered":"2024-09-11T06:49:56.121998+0000","last_unstale":"2024-09-11T06:50:08.934667+0000","last_undegraded":"2024-09-11T06:50:08.934667+0000","last_fullsized":"2024-09-11T06:50:08.934667+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:24.336670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":112,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T06:50:08.938334+0000","last_change":"2024-09-11T06:49:55.961845+0000","last_active":"2024-09-11T06:50:08.938334+0000","last_peered":"2024-09-11T06:50:08.938334+0000","last_clean":"2024-09-11T06:50:08.938334+0000","last_became_active":"2024-09-11T06:49:55.955837+0000","last_became_peered":"2024-09-11T06:49:55.955837+0000","last_unstale":"2024-09-11T06:50:08.938334+0000","last_undegraded":"2024-09-11T06:50:08.938334+0000","last_fullsized":"2024-09-11T06:50:08.938334+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:05:32.670233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001910244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":95,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T06:50:08.931800+0000","last_change":"2024-09-11T06:49:55.948809+0000","last_active":"2024-09-11T06:50:08.931800+0000","last_peered":"2024-09-11T06:50:08.931800+0000","last_clean":"2024-09-11T06:50:08.931800+0000","last_became_active":"2024-09-11T06:49:55.948499+0000","last_became_peered":"2024-09-11T06:49:55.948499+0000","last_unstale":"2024-09-11T06:50:08.931800+0000","last_undegraded":"2024-09-11T06:50:08.931800+0000","last_fullsized":"2024-09-11T06:50:08.931800+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:43:31.563009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010163780000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":100,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T06:50:08.932912+0000","last_change":"2024-09-11T06:49:55.951569+0000","last_active":"2024-09-11T06:50:08.932912+0000","last_peered":"2024-09-11T06:50:08.932912+0000","last_clean":"2024-09-11T06:50:08.932912+0000","last_became_active":"2024-09-11T06:49:55.950989+0000","last_became_peered":"2024-09-11T06:49:55.950989+0000","last_unstale":"2024-09-11T06:50:08.932912+0000","last_undegraded":"2024-09-11T06:50:08.932912+0000","last_fullsized":"2024-09-11T06:50:08.932912+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:07.743579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077615599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T06:50:08.938334+0000","last_change":"2024-09-11T06:49:13.251115+0000","last_active":"2024-09-11T06:50:08.938334+0000","last_peered":"2024-09-11T06:50:08.938334+0000","last_clean":"2024-09-11T06:50:08.938334+0000","last_became_active":"2024-09-11T06:49:11.235231+0000","last_became_peered":"2024-09-11T06:49:11.235231+0000","last_unstale":"2024-09-11T06:50:08.938334+0000","last_undegraded":"2024-09-11T06:50:08.938334+0000","last_fullsized":"2024-09-11T06:50:08.938334+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:01:21.028466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011212469999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":269,"reported_epoch":50,"state":"peering","last_fresh":"2024-09-11T06:50:07.946328+0000","last_change":"2024-09-11T06:50:08.933378+0000","last_active":"2024-09-11T06:50:08.933378+0000","last_peered":"2024-09-11T06:50:07.946328+0000","last_clean":"2024-09-11T06:50:05.011113+0000","last_became_active":"2024-09-11T06:50:07.937998+0000","last_became_peered":"2024-09-11T06:50:07.937998+0000","last_unstale":"2024-09-11T06:50:07.946328+0000","last_undegraded":"2024-09-11T06:50:07.946328+0000","last_fullsized":"2024-09-11T06:50:07.931924+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_clean_scrub_stamp":"2024-09-11T06:49:05.088839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640271,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":569344,"data_stored":203988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640272,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":565248,"data_stored":203969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640272,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28160,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343680,"statfs":{"total":96636764160,"available":96607928320,"internally_reserved":0,"allocated":1159168,"data_stored":794337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640273,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":565248,"data_stored":203969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640273,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28164,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343676,"statfs":{"total":96636764160,"available":96607924224,"internally_reserved":0,"allocated":1163264,"data_stored":794356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640274,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28168,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343672,"statfs":{"total":96636764160,"available":96607920128,"internally_reserved":0,"allocated":1159168,"data_stored":794337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:50:11.401 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:50:11.440 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:11.441 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:11.441 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:11.440+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:11.455 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:50:11.458 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:50:11.458 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:50:11.473 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:50:11.473 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:50:11.489 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:50:11.489 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:50:11.506 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:50:11.506 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:50:11.523 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:50:11.523 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:50:11.538 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:11.536+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:11.539 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:50:11.540 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:50:11.577 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:11.577 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:11.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:11.577 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:50:11.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.577 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:50:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 65.404352770000003, 2024-09-11T06:50:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011420669, 2024-09-11T06:50:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 37 2024-09-11T06:50:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:50:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:50:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170509+0000", 2024-09-11T06:50:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.13e-06 2024-09-11T06:50:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170521+0000", 2024-09-11T06:50:11.580 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1776999999999999e-05 2024-09-11T06:50:11.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.580 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:11.580 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170538+0000", 2024-09-11T06:50:11.580 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7413000000000001e-05 2024-09-11T06:50:11.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.580 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:11.580 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170570+0000", 2024-09-11T06:50:11.580 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2313e-05 2024-09-11T06:50:11.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.580 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:11.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179768+0000", 2024-09-11T06:50:11.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0091978010000000002 2024-09-11T06:50:11.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.581 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:11.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179951+0000", 2024-09-11T06:50:11.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018227900000000001 2024-09-11T06:50:11.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.581 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:11.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.180175+0000", 2024-09-11T06:50:11.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022432800000000001 2024-09-11T06:50:11.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:11.582 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181112+0000", 2024-09-11T06:50:11.582 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00093712500000000002 2024-09-11T06:50:11.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:11.582 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181789+0000", 2024-09-11T06:50:11.582 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00067705100000000002 2024-09-11T06:50:11.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:11.582 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181871+0000", 2024-09-11T06:50:11.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1816000000000004e-05 2024-09-11T06:50:11.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:11.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181929+0000", 2024-09-11T06:50:11.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7636000000000002e-05 2024-09-11T06:50:11.583 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.583 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:11.583 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.583 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:11.583 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:11.583 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 65.303093809000003, 2024-09-11T06:50:11.584 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283336, 2024-09-11T06:50:11.584 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:11.584 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:11.584 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:11.584 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:11.584 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:11.584 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:11.584 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 87 2024-09-11T06:50:11.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.584 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:11.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.584 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:11.584 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:11.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.585 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:11.585 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:11.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.585 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271768+0000", 2024-09-11T06:50:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7599999999999996e-07 2024-09-11T06:50:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271860+0000", 2024-09-11T06:50:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2639999999999994e-05 2024-09-11T06:50:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271873+0000", 2024-09-11T06:50:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3139000000000001e-05 2024-09-11T06:50:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271882+0000", 2024-09-11T06:50:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2369999999999992e-06 2024-09-11T06:50:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277150+0000", 2024-09-11T06:50:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052684419999999999 2024-09-11T06:50:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277273+0000", 2024-09-11T06:50:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012278899999999999 2024-09-11T06:50:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.588 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:11.588 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277466+0000", 2024-09-11T06:50:11.588 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193578 2024-09-11T06:50:11.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.588 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:11.588 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.280808+0000", 2024-09-11T06:50:11.588 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003341971 2024-09-11T06:50:11.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.588 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:11.588 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284419+0000", 2024-09-11T06:50:11.589 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003610982 2024-09-11T06:50:11.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.589 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:11.589 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284531+0000", 2024-09-11T06:50:11.589 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011121299999999999 2024-09-11T06:50:11.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.589 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:11.589 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284600+0000", 2024-09-11T06:50:11.589 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9592999999999996e-05 2024-09-11T06:50:11.589 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.590 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:11.590 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.590 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:11.590 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:11.590 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 65.302963556999998, 2024-09-11T06:50:11.590 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283314, 2024-09-11T06:50:11.590 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:11.590 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:11.590 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:11.590 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:11.590 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:11.591 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:11.591 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 88 2024-09-11T06:50:11.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.591 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:11.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.591 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:11.591 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:11.591 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.591 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:11.591 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:11.591 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.592 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:11.592 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271898+0000", 2024-09-11T06:50:11.592 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2999999999999999e-07 2024-09-11T06:50:11.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.592 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:11.592 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271904+0000", 2024-09-11T06:50:11.592 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.767e-06 2024-09-11T06:50:11.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.592 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:11.593 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271913+0000", 2024-09-11T06:50:11.593 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7200000000000001e-06 2024-09-11T06:50:11.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.593 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:11.593 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271920+0000", 2024-09-11T06:50:11.593 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7100000000000001e-06 2024-09-11T06:50:11.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.593 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:11.593 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278684+0000", 2024-09-11T06:50:11.596 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0067642650000000002 2024-09-11T06:50:11.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.596 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:11.597 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278838+0000", 2024-09-11T06:50:11.597 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015371499999999999 2024-09-11T06:50:11.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.597 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:11.597 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.279040+0000", 2024-09-11T06:50:11.597 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020207499999999999 2024-09-11T06:50:11.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.597 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:11.597 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.281448+0000", 2024-09-11T06:50:11.597 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002407686 2024-09-11T06:50:11.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.598 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:11.598 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284615+0000", 2024-09-11T06:50:11.598 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031675499999999999 2024-09-11T06:50:11.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.598 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:11.598 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284682+0000", 2024-09-11T06:50:11.598 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7032000000000005e-05 2024-09-11T06:50:11.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.598 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:11.598 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284730+0000", 2024-09-11T06:50:11.598 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7790000000000002e-05 2024-09-11T06:50:11.598 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.599 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:11.599 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.599 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:11.599 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:50:11.599 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 65.135062335000001, 2024-09-11T06:50:11.599 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011114877, 2024-09-11T06:50:11.599 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:11.599 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:11.599 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:11.599 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:11.599 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:11.600 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:11.600 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 107 2024-09-11T06:50:11.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.600 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:11.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.600 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:11.600 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:50:11.600 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.600 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:11.600 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:50:11.600 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.601 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:11.601 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439799+0000", 2024-09-11T06:50:11.601 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.16e-07 2024-09-11T06:50:11.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.601 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:11.601 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439806+0000", 2024-09-11T06:50:11.601 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3640000000000004e-06 2024-09-11T06:50:11.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.601 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:11.601 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439816+0000", 2024-09-11T06:50:11.601 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0485999999999999e-05 2024-09-11T06:50:11.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.602 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:11.602 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439824+0000", 2024-09-11T06:50:11.602 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.6699999999999994e-06 2024-09-11T06:50:11.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.602 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:11.602 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443328+0000", 2024-09-11T06:50:11.602 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035044329999999999 2024-09-11T06:50:11.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.602 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:11.602 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443461+0000", 2024-09-11T06:50:11.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013272199999999999 2024-09-11T06:50:11.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.603 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:11.603 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443678+0000", 2024-09-11T06:50:11.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021674500000000001 2024-09-11T06:50:11.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.603 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:11.603 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449635+0000", 2024-09-11T06:50:11.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0059572119999999999 2024-09-11T06:50:11.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.604 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:11.604 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450750+0000", 2024-09-11T06:50:11.604 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001115654 2024-09-11T06:50:11.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.604 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:11.604 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450832+0000", 2024-09-11T06:50:11.604 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1188999999999998e-05 2024-09-11T06:50:11.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.604 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:11.604 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450913+0000", 2024-09-11T06:50:11.604 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1485999999999999e-05 2024-09-11T06:50:11.604 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.605 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:11.605 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.605 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:11.605 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:50:11.605 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 65.134471360000006, 2024-09-11T06:50:11.605 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011236556, 2024-09-11T06:50:11.605 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:11.605 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:11.605 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:11.605 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:11.605 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:11.606 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:11.606 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 109 2024-09-11T06:50:11.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.606 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:11.606 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.606 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:11.606 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:50:11.606 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.606 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.606 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:11.606 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:50:11.606 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.607 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:11.607 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440391+0000", 2024-09-11T06:50:11.607 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8360000000000001e-06 2024-09-11T06:50:11.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.607 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:11.607 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440560+0000", 2024-09-11T06:50:11.607 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000169085 2024-09-11T06:50:11.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.607 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:11.607 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440585+0000", 2024-09-11T06:50:11.607 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4530000000000001e-05 2024-09-11T06:50:11.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.608 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:11.608 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440606+0000", 2024-09-11T06:50:11.608 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1277e-05 2024-09-11T06:50:11.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.608 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:11.608 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446479+0000", 2024-09-11T06:50:11.608 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0058734989999999999 2024-09-11T06:50:11.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.608 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:11.608 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446649+0000", 2024-09-11T06:50:11.609 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016948199999999999 2024-09-11T06:50:11.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.609 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:11.609 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446854+0000", 2024-09-11T06:50:11.609 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020491500000000001 2024-09-11T06:50:11.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.609 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:11.609 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451380+0000", 2024-09-11T06:50:11.609 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0045264110000000001 2024-09-11T06:50:11.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.609 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:11.610 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451479+0000", 2024-09-11T06:50:11.610 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.8975999999999996e-05 2024-09-11T06:50:11.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.610 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:11.610 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451562+0000", 2024-09-11T06:50:11.610 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3213000000000004e-05 2024-09-11T06:50:11.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.610 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:11.610 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451626+0000", 2024-09-11T06:50:11.610 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3331999999999996e-05 2024-09-11T06:50:11.610 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.610 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:11.611 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.611 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.611 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.611 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:11.611 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:50:11.611 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 65.134222231999999, 2024-09-11T06:50:11.611 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011252866, 2024-09-11T06:50:11.611 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:11.611 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:11.611 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:11.611 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:11.611 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:11.611 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:11.611 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 110 2024-09-11T06:50:11.612 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.612 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:11.612 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.612 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:11.612 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:50:11.612 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.612 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.612 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.612 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:11.612 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:50:11.612 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.612 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.612 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.612 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:11.613 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440640+0000", 2024-09-11T06:50:11.613 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5430000000000001e-06 2024-09-11T06:50:11.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.613 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.613 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:11.613 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440651+0000", 2024-09-11T06:50:11.613 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0672999999999999e-05 2024-09-11T06:50:11.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.613 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.613 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:11.613 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440670+0000", 2024-09-11T06:50:11.613 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9726999999999999e-05 2024-09-11T06:50:11.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.613 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.614 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:11.614 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440691+0000", 2024-09-11T06:50:11.614 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0526e-05 2024-09-11T06:50:11.614 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.614 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.614 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:11.614 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448191+0000", 2024-09-11T06:50:11.614 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0075005990000000002 2024-09-11T06:50:11.614 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.614 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.614 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:11.614 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448390+0000", 2024-09-11T06:50:11.614 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000198815 2024-09-11T06:50:11.614 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.614 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.615 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:11.615 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448735+0000", 2024-09-11T06:50:11.615 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034424999999999999 2024-09-11T06:50:11.615 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.615 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.615 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:11.615 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451742+0000", 2024-09-11T06:50:11.615 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030077440000000001 2024-09-11T06:50:11.615 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.615 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.615 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:11.615 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451761+0000", 2024-09-11T06:50:11.615 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9196999999999999e-05 2024-09-11T06:50:11.615 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.615 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.616 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:11.616 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451834+0000", 2024-09-11T06:50:11.616 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2071999999999997e-05 2024-09-11T06:50:11.616 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.616 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.616 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:11.616 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451891+0000", 2024-09-11T06:50:11.616 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7720000000000003e-05 2024-09-11T06:50:11.616 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.616 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:11.616 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.616 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.616 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.617 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:11.617 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:11.617 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 65.134114357000001, 2024-09-11T06:50:11.617 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014944941, 2024-09-11T06:50:11.617 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:11.617 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:11.617 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:11.617 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:11.617 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:11.617 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:11.617 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 111 2024-09-11T06:50:11.617 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.617 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:11.617 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.618 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:11.618 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:11.618 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.618 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.618 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.618 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:11.618 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:11.618 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.618 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.618 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.618 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:11.618 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440748+0000", 2024-09-11T06:50:11.618 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:50:11.618 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.619 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.619 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:11.619 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440760+0000", 2024-09-11T06:50:11.619 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1897e-05 2024-09-11T06:50:11.619 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.619 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.619 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:11.619 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440782+0000", 2024-09-11T06:50:11.619 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2552999999999999e-05 2024-09-11T06:50:11.619 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.619 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.619 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:11.619 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440793+0000", 2024-09-11T06:50:11.619 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0643e-05 2024-09-11T06:50:11.620 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.620 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.620 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:11.620 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449755+0000", 2024-09-11T06:50:11.620 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089622620000000004 2024-09-11T06:50:11.620 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.620 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.620 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:11.620 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449918+0000", 2024-09-11T06:50:11.620 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163086 2024-09-11T06:50:11.620 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.620 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.620 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:11.620 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450150+0000", 2024-09-11T06:50:11.621 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023204500000000001 2024-09-11T06:50:11.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.621 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.621 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:11.621 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454533+0000", 2024-09-11T06:50:11.621 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0043831349999999998 2024-09-11T06:50:11.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.621 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.621 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:11.621 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455511+0000", 2024-09-11T06:50:11.621 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0009776050000000001 2024-09-11T06:50:11.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.621 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.621 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:11.622 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455597+0000", 2024-09-11T06:50:11.622 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6446000000000005e-05 2024-09-11T06:50:11.622 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.622 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.622 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:11.622 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455691+0000", 2024-09-11T06:50:11.622 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3832000000000006e-05 2024-09-11T06:50:11.622 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.622 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:11.622 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.622 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.622 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.622 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:11.622 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:11.623 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 65.134044594000002, 2024-09-11T06:50:11.623 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018885411000000001, 2024-09-11T06:50:11.623 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:11.623 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:11.623 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:11.623 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:11.623 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:11.623 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:11.623 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 112 2024-09-11T06:50:11.623 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.623 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:11.623 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.623 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:11.623 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:11.624 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.624 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.624 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.624 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:11.624 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:11.624 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.624 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.624 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.624 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:11.624 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440817+0000", 2024-09-11T06:50:11.624 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:50:11.624 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.624 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.624 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:11.625 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441003+0000", 2024-09-11T06:50:11.625 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018661200000000001 2024-09-11T06:50:11.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.625 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:11.625 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441027+0000", 2024-09-11T06:50:11.625 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3920000000000001e-05 2024-09-11T06:50:11.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.625 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:11.625 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441062+0000", 2024-09-11T06:50:11.625 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4663000000000003e-05 2024-09-11T06:50:11.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.626 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:11.626 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451981+0000", 2024-09-11T06:50:11.626 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010918926000000001 2024-09-11T06:50:11.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.626 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.626 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:11.626 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452111+0000", 2024-09-11T06:50:11.626 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129529 2024-09-11T06:50:11.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.626 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.626 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:11.626 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452311+0000", 2024-09-11T06:50:11.626 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020013799999999999 2024-09-11T06:50:11.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.626 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.627 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:11.627 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455809+0000", 2024-09-11T06:50:11.627 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003498443 2024-09-11T06:50:11.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.627 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:11.627 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459566+0000", 2024-09-11T06:50:11.627 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003757271 2024-09-11T06:50:11.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.627 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:11.627 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459640+0000", 2024-09-11T06:50:11.628 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3949000000000006e-05 2024-09-11T06:50:11.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.628 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.628 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:11.628 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459701+0000", 2024-09-11T06:50:11.628 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1117000000000005e-05 2024-09-11T06:50:11.628 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.628 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:11.628 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.628 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.628 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:11.628 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:11.628 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 65.133765742999998, 2024-09-11T06:50:11.629 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018760915, 2024-09-11T06:50:11.629 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:11.629 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:11.629 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:11.629 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:11.629 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:11.629 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:11.629 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 113 2024-09-11T06:50:11.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.629 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:11.629 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.629 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:11.630 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:11.630 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.630 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.630 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.630 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:11.630 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:11.630 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.630 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.630 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.630 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:11.630 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441096+0000", 2024-09-11T06:50:11.630 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4130000000000001e-06 2024-09-11T06:50:11.630 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.630 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.631 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:11.631 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441110+0000", 2024-09-11T06:50:11.631 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3900000000000001e-05 2024-09-11T06:50:11.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.631 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:11.631 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441131+0000", 2024-09-11T06:50:11.631 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0636999999999999e-05 2024-09-11T06:50:11.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.631 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:11.631 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441147+0000", 2024-09-11T06:50:11.631 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6512999999999999e-05 2024-09-11T06:50:11.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.632 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:11.632 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453274+0000", 2024-09-11T06:50:11.632 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012126764999999999 2024-09-11T06:50:11.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.632 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:11.632 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453498+0000", 2024-09-11T06:50:11.632 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000223568 2024-09-11T06:50:11.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.632 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:11.632 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453872+0000", 2024-09-11T06:50:11.633 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037467099999999998 2024-09-11T06:50:11.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.633 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:11.633 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458709+0000", 2024-09-11T06:50:11.633 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0048365680000000003 2024-09-11T06:50:11.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.633 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:11.633 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459716+0000", 2024-09-11T06:50:11.633 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010075240000000001 2024-09-11T06:50:11.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.633 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:11.634 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459779+0000", 2024-09-11T06:50:11.634 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2336999999999996e-05 2024-09-11T06:50:11.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.634 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.634 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:11.634 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459856+0000", 2024-09-11T06:50:11.634 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7019000000000002e-05 2024-09-11T06:50:11.634 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.634 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:11.634 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.634 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.634 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:11.634 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:11.635 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 65.133689004000004, 2024-09-11T06:50:11.635 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018963114, 2024-09-11T06:50:11.635 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:11.635 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:11.635 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:11.635 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:11.635 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:11.635 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:11.635 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 114 2024-09-11T06:50:11.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.635 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:11.635 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.635 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:11.635 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:11.635 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.636 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.636 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:11.636 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:11.636 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.637 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:11.637 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:11.637 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9299999999999996e-07 2024-09-11T06:50:11.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.637 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:11.637 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441180+0000", 2024-09-11T06:50:11.637 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7200000000000006e-06 2024-09-11T06:50:11.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.637 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:11.637 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441265+0000", 2024-09-11T06:50:11.638 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4753000000000007e-05 2024-09-11T06:50:11.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.638 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:11.638 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441288+0000", 2024-09-11T06:50:11.638 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2860000000000001e-05 2024-09-11T06:50:11.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.638 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:11.638 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454585+0000", 2024-09-11T06:50:11.638 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013297286 2024-09-11T06:50:11.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.639 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:11.639 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454714+0000", 2024-09-11T06:50:11.639 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000128465 2024-09-11T06:50:11.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.639 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:11.639 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454909+0000", 2024-09-11T06:50:11.639 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019522499999999999 2024-09-11T06:50:11.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.639 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:11.639 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459868+0000", 2024-09-11T06:50:11.639 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0049591239999999997 2024-09-11T06:50:11.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.640 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.640 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:11.640 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459988+0000", 2024-09-11T06:50:11.640 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000120532 2024-09-11T06:50:11.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.640 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.640 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:11.640 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460071+0000", 2024-09-11T06:50:11.640 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2256000000000007e-05 2024-09-11T06:50:11.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.640 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.640 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:11.640 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460135+0000", 2024-09-11T06:50:11.641 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3999999999999997e-05 2024-09-11T06:50:11.641 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.641 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:11.641 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.641 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:11.641 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:11.641 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 65.133531375000004, 2024-09-11T06:50:11.641 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.019036849000000002, 2024-09-11T06:50:11.641 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:11.641 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:11.641 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:11.642 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:11.642 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:11.642 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:11.642 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 115 2024-09-11T06:50:11.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.642 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:11.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.642 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:11.642 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:11.642 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.642 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:11.642 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:11.643 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.643 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:11.643 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441330+0000", 2024-09-11T06:50:11.643 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1930000000000001e-06 2024-09-11T06:50:11.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.643 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:11.643 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441544+0000", 2024-09-11T06:50:11.643 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021405799999999999 2024-09-11T06:50:11.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.644 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:11.644 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441572+0000", 2024-09-11T06:50:11.644 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7317e-05 2024-09-11T06:50:11.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.644 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:11.644 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441593+0000", 2024-09-11T06:50:11.644 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1702999999999999e-05 2024-09-11T06:50:11.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.644 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:11.644 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455891+0000", 2024-09-11T06:50:11.644 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01429708 2024-09-11T06:50:11.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:11.645 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456046+0000", 2024-09-11T06:50:11.645 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000155682 2024-09-11T06:50:11.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:11.645 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456242+0000", 2024-09-11T06:50:11.645 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000195692 2024-09-11T06:50:11.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:11.645 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459876+0000", 2024-09-11T06:50:11.645 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036336480000000002 2024-09-11T06:50:11.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.646 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:11.646 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460245+0000", 2024-09-11T06:50:11.646 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036901700000000002 2024-09-11T06:50:11.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.646 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:11.646 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460316+0000", 2024-09-11T06:50:11.646 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1846000000000003e-05 2024-09-11T06:50:11.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.646 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:11.647 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460366+0000", 2024-09-11T06:50:11.647 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9613000000000001e-05 2024-09-11T06:50:11.647 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.647 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:11.647 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.647 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:11.647 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:11.647 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 65.133209295, 2024-09-11T06:50:11.647 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01904432, 2024-09-11T06:50:11.647 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:11.647 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:11.647 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:11.648 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:11.648 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:11.648 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:11.648 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 116 2024-09-11T06:50:11.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.648 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:11.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.648 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:11.648 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:11.648 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.648 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:11.648 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:11.649 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.649 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:11.649 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441653+0000", 2024-09-11T06:50:11.649 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:50:11.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.649 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:11.649 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441665+0000", 2024-09-11T06:50:11.649 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1653e-05 2024-09-11T06:50:11.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.649 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:11.650 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441684+0000", 2024-09-11T06:50:11.650 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9490000000000001e-05 2024-09-11T06:50:11.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.650 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:11.650 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441699+0000", 2024-09-11T06:50:11.650 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5044e-05 2024-09-11T06:50:11.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.650 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:11.650 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457297+0000", 2024-09-11T06:50:11.650 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015598241000000001 2024-09-11T06:50:11.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.651 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:11.651 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457577+0000", 2024-09-11T06:50:11.651 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027930799999999998 2024-09-11T06:50:11.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.651 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:11.651 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457957+0000", 2024-09-11T06:50:11.651 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00038088 2024-09-11T06:50:11.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.651 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:11.651 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460432+0000", 2024-09-11T06:50:11.651 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024743349999999998 2024-09-11T06:50:11.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.652 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:11.652 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460584+0000", 2024-09-11T06:50:11.652 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015232900000000001 2024-09-11T06:50:11.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.652 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:11.652 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460646+0000", 2024-09-11T06:50:11.652 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1883000000000004e-05 2024-09-11T06:50:11.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.652 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:11.652 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460696+0000", 2024-09-11T06:50:11.652 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9589999999999998e-05 2024-09-11T06:50:11.653 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.653 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:11.653 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.653 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:11.653 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:11.653 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 65.133137228999999, 2024-09-11T06:50:11.653 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.021834473, 2024-09-11T06:50:11.653 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:11.653 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:11.653 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:11.653 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:11.653 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:11.654 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:11.654 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 117 2024-09-11T06:50:11.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.654 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:11.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.654 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:11.654 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:11.654 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.654 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:11.654 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:11.654 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.655 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:11.655 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441725+0000", 2024-09-11T06:50:11.655 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5069999999999999e-06 2024-09-11T06:50:11.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.655 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:11.655 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441738+0000", 2024-09-11T06:50:11.655 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2813000000000001e-05 2024-09-11T06:50:11.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.655 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:11.655 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441759+0000", 2024-09-11T06:50:11.655 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1246999999999998e-05 2024-09-11T06:50:11.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.656 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:11.656 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441779+0000", 2024-09-11T06:50:11.656 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0392999999999999e-05 2024-09-11T06:50:11.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.656 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:11.656 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458804+0000", 2024-09-11T06:50:11.656 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.017024182999999998 2024-09-11T06:50:11.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.656 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:11.656 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458977+0000", 2024-09-11T06:50:11.657 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000173926 2024-09-11T06:50:11.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.657 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:11.657 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459170+0000", 2024-09-11T06:50:11.657 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000192184 2024-09-11T06:50:11.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.657 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:11.657 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.462300+0000", 2024-09-11T06:50:11.657 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031304969999999999 2024-09-11T06:50:11.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.657 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:11.658 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463408+0000", 2024-09-11T06:50:11.658 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011077839999999999 2024-09-11T06:50:11.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.658 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:11.658 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463497+0000", 2024-09-11T06:50:11.658 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8828999999999999e-05 2024-09-11T06:50:11.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.658 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:11.658 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463558+0000", 2024-09-11T06:50:11.658 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1110000000000003e-05 2024-09-11T06:50:11.658 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.658 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:11.658 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.659 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:11.659 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:50:11.659 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 64.756629540999995, 2024-09-11T06:50:11.659 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012216898, 2024-09-11T06:50:11.659 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:11.659 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:11.659 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:11.659 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:11.659 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:11.659 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:11.659 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 137 2024-09-11T06:50:11.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.660 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:11.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.660 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:11.660 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:50:11.660 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.660 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:11.660 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:50:11.660 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.661 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:11.661 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818232+0000", 2024-09-11T06:50:11.661 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2839999999999999e-06 2024-09-11T06:50:11.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.661 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.661 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:11.661 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818326+0000", 2024-09-11T06:50:11.661 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3531999999999998e-05 2024-09-11T06:50:11.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.661 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.661 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:11.661 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818337+0000", 2024-09-11T06:50:11.661 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0655999999999999e-05 2024-09-11T06:50:11.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.662 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.662 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:11.662 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818342+0000", 2024-09-11T06:50:11.662 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4600000000000002e-06 2024-09-11T06:50:11.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.662 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.662 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:11.662 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823890+0000", 2024-09-11T06:50:11.662 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0055480700000000004 2024-09-11T06:50:11.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.662 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.662 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:11.662 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823953+0000", 2024-09-11T06:50:11.662 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2848999999999994e-05 2024-09-11T06:50:11.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.663 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:11.663 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824074+0000", 2024-09-11T06:50:11.663 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121309 2024-09-11T06:50:11.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.663 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:11.663 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828043+0000", 2024-09-11T06:50:11.663 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0039685240000000002 2024-09-11T06:50:11.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.663 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:11.663 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830293+0000", 2024-09-11T06:50:11.664 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0022503179999999998 2024-09-11T06:50:11.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.664 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.664 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:11.664 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830366+0000", 2024-09-11T06:50:11.664 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2539999999999993e-05 2024-09-11T06:50:11.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.664 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.664 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:11.664 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830448+0000", 2024-09-11T06:50:11.664 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2355999999999996e-05 2024-09-11T06:50:11.664 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.664 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:11.664 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.665 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.665 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.665 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:11.665 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:11.665 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 64.756495727000001, 2024-09-11T06:50:11.665 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013136297, 2024-09-11T06:50:11.665 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:11.665 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:11.665 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:11.665 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:11.665 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:11.665 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:11.665 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 138 2024-09-11T06:50:11.665 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.665 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:11.666 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.666 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:11.666 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:11.666 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.666 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.666 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.666 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:11.666 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:11.666 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.666 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.666 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.666 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:11.666 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:11.666 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.89e-07 2024-09-11T06:50:11.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.667 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.667 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:11.667 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818369+0000", 2024-09-11T06:50:11.667 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4170000000000001e-06 2024-09-11T06:50:11.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.667 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.667 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:11.667 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818376+0000", 2024-09-11T06:50:11.667 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4270000000000002e-06 2024-09-11T06:50:11.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.667 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.667 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:11.668 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818383+0000", 2024-09-11T06:50:11.668 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3060000000000003e-06 2024-09-11T06:50:11.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.668 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:11.668 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824783+0000", 2024-09-11T06:50:11.668 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064003020000000001 2024-09-11T06:50:11.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.668 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:11.668 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825013+0000", 2024-09-11T06:50:11.668 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023039499999999999 2024-09-11T06:50:11.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.669 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:11.669 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825484+0000", 2024-09-11T06:50:11.669 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000470562 2024-09-11T06:50:11.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.669 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.669 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:11.669 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828633+0000", 2024-09-11T06:50:11.669 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031496689999999999 2024-09-11T06:50:11.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.669 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.669 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:11.669 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831413+0000", 2024-09-11T06:50:11.669 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027793219999999999 2024-09-11T06:50:11.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.670 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:11.670 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831450+0000", 2024-09-11T06:50:11.670 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7370999999999998e-05 2024-09-11T06:50:11.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.670 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:11.670 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831501+0000", 2024-09-11T06:50:11.670 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1036999999999999e-05 2024-09-11T06:50:11.670 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.670 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:11.670 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.671 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:11.671 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:11.671 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 64.756471246999993, 2024-09-11T06:50:11.671 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013223011, 2024-09-11T06:50:11.671 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:11.671 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:11.671 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:11.671 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:11.671 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:11.671 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:11.671 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 139 2024-09-11T06:50:11.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.671 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:11.671 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.672 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:11.672 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:11.672 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.672 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.672 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:11.672 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:11.672 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.672 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.672 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:11.672 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818390+0000", 2024-09-11T06:50:11.672 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5600000000000001e-07 2024-09-11T06:50:11.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.673 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:11.673 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818394+0000", 2024-09-11T06:50:11.673 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5929999999999999e-06 2024-09-11T06:50:11.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.673 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:11.673 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818399+0000", 2024-09-11T06:50:11.673 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7740000000000004e-06 2024-09-11T06:50:11.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.673 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:11.673 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818412+0000", 2024-09-11T06:50:11.673 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2625e-05 2024-09-11T06:50:11.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.674 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.674 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:11.674 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826205+0000", 2024-09-11T06:50:11.674 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0077930730000000002 2024-09-11T06:50:11.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.674 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.674 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:11.674 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826281+0000", 2024-09-11T06:50:11.674 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5876000000000003e-05 2024-09-11T06:50:11.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.674 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.674 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:11.674 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826384+0000", 2024-09-11T06:50:11.675 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103838 2024-09-11T06:50:11.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.675 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.675 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:11.675 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830642+0000", 2024-09-11T06:50:11.675 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004257083 2024-09-11T06:50:11.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.675 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.675 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:11.675 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831509+0000", 2024-09-11T06:50:11.675 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086730900000000003 2024-09-11T06:50:11.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.675 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.675 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:11.676 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831547+0000", 2024-09-11T06:50:11.676 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7793000000000002e-05 2024-09-11T06:50:11.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.676 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:11.676 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831612+0000", 2024-09-11T06:50:11.676 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5691000000000005e-05 2024-09-11T06:50:11.676 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.676 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:11.676 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.676 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:11.676 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:11.676 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 64.756442552999999, 2024-09-11T06:50:11.677 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016098024999999998, 2024-09-11T06:50:11.677 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:11.677 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:11.677 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:11.677 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:11.677 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:11.677 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:11.677 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 140 2024-09-11T06:50:11.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.677 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:11.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.677 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:11.677 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:11.677 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.678 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.678 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:11.678 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:11.678 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.678 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.678 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:11.678 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:11.678 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5199999999999998e-07 2024-09-11T06:50:11.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.678 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.678 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:11.678 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818479+0000", 2024-09-11T06:50:11.678 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0368000000000002e-05 2024-09-11T06:50:11.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.679 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:11.679 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818487+0000", 2024-09-11T06:50:11.679 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2339999999999994e-06 2024-09-11T06:50:11.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.679 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:11.679 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818491+0000", 2024-09-11T06:50:11.679 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4329999999999997e-06 2024-09-11T06:50:11.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.679 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:11.680 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828080+0000", 2024-09-11T06:50:11.680 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0095890449999999992 2024-09-11T06:50:11.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.680 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:11.680 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828130+0000", 2024-09-11T06:50:11.680 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9239000000000001e-05 2024-09-11T06:50:11.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.680 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:11.680 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828208+0000", 2024-09-11T06:50:11.680 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.852e-05 2024-09-11T06:50:11.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.681 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:11.681 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831625+0000", 2024-09-11T06:50:11.681 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034169410000000002 2024-09-11T06:50:11.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.681 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:11.681 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834411+0000", 2024-09-11T06:50:11.681 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002786111 2024-09-11T06:50:11.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.681 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:11.681 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834457+0000", 2024-09-11T06:50:11.681 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6041000000000002e-05 2024-09-11T06:50:11.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.682 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:11.682 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834516+0000", 2024-09-11T06:50:11.682 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8740999999999999e-05 2024-09-11T06:50:11.682 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.682 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:11.682 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.682 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:11.682 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:11.682 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 64.756361486000003, 2024-09-11T06:50:11.682 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016588772000000002, 2024-09-11T06:50:11.683 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:11.683 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:11.683 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:11.683 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:11.683 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:11.683 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:11.683 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 141 2024-09-11T06:50:11.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.683 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:11.683 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.683 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:11.683 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:11.683 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.684 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:11.684 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:11.684 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.684 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:11.684 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818500+0000", 2024-09-11T06:50:11.684 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0800000000000005e-07 2024-09-11T06:50:11.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.684 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:11.684 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818505+0000", 2024-09-11T06:50:11.685 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2429999999999996e-06 2024-09-11T06:50:11.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.685 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:11.685 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818510+0000", 2024-09-11T06:50:11.685 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9339999999999997e-06 2024-09-11T06:50:11.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.685 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:11.685 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818514+0000", 2024-09-11T06:50:11.685 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7270000000000001e-06 2024-09-11T06:50:11.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.686 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:11.686 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828711+0000", 2024-09-11T06:50:11.686 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010197031 2024-09-11T06:50:11.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.686 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:11.686 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828853+0000", 2024-09-11T06:50:11.686 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014201200000000001 2024-09-11T06:50:11.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.686 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:11.686 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.829115+0000", 2024-09-11T06:50:11.686 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026257500000000003 2024-09-11T06:50:11.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.687 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:11.687 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831629+0000", 2024-09-11T06:50:11.687 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025138169999999998 2024-09-11T06:50:11.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.687 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:11.687 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834991+0000", 2024-09-11T06:50:11.687 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033624459999999998 2024-09-11T06:50:11.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.687 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:11.687 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835032+0000", 2024-09-11T06:50:11.687 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1001000000000003e-05 2024-09-11T06:50:11.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.688 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:11.688 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835088+0000", 2024-09-11T06:50:11.688 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5478e-05 2024-09-11T06:50:11.688 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.688 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:11.688 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.688 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.688 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:50:11.688 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:50:11.688 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 64.132726582999993, 2024-09-11T06:50:11.688 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011161912, 2024-09-11T06:50:11.689 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:11.689 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:11.689 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:11.689 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:11.689 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:11.689 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:11.689 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 158 2024-09-11T06:50:11.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.689 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:11.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.689 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:11.689 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:50:11.689 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.690 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:11.690 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:50:11.690 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.690 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:11.690 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442135+0000", 2024-09-11T06:50:11.690 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3e-07 2024-09-11T06:50:11.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.690 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:11.690 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442259+0000", 2024-09-11T06:50:11.691 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000123903 2024-09-11T06:50:11.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.691 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:11.691 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442289+0000", 2024-09-11T06:50:11.691 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0833000000000001e-05 2024-09-11T06:50:11.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.691 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:11.691 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442301+0000", 2024-09-11T06:50:11.691 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1433e-05 2024-09-11T06:50:11.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.691 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:11.691 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449796+0000", 2024-09-11T06:50:11.692 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0074954959999999999 2024-09-11T06:50:11.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.692 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:11.692 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449917+0000", 2024-09-11T06:50:11.692 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121033 2024-09-11T06:50:11.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.692 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:11.692 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.450166+0000", 2024-09-11T06:50:11.692 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00024832400000000002 2024-09-11T06:50:11.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.692 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:11.693 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.452630+0000", 2024-09-11T06:50:11.693 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002464188 2024-09-11T06:50:11.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.693 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:11.693 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453149+0000", 2024-09-11T06:50:11.693 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00051860999999999997 2024-09-11T06:50:11.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.693 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:11.693 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453234+0000", 2024-09-11T06:50:11.693 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.5865000000000005e-05 2024-09-11T06:50:11.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.694 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:11.694 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453296+0000", 2024-09-11T06:50:11.694 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1496999999999998e-05 2024-09-11T06:50:11.694 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.694 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:11.694 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.694 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:50:11.694 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:50:11.694 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 64.132540481000007, 2024-09-11T06:50:11.694 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011561900999999999, 2024-09-11T06:50:11.694 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:11.695 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:11.695 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:11.695 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:11.695 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:11.695 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:11.695 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 159 2024-09-11T06:50:11.695 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.695 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:11.695 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.695 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:11.695 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:50:11.695 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.695 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.695 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.695 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:11.695 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:50:11.696 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:11.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.696 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.696 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:11.696 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442321+0000", 2024-09-11T06:50:11.696 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8400000000000003e-07 2024-09-11T06:50:11.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.696 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.696 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:11.696 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442328+0000", 2024-09-11T06:50:11.696 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5130000000000002e-06 2024-09-11T06:50:11.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.696 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.696 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:11.697 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442341+0000", 2024-09-11T06:50:11.697 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.363e-05 2024-09-11T06:50:11.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.697 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.697 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:11.697 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442349+0000", 2024-09-11T06:50:11.697 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4760000000000001e-06 2024-09-11T06:50:11.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.697 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.697 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:11.697 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451257+0000", 2024-09-11T06:50:11.697 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089088569999999992 2024-09-11T06:50:11.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.698 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.698 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:11.698 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451389+0000", 2024-09-11T06:50:11.698 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013110899999999999 2024-09-11T06:50:11.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.698 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.698 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:11.698 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451608+0000", 2024-09-11T06:50:11.698 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000219588 2024-09-11T06:50:11.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.698 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.698 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:11.698 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453517+0000", 2024-09-11T06:50:11.698 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0019088779999999999 2024-09-11T06:50:11.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.699 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:11.699 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453727+0000", 2024-09-11T06:50:11.699 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020955100000000001 2024-09-11T06:50:11.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.699 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:11.699 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453815+0000", 2024-09-11T06:50:11.699 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8858999999999997e-05 2024-09-11T06:50:11.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:11.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:11.699 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:11.699 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453882+0000", 2024-09-11T06:50:11.699 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.6556000000000004e-05 2024-09-11T06:50:11.699 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.700 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:11.700 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.700 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:11.700 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:11.700 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:11.701 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:50:11.702 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:11.700+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:11.706 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:50:11.706 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:50:11.722 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-09-11T06:50:11.039568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1771161,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":167256,"kb_used_data":5060,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566063784,"statfs":{"total":579820584960,"available":579649314816,"internally_reserved":0,"allocated":5181440,"data_stored":2994956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9565,"internal_metadata":165927587},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.013858"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T06:50:08.932993+0000","last_change":"2024-09-11T06:49:13.393788+0000","last_active":"2024-09-11T06:50:08.932993+0000","last_peered":"2024-09-11T06:50:08.932993+0000","last_clean":"2024-09-11T06:50:08.932993+0000","last_became_active":"2024-09-11T06:49:11.433437+0000","last_became_peered":"2024-09-11T06:49:11.433437+0000","last_unstale":"2024-09-11T06:50:08.932993+0000","last_undegraded":"2024-09-11T06:50:08.932993+0000","last_fullsized":"2024-09-11T06:50:08.932993+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:20:04.339208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001403348,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":28,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T06:50:08.934792+0000","last_change":"2024-09-11T06:49:55.948281+0000","last_active":"2024-09-11T06:50:08.934792+0000","last_peered":"2024-09-11T06:50:08.934792+0000","last_clean":"2024-09-11T06:50:08.934792+0000","last_became_active":"2024-09-11T06:49:55.948006+0000","last_became_peered":"2024-09-11T06:49:55.948006+0000","last_unstale":"2024-09-11T06:50:08.934792+0000","last_undegraded":"2024-09-11T06:50:08.934792+0000","last_fullsized":"2024-09-11T06:50:08.934792+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:10:41.665226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":95,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T06:50:08.932956+0000","last_change":"2024-09-11T06:50:06.933758+0000","last_active":"2024-09-11T06:50:08.932956+0000","last_peered":"2024-09-11T06:50:08.932956+0000","last_clean":"2024-09-11T06:50:08.932956+0000","last_became_active":"2024-09-11T06:50:06.932797+0000","last_became_peered":"2024-09-11T06:50:06.932797+0000","last_unstale":"2024-09-11T06:50:08.932956+0000","last_undegraded":"2024-09-11T06:50:08.932956+0000","last_fullsized":"2024-09-11T06:50:08.932956+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:22:15.086364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095172900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T06:50:08.934667+0000","last_change":"2024-09-11T06:49:56.122331+0000","last_active":"2024-09-11T06:50:08.934667+0000","last_peered":"2024-09-11T06:50:08.934667+0000","last_clean":"2024-09-11T06:50:08.934667+0000","last_became_active":"2024-09-11T06:49:56.121998+0000","last_became_peered":"2024-09-11T06:49:56.121998+0000","last_unstale":"2024-09-11T06:50:08.934667+0000","last_undegraded":"2024-09-11T06:50:08.934667+0000","last_fullsized":"2024-09-11T06:50:08.934667+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:24.336670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":112,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T06:50:08.938334+0000","last_change":"2024-09-11T06:49:55.961845+0000","last_active":"2024-09-11T06:50:08.938334+0000","last_peered":"2024-09-11T06:50:08.938334+0000","last_clean":"2024-09-11T06:50:08.938334+0000","last_became_active":"2024-09-11T06:49:55.955837+0000","last_became_peered":"2024-09-11T06:49:55.955837+0000","last_unstale":"2024-09-11T06:50:08.938334+0000","last_undegraded":"2024-09-11T06:50:08.938334+0000","last_fullsized":"2024-09-11T06:50:08.938334+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:05:32.670233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001910244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":95,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T06:50:08.931800+0000","last_change":"2024-09-11T06:49:55.948809+0000","last_active":"2024-09-11T06:50:08.931800+0000","last_peered":"2024-09-11T06:50:08.931800+0000","last_clean":"2024-09-11T06:50:08.931800+0000","last_became_active":"2024-09-11T06:49:55.948499+0000","last_became_peered":"2024-09-11T06:49:55.948499+0000","last_unstale":"2024-09-11T06:50:08.931800+0000","last_undegraded":"2024-09-11T06:50:08.931800+0000","last_fullsized":"2024-09-11T06:50:08.931800+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:43:31.563009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010163780000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":100,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T06:50:08.932912+0000","last_change":"2024-09-11T06:49:55.951569+0000","last_active":"2024-09-11T06:50:08.932912+0000","last_peered":"2024-09-11T06:50:08.932912+0000","last_clean":"2024-09-11T06:50:08.932912+0000","last_became_active":"2024-09-11T06:49:55.950989+0000","last_became_peered":"2024-09-11T06:49:55.950989+0000","last_unstale":"2024-09-11T06:50:08.932912+0000","last_undegraded":"2024-09-11T06:50:08.932912+0000","last_fullsized":"2024-09-11T06:50:08.932912+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:07.743579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077615599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T06:50:08.938334+0000","last_change":"2024-09-11T06:49:13.251115+0000","last_active":"2024-09-11T06:50:08.938334+0000","last_peered":"2024-09-11T06:50:08.938334+0000","last_clean":"2024-09-11T06:50:08.938334+0000","last_became_active":"2024-09-11T06:49:11.235231+0000","last_became_peered":"2024-09-11T06:49:11.235231+0000","last_unstale":"2024-09-11T06:50:08.938334+0000","last_undegraded":"2024-09-11T06:50:08.938334+0000","last_fullsized":"2024-09-11T06:50:08.938334+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:01:21.028466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011212469999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":269,"reported_epoch":50,"state":"peering","last_fresh":"2024-09-11T06:50:07.946328+0000","last_change":"2024-09-11T06:50:08.933378+0000","last_active":"2024-09-11T06:50:08.933378+0000","last_peered":"2024-09-11T06:50:07.946328+0000","last_clean":"2024-09-11T06:50:05.011113+0000","last_became_active":"2024-09-11T06:50:07.937998+0000","last_became_peered":"2024-09-11T06:50:07.937998+0000","last_unstale":"2024-09-11T06:50:07.946328+0000","last_undegraded":"2024-09-11T06:50:07.946328+0000","last_fullsized":"2024-09-11T06:50:07.931924+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_clean_scrub_stamp":"2024-09-11T06:49:05.088839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640271,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":569344,"data_stored":203988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640272,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":565248,"data_stored":203969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640272,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28160,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343680,"statfs":{"total":96636764160,"available":96607928320,"internally_reserved":0,"allocated":1159168,"data_stored":794337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640273,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":565248,"data_stored":203969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640273,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28164,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343676,"statfs":{"total":96636764160,"available":96607924224,"internally_reserved":0,"allocated":1163264,"data_stored":794356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640274,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28168,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343672,"statfs":{"total":96636764160,"available":96607920128,"internally_reserved":0,"allocated":1159168,"data_stored":794337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:50:11.723 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T06:50:11.809 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:11.808+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:11.828 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:11.828 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:11.829 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:50:11.829 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:11.843 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:50:11.906 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:11.904+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:11.962 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:11.962 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:11.962 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:50:11.962 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:50:11.962 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:11.976 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:50:12.007 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:12.004+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:12.107 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:12.104+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:12.121 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:50:12.121 INFO:teuthology.orchestra.run.smithi071.stdout:{"fsid":"bd5dcb33-e9f7-4a5f-a26c-a451452e8aa2","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub,nodeep-scrub flag(s) set","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":79,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":6,"num_up_osds":6,"osd_up_since":1726037344,"num_in_osds":6,"osd_in_since":1726037394,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":8},{"state_name":"peering","count":1}],"num_pgs":9,"num_pools":2,"num_objects":4,"data_bytes":590387,"bytes_used":171270144,"bytes_avail":579649314816,"bytes_total":579820584960,"inactive_pgs_ratio":0.1111111119389534,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":144004,"recovering_keys_per_sec":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T06:48:52:191189+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-11T06:50:05.025257+0000","services":{}},"progress_events":{}} 2024-09-11T06:50:12.137 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:12.137 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:12.137 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:12.137 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [] 2024-09-11T06:50:12.137 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:12.140 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-11T06:50:12.140 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:50:12.151 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:50:12.208 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:12.208+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:12.260 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:12.260 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:12.261 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:50:12.261 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:12.273 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:50:12.309 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:12.308+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:12.411 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:12.408+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:12.434 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:12.435 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:12.435 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:50:12.435 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:50:12.435 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:12.449 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:50:12.474 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:50:12.474 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:50:12.495 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-09-11T06:50:11.039568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1771161,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":167256,"kb_used_data":5060,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566063784,"statfs":{"total":579820584960,"available":579649314816,"internally_reserved":0,"allocated":5181440,"data_stored":2994956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9565,"internal_metadata":165927587},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.013858"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T06:50:08.932993+0000","last_change":"2024-09-11T06:49:13.393788+0000","last_active":"2024-09-11T06:50:08.932993+0000","last_peered":"2024-09-11T06:50:08.932993+0000","last_clean":"2024-09-11T06:50:08.932993+0000","last_became_active":"2024-09-11T06:49:11.433437+0000","last_became_peered":"2024-09-11T06:49:11.433437+0000","last_unstale":"2024-09-11T06:50:08.932993+0000","last_undegraded":"2024-09-11T06:50:08.932993+0000","last_fullsized":"2024-09-11T06:50:08.932993+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:20:04.339208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001403348,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":28,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T06:50:08.934792+0000","last_change":"2024-09-11T06:49:55.948281+0000","last_active":"2024-09-11T06:50:08.934792+0000","last_peered":"2024-09-11T06:50:08.934792+0000","last_clean":"2024-09-11T06:50:08.934792+0000","last_became_active":"2024-09-11T06:49:55.948006+0000","last_became_peered":"2024-09-11T06:49:55.948006+0000","last_unstale":"2024-09-11T06:50:08.934792+0000","last_undegraded":"2024-09-11T06:50:08.934792+0000","last_fullsized":"2024-09-11T06:50:08.934792+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:10:41.665226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":95,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T06:50:08.932956+0000","last_change":"2024-09-11T06:50:06.933758+0000","last_active":"2024-09-11T06:50:08.932956+0000","last_peered":"2024-09-11T06:50:08.932956+0000","last_clean":"2024-09-11T06:50:08.932956+0000","last_became_active":"2024-09-11T06:50:06.932797+0000","last_became_peered":"2024-09-11T06:50:06.932797+0000","last_unstale":"2024-09-11T06:50:08.932956+0000","last_undegraded":"2024-09-11T06:50:08.932956+0000","last_fullsized":"2024-09-11T06:50:08.932956+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:22:15.086364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095172900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T06:50:08.934667+0000","last_change":"2024-09-11T06:49:56.122331+0000","last_active":"2024-09-11T06:50:08.934667+0000","last_peered":"2024-09-11T06:50:08.934667+0000","last_clean":"2024-09-11T06:50:08.934667+0000","last_became_active":"2024-09-11T06:49:56.121998+0000","last_became_peered":"2024-09-11T06:49:56.121998+0000","last_unstale":"2024-09-11T06:50:08.934667+0000","last_undegraded":"2024-09-11T06:50:08.934667+0000","last_fullsized":"2024-09-11T06:50:08.934667+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:24.336670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":112,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T06:50:08.938334+0000","last_change":"2024-09-11T06:49:55.961845+0000","last_active":"2024-09-11T06:50:08.938334+0000","last_peered":"2024-09-11T06:50:08.938334+0000","last_clean":"2024-09-11T06:50:08.938334+0000","last_became_active":"2024-09-11T06:49:55.955837+0000","last_became_peered":"2024-09-11T06:49:55.955837+0000","last_unstale":"2024-09-11T06:50:08.938334+0000","last_undegraded":"2024-09-11T06:50:08.938334+0000","last_fullsized":"2024-09-11T06:50:08.938334+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:05:32.670233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001910244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":95,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T06:50:08.931800+0000","last_change":"2024-09-11T06:49:55.948809+0000","last_active":"2024-09-11T06:50:08.931800+0000","last_peered":"2024-09-11T06:50:08.931800+0000","last_clean":"2024-09-11T06:50:08.931800+0000","last_became_active":"2024-09-11T06:49:55.948499+0000","last_became_peered":"2024-09-11T06:49:55.948499+0000","last_unstale":"2024-09-11T06:50:08.931800+0000","last_undegraded":"2024-09-11T06:50:08.931800+0000","last_fullsized":"2024-09-11T06:50:08.931800+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:43:31.563009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010163780000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":100,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T06:50:08.932912+0000","last_change":"2024-09-11T06:49:55.951569+0000","last_active":"2024-09-11T06:50:08.932912+0000","last_peered":"2024-09-11T06:50:08.932912+0000","last_clean":"2024-09-11T06:50:08.932912+0000","last_became_active":"2024-09-11T06:49:55.950989+0000","last_became_peered":"2024-09-11T06:49:55.950989+0000","last_unstale":"2024-09-11T06:50:08.932912+0000","last_undegraded":"2024-09-11T06:50:08.932912+0000","last_fullsized":"2024-09-11T06:50:08.932912+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:07.743579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077615599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T06:50:08.938334+0000","last_change":"2024-09-11T06:49:13.251115+0000","last_active":"2024-09-11T06:50:08.938334+0000","last_peered":"2024-09-11T06:50:08.938334+0000","last_clean":"2024-09-11T06:50:08.938334+0000","last_became_active":"2024-09-11T06:49:11.235231+0000","last_became_peered":"2024-09-11T06:49:11.235231+0000","last_unstale":"2024-09-11T06:50:08.938334+0000","last_undegraded":"2024-09-11T06:50:08.938334+0000","last_fullsized":"2024-09-11T06:50:08.938334+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:01:21.028466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011212469999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":269,"reported_epoch":50,"state":"peering","last_fresh":"2024-09-11T06:50:07.946328+0000","last_change":"2024-09-11T06:50:08.933378+0000","last_active":"2024-09-11T06:50:08.933378+0000","last_peered":"2024-09-11T06:50:07.946328+0000","last_clean":"2024-09-11T06:50:05.011113+0000","last_became_active":"2024-09-11T06:50:07.937998+0000","last_became_peered":"2024-09-11T06:50:07.937998+0000","last_unstale":"2024-09-11T06:50:07.946328+0000","last_undegraded":"2024-09-11T06:50:07.946328+0000","last_fullsized":"2024-09-11T06:50:07.931924+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_clean_scrub_stamp":"2024-09-11T06:49:05.088839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640271,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":569344,"data_stored":203988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640272,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":565248,"data_stored":203969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640272,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28160,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343680,"statfs":{"total":96636764160,"available":96607928320,"internally_reserved":0,"allocated":1159168,"data_stored":794337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640273,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":565248,"data_stored":203969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640273,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28164,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343676,"statfs":{"total":96636764160,"available":96607924224,"internally_reserved":0,"allocated":1163264,"data_stored":794356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640274,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28168,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343672,"statfs":{"total":96636764160,"available":96607920128,"internally_reserved":0,"allocated":1159168,"data_stored":794337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:50:12.512 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:12.512+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:12.569 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:12.569 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:12.569 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:12.569 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:50:12.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.569 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:50:12.570 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:50:12.570 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 61.338862552999998, 2024-09-11T06:50:12.570 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010553425999999999, 2024-09-11T06:50:12.570 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:12.570 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:12.570 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:12.570 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:12.570 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:12.570 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:12.570 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:50:12.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.570 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:12.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.571 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:12.571 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:50:12.571 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:12.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.571 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:12.571 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:50:12.571 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:12.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.571 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:12.571 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230274+0000", 2024-09-11T06:50:12.571 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.533e-06 2024-09-11T06:50:12.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.572 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:12.572 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230285+0000", 2024-09-11T06:50:12.572 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0540000000000001e-05 2024-09-11T06:50:12.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.572 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:12.572 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230309+0000", 2024-09-11T06:50:12.572 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4223000000000002e-05 2024-09-11T06:50:12.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.572 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:12.572 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230327+0000", 2024-09-11T06:50:12.572 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:50:12.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.573 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:12.573 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230478+0000", 2024-09-11T06:50:12.573 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000151459 2024-09-11T06:50:12.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.573 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:12.573 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230490+0000", 2024-09-11T06:50:12.573 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.147e-05 2024-09-11T06:50:12.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.573 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:12.573 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231803+0000", 2024-09-11T06:50:12.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001313158 2024-09-11T06:50:12.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.574 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:50:12.574 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231817+0000", 2024-09-11T06:50:12.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4197e-05 2024-09-11T06:50:12.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.574 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:12.574 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.235945+0000", 2024-09-11T06:50:12.575 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0041282480000000002 2024-09-11T06:50:12.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.575 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:12.575 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236312+0000", 2024-09-11T06:50:12.575 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036652700000000001 2024-09-11T06:50:12.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.575 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:50:12.575 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236689+0000", 2024-09-11T06:50:12.575 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000377117 2024-09-11T06:50:12.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.575 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:12.576 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238460+0000", 2024-09-11T06:50:12.576 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001770918 2024-09-11T06:50:12.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.576 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:12.576 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240627+0000", 2024-09-11T06:50:12.576 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021670779999999998 2024-09-11T06:50:12.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.576 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:12.576 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240751+0000", 2024-09-11T06:50:12.576 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012384900000000001 2024-09-11T06:50:12.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:12.577 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240821+0000", 2024-09-11T06:50:12.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0516e-05 2024-09-11T06:50:12.577 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:12.577 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:12.577 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:12.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.577 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop_reply(client.4230.0:1 2.3 e16/15 ondisk, result = 0)", 2024-09-11T06:50:12.577 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.238282+0000", 2024-09-11T06:50:12.577 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 61.330848228000001, 2024-09-11T06:50:12.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018970799999999999, 2024-09-11T06:50:12.577 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:12.578 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:12.578 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:12.578 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:12.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.578 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:12.578 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238282+0000", 2024-09-11T06:50:12.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:12.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.578 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:12.578 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238282+0000", 2024-09-11T06:50:12.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:12.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.579 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:12.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238284+0000", 2024-09-11T06:50:12.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:50:12.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.579 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:12.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238292+0000", 2024-09-11T06:50:12.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2390000000000002e-06 2024-09-11T06:50:12.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.579 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:12.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238314+0000", 2024-09-11T06:50:12.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1776999999999998e-05 2024-09-11T06:50:12.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.580 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:12.580 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238339+0000", 2024-09-11T06:50:12.580 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5119000000000001e-05 2024-09-11T06:50:12.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.580 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:12.580 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238438+0000", 2024-09-11T06:50:12.580 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9553e-05 2024-09-11T06:50:12.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.580 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:12.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238451+0000", 2024-09-11T06:50:12.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2286000000000001e-05 2024-09-11T06:50:12.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.581 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:12.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238472+0000", 2024-09-11T06:50:12.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1166999999999999e-05 2024-09-11T06:50:12.582 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:12.582 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:12.582 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:12.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.582 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:50:12.582 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.244627+0000", 2024-09-11T06:50:12.582 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 61.324503530000001, 2024-09-11T06:50:12.582 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00046141999999999998, 2024-09-11T06:50:12.582 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:12.582 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:12.582 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:12.582 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:12.583 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:12.583 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:12.583 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 2 2024-09-11T06:50:12.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.583 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:12.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:12.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244627+0000", 2024-09-11T06:50:12.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:12.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:12.584 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244627+0000", 2024-09-11T06:50:12.584 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:12.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.584 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:12.584 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244629+0000", 2024-09-11T06:50:12.584 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.95e-06 2024-09-11T06:50:12.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.584 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:12.584 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244640+0000", 2024-09-11T06:50:12.584 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1494e-05 2024-09-11T06:50:12.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.585 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:12.585 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244664+0000", 2024-09-11T06:50:12.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4006e-05 2024-09-11T06:50:12.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.585 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:12.585 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244684+0000", 2024-09-11T06:50:12.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.961e-05 2024-09-11T06:50:12.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:12.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.244862+0000", 2024-09-11T06:50:12.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017863500000000001 2024-09-11T06:50:12.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:12.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245088+0000", 2024-09-11T06:50:12.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000225725 2024-09-11T06:50:12.586 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:12.586 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:12.587 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:12.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.587 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:50:12.587 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.245492+0000", 2024-09-11T06:50:12.587 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 61.323638586999998, 2024-09-11T06:50:12.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00042092600000000001, 2024-09-11T06:50:12.587 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:12.587 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:12.587 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:12.587 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:12.588 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:12.588 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:12.588 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 3 2024-09-11T06:50:12.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.588 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:12.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.588 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:12.588 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245492+0000", 2024-09-11T06:50:12.588 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:12.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.589 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:12.589 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245492+0000", 2024-09-11T06:50:12.589 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:12.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.589 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:12.589 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245493+0000", 2024-09-11T06:50:12.589 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.657e-06 2024-09-11T06:50:12.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.589 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:12.589 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245501+0000", 2024-09-11T06:50:12.589 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2490000000000004e-06 2024-09-11T06:50:12.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:12.590 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245523+0000", 2024-09-11T06:50:12.590 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2016999999999999e-05 2024-09-11T06:50:12.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:12.590 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245548+0000", 2024-09-11T06:50:12.590 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5060000000000001e-05 2024-09-11T06:50:12.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:12.590 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245720+0000", 2024-09-11T06:50:12.591 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017190200000000001 2024-09-11T06:50:12.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.591 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:12.591 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.245913+0000", 2024-09-11T06:50:12.591 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193041 2024-09-11T06:50:12.591 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:12.591 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:12.591 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:12.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.591 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-11T06:50:12.591 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:50:12.592 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 60.336691709999997, 2024-09-11T06:50:12.592 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069904770000000001, 2024-09-11T06:50:12.592 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:12.592 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:12.592 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:12.592 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:12.592 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:12.592 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:12.592 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 5 2024-09-11T06:50:12.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.592 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:12.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.592 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:12.592 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:50:12.593 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:12.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.593 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:12.593 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:50:12.593 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:12.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.593 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:12.593 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232442+0000", 2024-09-11T06:50:12.593 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.044e-06 2024-09-11T06:50:12.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.594 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:12.594 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232460+0000", 2024-09-11T06:50:12.594 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7863000000000002e-05 2024-09-11T06:50:12.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.594 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:12.594 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232492+0000", 2024-09-11T06:50:12.594 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2812999999999999e-05 2024-09-11T06:50:12.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.594 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:12.594 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232510+0000", 2024-09-11T06:50:12.595 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7552999999999999e-05 2024-09-11T06:50:12.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.595 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:12.595 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232640+0000", 2024-09-11T06:50:12.595 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013029900000000001 2024-09-11T06:50:12.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.595 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:12.595 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232650+0000", 2024-09-11T06:50:12.596 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0023999999999999e-05 2024-09-11T06:50:12.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.596 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:12.596 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235641+0000", 2024-09-11T06:50:12.596 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00299089 2024-09-11T06:50:12.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.597 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:12.597 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235980+0000", 2024-09-11T06:50:12.597 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000338907 2024-09-11T06:50:12.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.597 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:12.597 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.236351+0000", 2024-09-11T06:50:12.597 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037087699999999998 2024-09-11T06:50:12.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.597 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:12.597 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238074+0000", 2024-09-11T06:50:12.597 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001722859 2024-09-11T06:50:12.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.598 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:12.598 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239160+0000", 2024-09-11T06:50:12.598 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001086074 2024-09-11T06:50:12.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.598 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:12.598 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239299+0000", 2024-09-11T06:50:12.598 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013933500000000001 2024-09-11T06:50:12.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.598 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:12.599 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239429+0000", 2024-09-11T06:50:12.599 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129939 2024-09-11T06:50:12.599 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:12.599 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:12.599 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:12.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.599 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop_reply(client.4230.0:5 2.2 e16/15 ondisk, result = 0)", 2024-09-11T06:50:12.599 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:12.238911+0000", 2024-09-11T06:50:12.599 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 60.330219421999999, 2024-09-11T06:50:12.599 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00053624900000000004, 2024-09-11T06:50:12.599 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:12.600 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:12.600 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:12.600 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:12.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.600 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:12.600 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238911+0000", 2024-09-11T06:50:12.600 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:12.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.600 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:12.600 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238911+0000", 2024-09-11T06:50:12.600 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:12.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.601 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:12.601 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238913+0000", 2024-09-11T06:50:12.601 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2069999999999998e-06 2024-09-11T06:50:12.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.601 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:12.601 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238923+0000", 2024-09-11T06:50:12.601 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0360000000000001e-05 2024-09-11T06:50:12.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.601 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:12.601 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238953+0000", 2024-09-11T06:50:12.601 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9258999999999999e-05 2024-09-11T06:50:12.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.602 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:12.602 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238967+0000", 2024-09-11T06:50:12.602 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3797e-05 2024-09-11T06:50:12.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.602 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:12.602 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239117+0000", 2024-09-11T06:50:12.602 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000150765 2024-09-11T06:50:12.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.603 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:12.603 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239143+0000", 2024-09-11T06:50:12.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5514e-05 2024-09-11T06:50:12.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.603 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:12.603 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239447+0000", 2024-09-11T06:50:12.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00030434700000000001 2024-09-11T06:50:12.603 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:12.603 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:12.603 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:12.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.603 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-11T06:50:12.604 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:50:12.604 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 59.322603758, 2024-09-11T06:50:12.604 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069253250000000004, 2024-09-11T06:50:12.604 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:12.604 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:12.604 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:12.604 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:12.604 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:12.604 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:12.604 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 7 2024-09-11T06:50:12.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.604 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:12.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.605 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:12.605 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:50:12.605 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:12.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.605 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:12.605 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:50:12.605 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:12.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.605 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:12.605 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246530+0000", 2024-09-11T06:50:12.605 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9469999999999999e-06 2024-09-11T06:50:12.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.606 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.606 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:12.606 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246545+0000", 2024-09-11T06:50:12.606 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5973000000000001e-05 2024-09-11T06:50:12.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.606 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.606 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:12.607 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246578+0000", 2024-09-11T06:50:12.607 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2759999999999998e-05 2024-09-11T06:50:12.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.607 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:12.607 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246595+0000", 2024-09-11T06:50:12.607 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6939000000000002e-05 2024-09-11T06:50:12.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.607 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:12.607 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246770+0000", 2024-09-11T06:50:12.608 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000174723 2024-09-11T06:50:12.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.608 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:12.608 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246789+0000", 2024-09-11T06:50:12.608 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9159000000000001e-05 2024-09-11T06:50:12.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.608 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:12.608 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250337+0000", 2024-09-11T06:50:12.608 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035480759999999998 2024-09-11T06:50:12.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.609 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:12.609 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250549+0000", 2024-09-11T06:50:12.609 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021192199999999999 2024-09-11T06:50:12.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.609 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:12.609 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250750+0000", 2024-09-11T06:50:12.609 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020104199999999999 2024-09-11T06:50:12.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.609 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:12.609 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.252455+0000", 2024-09-11T06:50:12.610 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001705249 2024-09-11T06:50:12.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.610 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:12.610 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253305+0000", 2024-09-11T06:50:12.610 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00084941200000000002 2024-09-11T06:50:12.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.610 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:12.610 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253388+0000", 2024-09-11T06:50:12.610 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3642999999999999e-05 2024-09-11T06:50:12.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.610 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:12.611 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253452+0000", 2024-09-11T06:50:12.611 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3479999999999995e-05 2024-09-11T06:50:12.611 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:12.611 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:12.611 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:12.611 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.611 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.611 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop_reply(client.4230.0:7 2.2 e18/15 ondisk, result = 0)", 2024-09-11T06:50:12.611 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:13.253128+0000", 2024-09-11T06:50:12.611 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 59.316002183999998, 2024-09-11T06:50:12.611 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034023700000000002, 2024-09-11T06:50:12.611 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:12.611 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:12.611 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:12.612 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:12.612 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.612 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:12.612 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253128+0000", 2024-09-11T06:50:12.612 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:12.612 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.612 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.612 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:12.612 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253128+0000", 2024-09-11T06:50:12.612 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:12.612 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.612 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.612 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:12.612 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253135+0000", 2024-09-11T06:50:12.613 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.657e-06 2024-09-11T06:50:12.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.613 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.613 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:12.613 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253144+0000", 2024-09-11T06:50:12.613 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0869999999999996e-06 2024-09-11T06:50:12.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.613 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.613 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:12.613 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253167+0000", 2024-09-11T06:50:12.613 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3252e-05 2024-09-11T06:50:12.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.613 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.613 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:12.614 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253186+0000", 2024-09-11T06:50:12.614 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9239999999999999e-05 2024-09-11T06:50:12.614 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.614 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.614 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:12.614 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253282+0000", 2024-09-11T06:50:12.614 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.6093000000000003e-05 2024-09-11T06:50:12.614 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.614 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.614 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:12.614 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253294+0000", 2024-09-11T06:50:12.614 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1986e-05 2024-09-11T06:50:12.614 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.615 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.615 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:12.615 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253468+0000", 2024-09-11T06:50:12.615 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017392199999999999 2024-09-11T06:50:12.615 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:12.615 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:12.615 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:12.615 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.615 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.615 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPushReply(2.2 37/36 [PushReplyOp(2:5cea7035:::rbd_info:head)])", 2024-09-11T06:50:12.615 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:50.557624+0000", 2024-09-11T06:50:12.615 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 22.011506119, 2024-09-11T06:50:12.615 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00069280800000000005, 2024-09-11T06:50:12.615 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:12.615 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:12.616 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:12.616 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:12.616 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.616 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:12.616 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:50.557624+0000", 2024-09-11T06:50:12.616 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:12.616 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.616 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.616 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:12.616 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:50.557624+0000", 2024-09-11T06:50:12.616 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:12.616 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.616 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.617 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:12.617 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:50.557625+0000", 2024-09-11T06:50:12.617 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.37e-07 2024-09-11T06:50:12.617 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.617 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.617 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:12.617 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:50.557633+0000", 2024-09-11T06:50:12.617 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6270000000000006e-06 2024-09-11T06:50:12.618 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.620 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.620 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:12.620 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:50.557650+0000", 2024-09-11T06:50:12.620 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6350000000000001e-05 2024-09-11T06:50:12.620 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.620 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.621 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:12.621 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:50.557667+0000", 2024-09-11T06:50:12.621 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7668000000000001e-05 2024-09-11T06:50:12.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.621 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.621 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:12.621 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:50.557832+0000", 2024-09-11T06:50:12.621 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016408699999999999 2024-09-11T06:50:12.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:12.621 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:12.621 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:12.621 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:50.558317+0000", 2024-09-11T06:50:12.621 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000485439 2024-09-11T06:50:12.622 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:12.622 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:12.622 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:12.622 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:12.622 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:12.622 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:12.623 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:50:12.624 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:12.620+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:12.725 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:12.724+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:12.753 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:12.754 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:12.754 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:50:12.754 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:12.767 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:50:12.825 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:12.824+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:12.890 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:12.890 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:12.891 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:50:12.891 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:50:12.891 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:12.897 INFO:tasks.workunit.client.0.smithi176.stderr:Note: switching to 'daaf78de334647fc0c800f4ac495cc9475a26bae'. 2024-09-11T06:50:12.897 INFO:tasks.workunit.client.0.smithi176.stderr: 2024-09-11T06:50:12.897 INFO:tasks.workunit.client.0.smithi176.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-11T06:50:12.897 INFO:tasks.workunit.client.0.smithi176.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-11T06:50:12.897 INFO:tasks.workunit.client.0.smithi176.stderr:state without impacting any branches by switching back to a branch. 2024-09-11T06:50:12.897 INFO:tasks.workunit.client.0.smithi176.stderr: 2024-09-11T06:50:12.897 INFO:tasks.workunit.client.0.smithi176.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-11T06:50:12.898 INFO:tasks.workunit.client.0.smithi176.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-11T06:50:12.898 INFO:tasks.workunit.client.0.smithi176.stderr: 2024-09-11T06:50:12.898 INFO:tasks.workunit.client.0.smithi176.stderr: git switch -c 2024-09-11T06:50:12.898 INFO:tasks.workunit.client.0.smithi176.stderr: 2024-09-11T06:50:12.898 INFO:tasks.workunit.client.0.smithi176.stderr:Or undo this operation with: 2024-09-11T06:50:12.898 INFO:tasks.workunit.client.0.smithi176.stderr: 2024-09-11T06:50:12.898 INFO:tasks.workunit.client.0.smithi176.stderr: git switch - 2024-09-11T06:50:12.898 INFO:tasks.workunit.client.0.smithi176.stderr: 2024-09-11T06:50:12.898 INFO:tasks.workunit.client.0.smithi176.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-11T06:50:12.898 INFO:tasks.workunit.client.0.smithi176.stderr: 2024-09-11T06:50:12.898 INFO:tasks.workunit.client.0.smithi176.stderr:HEAD is now at daaf78de334 Merge pull request #59682 from zdover23/wip-doc-2024-09-10-backport-59677-to-squid 2024-09-11T06:50:12.904 DEBUG:teuthology.orchestra.run.smithi176:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-11T06:50:12.906 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:50:12.926 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:12.924+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:12.951 INFO:tasks.workunit.client.0.smithi176.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-11T06:50:12.954 INFO:tasks.workunit.client.0.smithi176.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T06:50:12.954 INFO:tasks.workunit.client.0.smithi176.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-11T06:50:13.027 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:13.024+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:13.067 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T06:50:13.073 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:13.073 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:50:13.073 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:50:13.073 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:50:13.073 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.073 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:54 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'32, mlcod=13'29)", 2024-09-11T06:50:13.073 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:50:13.074 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 66.859748785999997, 2024-09-11T06:50:13.074 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034271670000000001, 2024-09-11T06:50:13.074 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:13.074 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:13.074 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:13.074 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:13.074 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.074 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:13.074 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:50:13.074 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.074 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.074 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.074 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:13.074 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:50:13.074 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210361+0000", 2024-09-11T06:50:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1260000000000001e-06 2024-09-11T06:50:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210370+0000", 2024-09-11T06:50:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8200000000000003e-06 2024-09-11T06:50:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210390+0000", 2024-09-11T06:50:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0126e-05 2024-09-11T06:50:13.076 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.076 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.076 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:13.076 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210404+0000", 2024-09-11T06:50:13.076 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4589999999999999e-05 2024-09-11T06:50:13.076 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.076 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.076 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:13.076 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210538+0000", 2024-09-11T06:50:13.076 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000133719 2024-09-11T06:50:13.076 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.076 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.076 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:13.077 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210593+0000", 2024-09-11T06:50:13.077 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5016999999999999e-05 2024-09-11T06:50:13.077 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.077 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.077 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:13.077 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213695+0000", 2024-09-11T06:50:13.077 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003101866 2024-09-11T06:50:13.077 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.077 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.077 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:13.077 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213787+0000", 2024-09-11T06:50:13.077 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1903000000000004e-05 2024-09-11T06:50:13.077 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.078 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:13.078 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.078 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.078 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.078 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:55 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'33, mlcod=13'29)", 2024-09-11T06:50:13.078 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:50:13.078 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 66.858729547999999, 2024-09-11T06:50:13.078 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.005086091, 2024-09-11T06:50:13.078 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:13.078 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:13.078 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:13.078 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:13.078 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.078 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:13.079 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:50:13.079 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.079 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.079 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.079 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:13.079 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:50:13.079 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.079 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.079 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.079 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:13.079 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211380+0000", 2024-09-11T06:50:13.079 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.097e-06 2024-09-11T06:50:13.079 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.079 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.079 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:13.080 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211537+0000", 2024-09-11T06:50:13.080 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000157369 2024-09-11T06:50:13.080 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.080 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.080 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:13.080 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211559+0000", 2024-09-11T06:50:13.080 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000001e-05 2024-09-11T06:50:13.080 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.080 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.080 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:13.080 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211573+0000", 2024-09-11T06:50:13.080 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4946000000000001e-05 2024-09-11T06:50:13.080 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.081 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.081 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:13.081 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211676+0000", 2024-09-11T06:50:13.081 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0001026 2024-09-11T06:50:13.081 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.081 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.081 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:13.081 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211709+0000", 2024-09-11T06:50:13.081 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3071999999999998e-05 2024-09-11T06:50:13.081 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.081 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.081 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:13.081 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216379+0000", 2024-09-11T06:50:13.082 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0046693780000000001 2024-09-11T06:50:13.082 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.082 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.082 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:13.082 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216465+0000", 2024-09-11T06:50:13.082 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6469000000000002e-05 2024-09-11T06:50:13.082 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.082 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:13.082 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.082 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.082 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.082 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:56 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'34, mlcod=13'29)", 2024-09-11T06:50:13.082 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:50:13.082 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 66.857295688999997, 2024-09-11T06:50:13.082 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0060310629999999997, 2024-09-11T06:50:13.083 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:13.083 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:13.083 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:13.083 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:13.083 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.083 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:13.083 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:50:13.083 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.083 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.083 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.083 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:13.083 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:50:13.083 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.083 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.084 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.084 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:13.084 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212814+0000", 2024-09-11T06:50:13.084 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0869999999999999e-06 2024-09-11T06:50:13.084 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.084 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.084 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:13.084 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212822+0000", 2024-09-11T06:50:13.084 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0870000000000007e-06 2024-09-11T06:50:13.084 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.084 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.084 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:13.084 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212840+0000", 2024-09-11T06:50:13.085 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:50:13.085 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.085 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.085 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:13.085 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212854+0000", 2024-09-11T06:50:13.085 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4003e-05 2024-09-11T06:50:13.085 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.085 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.085 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:13.085 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213026+0000", 2024-09-11T06:50:13.085 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017207499999999999 2024-09-11T06:50:13.085 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.085 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.085 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:13.086 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213063+0000", 2024-09-11T06:50:13.086 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7187000000000002e-05 2024-09-11T06:50:13.086 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.086 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.086 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:13.086 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218762+0000", 2024-09-11T06:50:13.086 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0056993290000000004 2024-09-11T06:50:13.086 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.086 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.086 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:13.086 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218844+0000", 2024-09-11T06:50:13.086 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1701999999999998e-05 2024-09-11T06:50:13.086 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.086 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:13.087 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.087 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.087 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.087 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:60 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'38, mlcod=13'36)", 2024-09-11T06:50:13.087 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:50:13.087 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 66.843354340000005, 2024-09-11T06:50:13.087 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039254270000000004, 2024-09-11T06:50:13.087 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:13.087 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:13.087 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:13.087 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:13.087 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.087 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:13.087 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:50:13.088 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.088 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.088 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.088 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:13.088 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:50:13.088 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.088 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.088 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.088 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:13.088 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226756+0000", 2024-09-11T06:50:13.088 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-11T06:50:13.088 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.088 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.089 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:13.089 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226768+0000", 2024-09-11T06:50:13.089 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1919999999999999e-05 2024-09-11T06:50:13.089 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.089 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.089 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:13.089 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226797+0000", 2024-09-11T06:50:13.089 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9573e-05 2024-09-11T06:50:13.089 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.089 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.089 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:13.089 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226818+0000", 2024-09-11T06:50:13.089 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0432999999999999e-05 2024-09-11T06:50:13.090 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.090 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.090 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:13.090 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226909+0000", 2024-09-11T06:50:13.090 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0920000000000004e-05 2024-09-11T06:50:13.090 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.090 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.090 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:13.090 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226936+0000", 2024-09-11T06:50:13.090 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7472999999999999e-05 2024-09-11T06:50:13.090 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.090 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.090 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:13.090 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230596+0000", 2024-09-11T06:50:13.091 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0036602520000000001 2024-09-11T06:50:13.091 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.091 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.091 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:13.091 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230680+0000", 2024-09-11T06:50:13.091 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3106e-05 2024-09-11T06:50:13.091 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.091 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:13.092 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.092 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.092 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.092 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:107 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'69, mlcod=13'64)", 2024-09-11T06:50:13.092 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:50:13.092 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 66.626906883000004, 2024-09-11T06:50:13.092 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040728830000000002, 2024-09-11T06:50:13.092 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:13.092 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:13.092 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:13.092 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:13.092 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.092 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:13.093 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:50:13.093 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.093 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.093 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.093 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:13.093 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:50:13.093 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.093 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.093 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.093 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:13.093 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443203+0000", 2024-09-11T06:50:13.093 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1480000000000001e-06 2024-09-11T06:50:13.093 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.093 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.094 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:13.094 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443222+0000", 2024-09-11T06:50:13.094 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9040000000000001e-05 2024-09-11T06:50:13.094 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.094 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.094 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:13.094 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443246+0000", 2024-09-11T06:50:13.094 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4606000000000001e-05 2024-09-11T06:50:13.094 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.094 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.094 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:13.094 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443277+0000", 2024-09-11T06:50:13.094 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0207999999999999e-05 2024-09-11T06:50:13.095 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.095 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.095 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:13.095 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443340+0000", 2024-09-11T06:50:13.095 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3602000000000005e-05 2024-09-11T06:50:13.095 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.095 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.095 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:13.095 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443363+0000", 2024-09-11T06:50:13.095 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2331e-05 2024-09-11T06:50:13.095 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.095 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.095 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:13.095 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447216+0000", 2024-09-11T06:50:13.096 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0038539690000000001 2024-09-11T06:50:13.096 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.096 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.096 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:13.096 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447274+0000", 2024-09-11T06:50:13.096 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7979000000000002e-05 2024-09-11T06:50:13.096 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.096 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:13.096 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.096 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.096 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.096 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:111 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'73, mlcod=13'67)", 2024-09-11T06:50:13.096 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:50:13.096 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 66.620395252999998, 2024-09-11T06:50:13.097 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034372360000000002, 2024-09-11T06:50:13.097 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:13.097 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:13.097 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:13.097 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:13.097 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.097 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:13.097 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:50:13.097 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.097 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.097 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.097 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:13.097 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:50:13.098 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.098 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.098 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.098 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:13.098 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449714+0000", 2024-09-11T06:50:13.098 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5000000000000001e-07 2024-09-11T06:50:13.098 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.098 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.098 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:13.098 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449721+0000", 2024-09-11T06:50:13.098 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8530000000000003e-06 2024-09-11T06:50:13.098 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.098 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.098 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:13.098 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449736+0000", 2024-09-11T06:50:13.099 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5114e-05 2024-09-11T06:50:13.099 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.099 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.099 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:13.099 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449743+0000", 2024-09-11T06:50:13.099 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0210000000000002e-06 2024-09-11T06:50:13.099 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.099 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.099 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:13.099 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449804+0000", 2024-09-11T06:50:13.099 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0479999999999997e-05 2024-09-11T06:50:13.099 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.099 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.099 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:13.100 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449826+0000", 2024-09-11T06:50:13.100 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1922000000000001e-05 2024-09-11T06:50:13.100 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.100 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.100 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:13.100 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453104+0000", 2024-09-11T06:50:13.100 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003278652 2024-09-11T06:50:13.100 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.100 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.100 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:13.100 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453150+0000", 2024-09-11T06:50:13.100 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6244e-05 2024-09-11T06:50:13.100 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.100 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:13.101 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.101 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.101 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.101 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:114 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'76, mlcod=13'71)", 2024-09-11T06:50:13.101 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:50:13.101 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 66.615717181999997, 2024-09-11T06:50:13.101 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0042576660000000002, 2024-09-11T06:50:13.101 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:13.101 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:13.101 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:13.101 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:13.101 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.101 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:13.101 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:50:13.102 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.102 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.102 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.102 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:13.102 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:50:13.102 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.102 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.102 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.102 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:13.102 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454392+0000", 2024-09-11T06:50:13.102 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5999999999999991e-07 2024-09-11T06:50:13.102 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.102 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.102 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:13.103 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454399+0000", 2024-09-11T06:50:13.103 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.878e-06 2024-09-11T06:50:13.103 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.103 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.103 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:13.103 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454415+0000", 2024-09-11T06:50:13.103 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5659999999999999e-05 2024-09-11T06:50:13.103 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.103 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.103 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:13.103 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454422+0000", 2024-09-11T06:50:13.103 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:50:13.103 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.104 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.104 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:13.104 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454474+0000", 2024-09-11T06:50:13.104 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1860999999999997e-05 2024-09-11T06:50:13.104 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.104 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.104 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:13.104 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454497+0000", 2024-09-11T06:50:13.104 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3686e-05 2024-09-11T06:50:13.104 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.104 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.104 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:13.104 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458552+0000", 2024-09-11T06:50:13.104 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040544550000000002 2024-09-11T06:50:13.105 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.105 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.105 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:13.105 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458649+0000", 2024-09-11T06:50:13.105 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7005999999999999e-05 2024-09-11T06:50:13.105 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.105 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:13.105 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.105 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.105 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.105 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:115 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'77, mlcod=13'72)", 2024-09-11T06:50:13.105 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:13.105 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 66.614376570000005, 2024-09-11T06:50:13.105 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003355584, 2024-09-11T06:50:13.106 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:13.106 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:13.106 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:13.106 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:13.106 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.106 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:13.106 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:13.106 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.106 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.106 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.106 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:13.106 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:13.106 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.106 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.107 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.107 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:13.107 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:13.107 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5999999999999999e-07 2024-09-11T06:50:13.107 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.107 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.107 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:13.107 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455803+0000", 2024-09-11T06:50:13.107 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0310999999999998e-05 2024-09-11T06:50:13.107 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.107 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.107 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:13.107 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455815+0000", 2024-09-11T06:50:13.108 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2787e-05 2024-09-11T06:50:13.108 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.108 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.108 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:13.108 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455823+0000", 2024-09-11T06:50:13.108 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5959999999999998e-06 2024-09-11T06:50:13.108 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.108 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.108 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:13.108 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455875+0000", 2024-09-11T06:50:13.108 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2054e-05 2024-09-11T06:50:13.108 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.108 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.108 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:13.109 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455896+0000", 2024-09-11T06:50:13.109 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0554999999999999e-05 2024-09-11T06:50:13.109 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.109 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.109 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:13.109 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459000+0000", 2024-09-11T06:50:13.109 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003104842 2024-09-11T06:50:13.109 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.109 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.109 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:13.109 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459087+0000", 2024-09-11T06:50:13.109 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6978999999999995e-05 2024-09-11T06:50:13.109 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.109 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:13.110 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.110 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.110 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.110 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:120 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'82, mlcod=13'78)", 2024-09-11T06:50:13.110 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:50:13.110 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 66.603699285000005, 2024-09-11T06:50:13.110 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0064612549999999999, 2024-09-11T06:50:13.110 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:13.110 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:13.110 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:13.110 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:13.110 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.110 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:13.110 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:50:13.111 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.111 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.111 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.111 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:13.111 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:50:13.111 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.111 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.111 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.111 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:13.111 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466410+0000", 2024-09-11T06:50:13.111 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0389999999999999e-06 2024-09-11T06:50:13.111 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.111 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.112 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:13.112 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466496+0000", 2024-09-11T06:50:13.112 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5519000000000006e-05 2024-09-11T06:50:13.112 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.112 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.112 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:13.112 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466510+0000", 2024-09-11T06:50:13.112 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3842999999999999e-05 2024-09-11T06:50:13.112 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.112 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.112 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:13.112 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466517+0000", 2024-09-11T06:50:13.112 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5229999999999998e-06 2024-09-11T06:50:13.112 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.112 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.113 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:13.113 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466595+0000", 2024-09-11T06:50:13.113 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8019e-05 2024-09-11T06:50:13.113 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.113 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.113 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:13.113 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466612+0000", 2024-09-11T06:50:13.113 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7013000000000001e-05 2024-09-11T06:50:13.113 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.113 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.113 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:13.113 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472787+0000", 2024-09-11T06:50:13.113 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0061750269999999996 2024-09-11T06:50:13.113 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.114 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.114 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:13.114 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472870+0000", 2024-09-11T06:50:13.114 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3271999999999998e-05 2024-09-11T06:50:13.114 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.114 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:13.114 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.114 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.114 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.114 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:121 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'83, mlcod=13'78)", 2024-09-11T06:50:13.114 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:50:13.114 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 66.602474489000002, 2024-09-11T06:50:13.114 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055613069999999997, 2024-09-11T06:50:13.114 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:13.115 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:13.115 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:13.115 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:13.115 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.115 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:13.115 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:50:13.115 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.115 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.115 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.115 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:13.115 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:50:13.115 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.115 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.115 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.116 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:13.116 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467635+0000", 2024-09-11T06:50:13.116 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6899999999999996e-07 2024-09-11T06:50:13.116 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.116 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.116 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:13.116 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467727+0000", 2024-09-11T06:50:13.116 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1800999999999997e-05 2024-09-11T06:50:13.116 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.116 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.116 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:13.116 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467740+0000", 2024-09-11T06:50:13.116 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3747999999999999e-05 2024-09-11T06:50:13.117 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.117 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.117 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:13.117 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467753+0000", 2024-09-11T06:50:13.117 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2807e-05 2024-09-11T06:50:13.117 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.117 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.117 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:13.117 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467812+0000", 2024-09-11T06:50:13.117 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8381999999999998e-05 2024-09-11T06:50:13.117 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.117 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.117 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:13.117 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467828+0000", 2024-09-11T06:50:13.118 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6728e-05 2024-09-11T06:50:13.118 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.118 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.118 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:13.118 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473112+0000", 2024-09-11T06:50:13.118 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052833619999999998 2024-09-11T06:50:13.118 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.118 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.118 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:13.118 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473195+0000", 2024-09-11T06:50:13.118 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3609999999999994e-05 2024-09-11T06:50:13.118 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.118 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:13.118 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.118 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.119 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.119 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:131 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'89, mlcod=13'85)", 2024-09-11T06:50:13.119 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:50:13.119 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 66.251068774000004, 2024-09-11T06:50:13.119 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052492260000000001, 2024-09-11T06:50:13.119 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:13.119 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:13.119 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:13.119 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:13.119 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.119 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:13.119 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:50:13.119 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.119 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.119 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.119 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:13.120 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:50:13.120 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.120 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.120 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.120 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:13.120 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819041+0000", 2024-09-11T06:50:13.120 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4360000000000001e-06 2024-09-11T06:50:13.120 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.120 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.120 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:13.120 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819100+0000", 2024-09-11T06:50:13.120 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9225999999999998e-05 2024-09-11T06:50:13.120 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.120 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.121 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:13.121 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819128+0000", 2024-09-11T06:50:13.121 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.779e-05 2024-09-11T06:50:13.121 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.121 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.121 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:13.121 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819142+0000", 2024-09-11T06:50:13.121 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3573e-05 2024-09-11T06:50:13.121 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.121 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.121 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:13.121 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819263+0000", 2024-09-11T06:50:13.121 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000121086 2024-09-11T06:50:13.122 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.122 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.122 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:13.122 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819292+0000", 2024-09-11T06:50:13.122 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9363e-05 2024-09-11T06:50:13.122 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.122 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.122 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:13.122 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824185+0000", 2024-09-11T06:50:13.122 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0048924620000000002 2024-09-11T06:50:13.122 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.122 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.122 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:13.123 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824289+0000", 2024-09-11T06:50:13.123 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010429000000000001 2024-09-11T06:50:13.123 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.123 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:13.123 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.123 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.123 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.123 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:132 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'90, mlcod=13'85)", 2024-09-11T06:50:13.123 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:50:13.123 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 66.250294084000004, 2024-09-11T06:50:13.123 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.004532342, 2024-09-11T06:50:13.123 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:13.123 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:13.123 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:13.123 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:13.124 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.124 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:13.124 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:50:13.124 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.124 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.124 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.124 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:13.124 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:50:13.124 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.124 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.124 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.124 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:13.124 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819816+0000", 2024-09-11T06:50:13.124 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.61e-06 2024-09-11T06:50:13.125 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.125 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.125 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:13.125 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819827+0000", 2024-09-11T06:50:13.125 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1e-05 2024-09-11T06:50:13.125 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.125 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.125 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:13.125 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819851+0000", 2024-09-11T06:50:13.125 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4267e-05 2024-09-11T06:50:13.125 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.125 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.126 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:13.126 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819864+0000", 2024-09-11T06:50:13.126 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2767e-05 2024-09-11T06:50:13.126 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.126 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.126 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:13.126 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820071+0000", 2024-09-11T06:50:13.126 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000207061 2024-09-11T06:50:13.126 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.126 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.126 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:13.126 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820103+0000", 2024-09-11T06:50:13.126 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2326999999999997e-05 2024-09-11T06:50:13.127 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.127 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.127 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:13.127 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824302+0000", 2024-09-11T06:50:13.127 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0041986000000000002 2024-09-11T06:50:13.127 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.127 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.127 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:13.127 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824347+0000", 2024-09-11T06:50:13.127 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4709999999999997e-05 2024-09-11T06:50:13.127 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.127 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:13.128 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.128 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.128 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.128 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:133 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'91, mlcod=13'86)", 2024-09-11T06:50:13.128 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:50:13.128 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 66.249662424999997, 2024-09-11T06:50:13.128 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039421459999999997, 2024-09-11T06:50:13.128 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:13.128 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:13.128 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:13.128 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:13.128 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.128 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:13.128 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:50:13.129 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.129 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.129 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.129 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:13.129 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:50:13.129 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.129 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.129 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.129 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:13.129 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820448+0000", 2024-09-11T06:50:13.129 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.84e-06 2024-09-11T06:50:13.129 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.129 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.130 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:13.130 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820463+0000", 2024-09-11T06:50:13.130 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5356e-05 2024-09-11T06:50:13.130 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.130 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.130 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:13.130 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820499+0000", 2024-09-11T06:50:13.130 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.625e-05 2024-09-11T06:50:13.130 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.130 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.130 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:13.130 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820546+0000", 2024-09-11T06:50:13.130 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7017000000000001e-05 2024-09-11T06:50:13.131 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.131 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.131 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:13.131 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820882+0000", 2024-09-11T06:50:13.131 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033512 2024-09-11T06:50:13.131 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.131 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.131 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:13.131 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820941+0000", 2024-09-11T06:50:13.131 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8922999999999998e-05 2024-09-11T06:50:13.131 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.131 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.131 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:13.131 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824351+0000", 2024-09-11T06:50:13.132 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034101940000000001 2024-09-11T06:50:13.132 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.132 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.132 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:13.132 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824388+0000", 2024-09-11T06:50:13.132 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7446e-05 2024-09-11T06:50:13.132 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.132 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:13.132 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.132 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.132 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.132 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:134 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'92, mlcod=13'86)", 2024-09-11T06:50:13.132 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:50:13.132 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 66.249003306999995, 2024-09-11T06:50:13.133 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033248179999999998, 2024-09-11T06:50:13.133 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:13.133 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:13.133 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:13.133 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:13.133 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.133 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:13.133 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:50:13.133 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.133 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.133 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.133 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:13.133 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:50:13.133 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.134 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.134 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.134 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:13.134 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821107+0000", 2024-09-11T06:50:13.134 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7430000000000001e-06 2024-09-11T06:50:13.134 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.134 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.134 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:13.134 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821205+0000", 2024-09-11T06:50:13.134 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7666999999999998e-05 2024-09-11T06:50:13.134 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.134 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.134 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:13.134 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821234+0000", 2024-09-11T06:50:13.135 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9756000000000001e-05 2024-09-11T06:50:13.135 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.135 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.135 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:13.135 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821254+0000", 2024-09-11T06:50:13.135 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9446e-05 2024-09-11T06:50:13.135 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.135 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.135 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:13.135 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821846+0000", 2024-09-11T06:50:13.135 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00059187599999999997 2024-09-11T06:50:13.135 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.135 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.136 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:13.136 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821894+0000", 2024-09-11T06:50:13.136 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8766e-05 2024-09-11T06:50:13.136 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.136 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.136 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:13.136 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824392+0000", 2024-09-11T06:50:13.136 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0024977509999999999 2024-09-11T06:50:13.136 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.136 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.136 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:13.136 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824430+0000", 2024-09-11T06:50:13.136 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7812999999999999e-05 2024-09-11T06:50:13.136 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.137 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:13.137 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.137 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.137 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.137 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:137 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'95, mlcod=13'86)", 2024-09-11T06:50:13.137 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:50:13.137 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 66.246541166, 2024-09-11T06:50:13.137 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003084299, 2024-09-11T06:50:13.137 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:13.137 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:13.137 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:13.137 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:13.137 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.138 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:13.138 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:50:13.138 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.138 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.138 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.138 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:13.138 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823567+0000", 2024-09-11T06:50:13.138 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.138 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.138 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.138 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:13.138 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823568+0000", 2024-09-11T06:50:13.138 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1030000000000001e-06 2024-09-11T06:50:13.139 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.139 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.139 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:13.139 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823662+0000", 2024-09-11T06:50:13.139 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3529999999999994e-05 2024-09-11T06:50:13.139 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.139 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.139 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:13.139 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823686+0000", 2024-09-11T06:50:13.139 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4358999999999998e-05 2024-09-11T06:50:13.139 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.139 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.139 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:13.139 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.823701+0000", 2024-09-11T06:50:13.140 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4847e-05 2024-09-11T06:50:13.140 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.140 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.140 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:13.140 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824518+0000", 2024-09-11T06:50:13.140 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00081716700000000005 2024-09-11T06:50:13.140 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.140 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.140 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:13.140 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824546+0000", 2024-09-11T06:50:13.140 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7860000000000001e-05 2024-09-11T06:50:13.140 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.140 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.141 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:13.141 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826558+0000", 2024-09-11T06:50:13.141 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0020120009999999998 2024-09-11T06:50:13.141 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.141 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.141 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:13.141 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826652+0000", 2024-09-11T06:50:13.141 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3431999999999996e-05 2024-09-11T06:50:13.141 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.141 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:13.141 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.141 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.141 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.142 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:139 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'97, mlcod=13'86)", 2024-09-11T06:50:13.142 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:50:13.142 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 66.244252446999994, 2024-09-11T06:50:13.142 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003732221, 2024-09-11T06:50:13.142 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:13.142 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:13.142 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:13.142 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:13.142 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.142 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:13.142 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:50:13.142 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.143 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.143 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.143 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:13.143 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:50:13.143 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.143 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.143 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.143 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:13.143 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825858+0000", 2024-09-11T06:50:13.143 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5829999999999999e-06 2024-09-11T06:50:13.143 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.144 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.144 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:13.144 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825874+0000", 2024-09-11T06:50:13.144 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5967000000000001e-05 2024-09-11T06:50:13.144 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.144 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.144 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:13.144 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825909+0000", 2024-09-11T06:50:13.144 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5692999999999999e-05 2024-09-11T06:50:13.144 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.144 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.144 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:13.144 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825923+0000", 2024-09-11T06:50:13.144 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4080000000000001e-05 2024-09-11T06:50:13.145 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.145 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.145 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:13.145 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826010+0000", 2024-09-11T06:50:13.145 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6248999999999994e-05 2024-09-11T06:50:13.145 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.145 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.145 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:13.145 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826038+0000", 2024-09-11T06:50:13.145 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8010000000000001e-05 2024-09-11T06:50:13.145 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.145 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.145 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:13.146 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829499+0000", 2024-09-11T06:50:13.146 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003461783 2024-09-11T06:50:13.146 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.146 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.146 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:13.146 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829588+0000", 2024-09-11T06:50:13.146 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8856000000000004e-05 2024-09-11T06:50:13.146 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.146 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:13.146 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.146 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.146 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.146 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:140 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'98, mlcod=13'93)", 2024-09-11T06:50:13.146 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:50:13.147 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 66.242401717999996, 2024-09-11T06:50:13.147 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0057995929999999996, 2024-09-11T06:50:13.147 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:13.147 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:13.147 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:13.147 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:13.147 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.147 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:13.147 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:50:13.147 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.147 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.147 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.147 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:13.148 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:50:13.148 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.148 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.148 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:13.148 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827708+0000", 2024-09-11T06:50:13.148 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.547e-06 2024-09-11T06:50:13.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.148 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.148 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:13.148 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827765+0000", 2024-09-11T06:50:13.148 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6266e-05 2024-09-11T06:50:13.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.149 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.149 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:13.149 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827791+0000", 2024-09-11T06:50:13.149 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6098999999999999e-05 2024-09-11T06:50:13.149 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.149 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.149 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:13.149 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827806+0000", 2024-09-11T06:50:13.149 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5537e-05 2024-09-11T06:50:13.149 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.149 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.149 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:13.149 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827928+0000", 2024-09-11T06:50:13.149 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000122012 2024-09-11T06:50:13.150 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.150 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.150 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:13.150 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827961+0000", 2024-09-11T06:50:13.150 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2663999999999998e-05 2024-09-11T06:50:13.150 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.150 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.150 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:13.150 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833409+0000", 2024-09-11T06:50:13.150 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054479350000000001 2024-09-11T06:50:13.150 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.150 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.150 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:13.151 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833506+0000", 2024-09-11T06:50:13.151 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7533000000000003e-05 2024-09-11T06:50:13.151 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.151 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:13.151 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.151 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.151 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.151 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:141 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'99, mlcod=13'93)", 2024-09-11T06:50:13.151 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:50:13.151 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 66.241508335000006, 2024-09-11T06:50:13.151 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055394709999999998, 2024-09-11T06:50:13.151 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:13.151 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:13.151 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:13.151 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:13.152 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.152 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:13.152 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:50:13.152 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.152 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.152 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:13.152 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:50:13.152 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.152 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.152 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:13.152 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828602+0000", 2024-09-11T06:50:13.153 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.46e-06 2024-09-11T06:50:13.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.153 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.153 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:13.153 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828618+0000", 2024-09-11T06:50:13.153 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6592999999999999e-05 2024-09-11T06:50:13.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.153 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.153 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:13.153 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828646+0000", 2024-09-11T06:50:13.153 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7552999999999999e-05 2024-09-11T06:50:13.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.153 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.154 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:13.154 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828663+0000", 2024-09-11T06:50:13.154 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.719e-05 2024-09-11T06:50:13.154 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.154 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.154 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:13.154 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829023+0000", 2024-09-11T06:50:13.154 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036054799999999998 2024-09-11T06:50:13.154 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.154 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.154 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:13.154 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829062+0000", 2024-09-11T06:50:13.154 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8865999999999997e-05 2024-09-11T06:50:13.154 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.154 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.155 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:13.155 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834078+0000", 2024-09-11T06:50:13.155 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050157480000000004 2024-09-11T06:50:13.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.155 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.155 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:13.155 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834140+0000", 2024-09-11T06:50:13.155 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1513000000000005e-05 2024-09-11T06:50:13.155 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.155 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:13.155 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.155 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.155 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:144 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'102, mlcod=13'98)", 2024-09-11T06:50:13.156 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:50:13.156 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 66.232347722, 2024-09-11T06:50:13.156 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003210348, 2024-09-11T06:50:13.156 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:13.156 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:13.156 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:13.156 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:13.156 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.156 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:13.156 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:50:13.156 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.156 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.156 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.156 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:13.157 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:50:13.157 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.157 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.157 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:13.157 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837768+0000", 2024-09-11T06:50:13.157 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9330000000000004e-06 2024-09-11T06:50:13.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.157 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.157 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:13.157 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837858+0000", 2024-09-11T06:50:13.157 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0185999999999993e-05 2024-09-11T06:50:13.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.157 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.158 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:13.158 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837890+0000", 2024-09-11T06:50:13.158 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2437000000000001e-05 2024-09-11T06:50:13.158 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.158 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.158 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:13.158 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837905+0000", 2024-09-11T06:50:13.158 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:50:13.158 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.158 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.158 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:13.158 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838019+0000", 2024-09-11T06:50:13.158 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000114336 2024-09-11T06:50:13.158 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.158 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.159 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:13.159 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838052+0000", 2024-09-11T06:50:13.159 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3139000000000002e-05 2024-09-11T06:50:13.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.159 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:13.159 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840914+0000", 2024-09-11T06:50:13.159 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028612780000000001 2024-09-11T06:50:13.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.159 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:13.159 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840971+0000", 2024-09-11T06:50:13.159 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7402999999999999e-05 2024-09-11T06:50:13.160 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.160 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:13.160 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.160 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.160 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:145 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'103, mlcod=13'98)", 2024-09-11T06:50:13.160 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:50:13.160 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 66.231555838000006, 2024-09-11T06:50:13.160 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054455409999999999, 2024-09-11T06:50:13.160 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:13.160 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:13.160 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:13.160 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:13.161 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.161 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:13.161 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:50:13.161 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.161 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.161 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:13.161 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:50:13.161 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.161 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.161 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:13.161 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838554+0000", 2024-09-11T06:50:13.161 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3430000000000001e-06 2024-09-11T06:50:13.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.162 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.162 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:13.162 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838640+0000", 2024-09-11T06:50:13.162 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5569999999999996e-05 2024-09-11T06:50:13.162 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.162 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.162 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:13.162 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838660+0000", 2024-09-11T06:50:13.162 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0576000000000001e-05 2024-09-11T06:50:13.162 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.162 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.162 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:13.162 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838674+0000", 2024-09-11T06:50:13.163 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3529999999999999e-05 2024-09-11T06:50:13.163 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.163 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.163 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:13.163 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838838+0000", 2024-09-11T06:50:13.163 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016468799999999999 2024-09-11T06:50:13.163 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.163 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.163 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:13.163 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838885+0000", 2024-09-11T06:50:13.163 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6884e-05 2024-09-11T06:50:13.163 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.163 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.163 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:13.164 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843901+0000", 2024-09-11T06:50:13.164 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050162310000000003 2024-09-11T06:50:13.164 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.164 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.164 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:13.164 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843998+0000", 2024-09-11T06:50:13.164 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.6718999999999993e-05 2024-09-11T06:50:13.164 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.164 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:13.164 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.164 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.164 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:13.164 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:13.166 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:50:13.166 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:13.164+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:13.267 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:13.268+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:13.321 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:13.322 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:13.322 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:50:13.322 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:13.335 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:50:13.368 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:13.368+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:13.461 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:13.461 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:13.461 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:50:13.461 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:50:13.461 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:13.469 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:13.468+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:13.475 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:50:13.557 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:50:13.570 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:13.568+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:13.637 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:13.638 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:50:13.638 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:50:13.638 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:50:13.638 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.638 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:5 2.2 e17/15 2:5cea7035:::rbd_info:head v 17'1, mlcod=0'0)", 2024-09-11T06:50:13.638 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:50:13.638 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 61.401249616999998, 2024-09-11T06:50:13.638 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0019446719999999999, 2024-09-11T06:50:13.638 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:13.638 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:13.638 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:13.639 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:13.639 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.639 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:13.639 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:50:13.639 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.639 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.639 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.639 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:13.639 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:50:13.639 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.639 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.639 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.639 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:13.639 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236085+0000", 2024-09-11T06:50:13.640 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6000000000000003e-07 2024-09-11T06:50:13.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.640 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.640 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:13.640 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236108+0000", 2024-09-11T06:50:13.640 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3113e-05 2024-09-11T06:50:13.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.640 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.640 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:13.640 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236133+0000", 2024-09-11T06:50:13.640 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5723000000000001e-05 2024-09-11T06:50:13.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.640 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.641 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:13.641 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236154+0000", 2024-09-11T06:50:13.641 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0333999999999999e-05 2024-09-11T06:50:13.641 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.641 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.641 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:13.641 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236297+0000", 2024-09-11T06:50:13.641 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014278500000000001 2024-09-11T06:50:13.641 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.641 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.641 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:13.642 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236383+0000", 2024-09-11T06:50:13.642 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5927999999999995e-05 2024-09-11T06:50:13.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.642 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.642 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:13.642 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.237950+0000", 2024-09-11T06:50:13.642 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015673690000000001 2024-09-11T06:50:13.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.643 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.643 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:13.643 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.238029+0000", 2024-09-11T06:50:13.643 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8659999999999996e-05 2024-09-11T06:50:13.643 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.643 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:13.643 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.643 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.643 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.643 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:7 2.2 e18/15 2:5cea7035:::rbd_info:head v 18'2, mlcod=0'0)", 2024-09-11T06:50:13.643 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:50:13.643 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 60.386924409999999, 2024-09-11T06:50:13.643 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001907192, 2024-09-11T06:50:13.644 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:13.644 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:13.644 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:13.644 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:13.644 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.644 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:13.644 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:50:13.644 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.644 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.644 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:13.644 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:50:13.644 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:13.645 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.645 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.645 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:13.645 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250410+0000", 2024-09-11T06:50:13.645 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7700000000000004e-07 2024-09-11T06:50:13.645 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.645 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.645 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:13.645 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250445+0000", 2024-09-11T06:50:13.645 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4749000000000002e-05 2024-09-11T06:50:13.645 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.645 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.645 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:13.645 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250478+0000", 2024-09-11T06:50:13.646 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3269999999999998e-05 2024-09-11T06:50:13.646 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.646 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.646 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:13.646 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250501+0000", 2024-09-11T06:50:13.646 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3632999999999999e-05 2024-09-11T06:50:13.646 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.646 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.646 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:13.646 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250633+0000", 2024-09-11T06:50:13.646 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013143200000000001 2024-09-11T06:50:13.646 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.646 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.646 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:13.646 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250700+0000", 2024-09-11T06:50:13.647 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7002999999999995e-05 2024-09-11T06:50:13.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.647 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.647 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:13.647 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.252172+0000", 2024-09-11T06:50:13.647 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014719550000000001 2024-09-11T06:50:13.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:13.647 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:13.647 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:13.647 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.252316+0000", 2024-09-11T06:50:13.647 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144473 2024-09-11T06:50:13.647 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.647 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:13.647 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.648 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:13.648 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:13.648 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:13.654 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:50:13.703 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:50:13.704 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:50:13.704 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:13.700+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:13.735 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:50:13.735 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:50:13.756 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:50:13.756 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:50:13.772 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:13.768+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:13.776 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:50:13.777 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:50:13.785 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:13.785 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:13.789 INFO:tasks.workunit.client.0.smithi176.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-11T06:50:13.798 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:50:13.798 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:50:13.799 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:50:13.873 INFO:tasks.workunit.client.0.smithi176.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-11T06:50:13.873 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:13.872+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:13.923 INFO:tasks.workunit.client.0.smithi176.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T06:50:13.926 INFO:tasks.workunit.client.0.smithi176.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T06:50:13.926 INFO:tasks.workunit.client.0.smithi176.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-11T06:50:13.959 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:13.959 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:13.973 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:50:13.974 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:13.972+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:13.974 INFO:tasks.workunit.client.0.smithi176.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T06:50:13.979 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-11T06:50:13.979 DEBUG:teuthology.orchestra.run.smithi176:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-11T06:50:14.021 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:50:14.021 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:50:14.030 INFO:tasks.workunit:Running workunits matching rbd/rbd-nbd.sh on client.0... 2024-09-11T06:50:14.032 INFO:tasks.workunit:Running workunit rbd/rbd-nbd.sh... 2024-09-11T06:50:14.032 DEBUG:teuthology.orchestra.run.smithi176:workunit test rbd/rbd-nbd.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=daaf78de334647fc0c800f4ac495cc9475a26bae TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/rbd-nbd.sh 2024-09-11T06:50:14.047 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is unset 2024-09-11T06:50:14.074 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:14.072+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:14.083 INFO:tasks.workunit.client.0.smithi176.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/rbd-nbd.sh 2024-09-11T06:50:14.084 INFO:tasks.workunit.client.0.smithi176.stderr:+ . /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/../../standalone/ceph-helpers.sh 2024-09-11T06:50:14.084 INFO:tasks.workunit.client.0.smithi176.stderr:++ TIMEOUT=300 2024-09-11T06:50:14.085 INFO:tasks.workunit.client.0.smithi176.stderr:++ WAIT_FOR_CLEAN_TIMEOUT=90 2024-09-11T06:50:14.085 INFO:tasks.workunit.client.0.smithi176.stderr:++ MAX_TIMEOUT=15 2024-09-11T06:50:14.085 INFO:tasks.workunit.client.0.smithi176.stderr:++ PG_NUM=4 2024-09-11T06:50:14.085 INFO:tasks.workunit.client.0.smithi176.stderr:++ TMPDIR=/tmp 2024-09-11T06:50:14.085 INFO:tasks.workunit.client.0.smithi176.stderr:++ CEPH_BUILD_VIRTUALENV=/tmp 2024-09-11T06:50:14.085 INFO:tasks.workunit.client.0.smithi176.stderr:++ TESTDIR=/home/ubuntu/cephtest 2024-09-11T06:50:14.085 INFO:tasks.workunit.client.0.smithi176.stderr:++ type xmlstarlet 2024-09-11T06:50:14.085 INFO:tasks.workunit.client.0.smithi176.stderr:++ XMLSTARLET=xmlstarlet 2024-09-11T06:50:14.085 INFO:tasks.workunit.client.0.smithi176.stderr:+++ uname 2024-09-11T06:50:14.086 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' Linux = FreeBSD ']' 2024-09-11T06:50:14.086 INFO:tasks.workunit.client.0.smithi176.stderr:++ SED=sed 2024-09-11T06:50:14.086 INFO:tasks.workunit.client.0.smithi176.stderr:++ AWK=awk 2024-09-11T06:50:14.086 INFO:tasks.workunit.client.0.smithi176.stderr:+++ stty -a 2024-09-11T06:50:14.086 INFO:tasks.workunit.client.0.smithi176.stderr:+++ sed -e 's/.*columns \([0-9]*\).*/\1/' 2024-09-11T06:50:14.086 INFO:tasks.workunit.client.0.smithi176.stderr:+++ head -1 2024-09-11T06:50:14.091 INFO:tasks.workunit.client.0.smithi176.stderr:stty: 'standard input': Inappropriate ioctl for device 2024-09-11T06:50:14.092 INFO:tasks.workunit.client.0.smithi176.stderr:++ termwidth= 2024-09-11T06:50:14.092 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' -n '' -a '' '!=' 0 ']' 2024-09-11T06:50:14.092 INFO:tasks.workunit.client.0.smithi176.stderr:++ DIFFCOLOPTS='-y ' 2024-09-11T06:50:14.092 INFO:tasks.workunit.client.0.smithi176.stderr:++ KERNCORE=kernel.core_pattern 2024-09-11T06:50:14.092 INFO:tasks.workunit.client.0.smithi176.stderr:++ EXTRA_OPTS= 2024-09-11T06:50:14.096 INFO:tasks.workunit.client.0.smithi176.stderr:++ test '' = TESTS 2024-09-11T06:50:14.097 INFO:tasks.workunit.client.0.smithi176.stderr:+ POOL=rbd 2024-09-11T06:50:14.097 INFO:tasks.workunit.client.0.smithi176.stderr:+ ANOTHER_POOL=new_default_pool21607 2024-09-11T06:50:14.097 INFO:tasks.workunit.client.0.smithi176.stderr:+ NS=ns 2024-09-11T06:50:14.097 INFO:tasks.workunit.client.0.smithi176.stderr:+ IMAGE=testrbdnbd21607 2024-09-11T06:50:14.097 INFO:tasks.workunit.client.0.smithi176.stderr:+ SIZE=64 2024-09-11T06:50:14.098 INFO:tasks.workunit.client.0.smithi176.stderr:+ DATA= 2024-09-11T06:50:14.098 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV= 2024-09-11T06:50:14.098 INFO:tasks.workunit.client.0.smithi176.stderr:+ setup 2024-09-11T06:50:14.098 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns x 2024-09-11T06:50:14.098 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' -e CMakeCache.txt ']' 2024-09-11T06:50:14.098 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo echo test sudo 2024-09-11T06:50:14.098 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:14.098 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:14.099 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:14.099 INFO:tasks.workunit.client.0.smithi176.stderr:++ which echo 2024-09-11T06:50:14.100 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/echo 2024-09-11T06:50:14.100 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:14.100 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/echo test sudo 2024-09-11T06:50:14.107 INFO:tasks.workunit.client.0.smithi176.stdout:test sudo 2024-09-11T06:50:14.108 INFO:tasks.workunit.client.0.smithi176.stderr:+ trap cleanup INT TERM EXIT 2024-09-11T06:50:14.109 INFO:tasks.workunit.client.0.smithi176.stderr:++ mktemp -d 2024-09-11T06:50:14.134 INFO:tasks.workunit.client.0.smithi176.stderr:+ TEMPDIR=/tmp/tmp.IGWYJYR7sz 2024-09-11T06:50:14.134 INFO:tasks.workunit.client.0.smithi176.stderr:+ DATA=/tmp/tmp.IGWYJYR7sz/data 2024-09-11T06:50:14.134 INFO:tasks.workunit.client.0.smithi176.stderr:+ dd if=/dev/urandom of=/tmp/tmp.IGWYJYR7sz/data bs=1M count=64 2024-09-11T06:50:14.149 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:14.149 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:14.163 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:50:14.175 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:14.172+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:14.276 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:14.276+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:14.293 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:14.294 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:14.308 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:50:14.350 INFO:tasks.workunit.client.0.smithi176.stderr:64+0 records in 2024-09-11T06:50:14.350 INFO:tasks.workunit.client.0.smithi176.stderr:64+0 records out 2024-09-11T06:50:14.350 INFO:tasks.workunit.client.0.smithi176.stderr:67108864 bytes (67 MB, 64 MiB) copied, 0.214827 s, 312 MB/s 2024-09-11T06:50:14.350 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd namespace create rbd/ns 2024-09-11T06:50:14.395 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:14.396+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:14.422 INFO:tasks.workunit.client.0.smithi176.stderr:+ for ns in '' ${NS} 2024-09-11T06:50:14.423 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd --dest-pool rbd --dest-namespace '' --no-progress import /tmp/tmp.IGWYJYR7sz/data testrbdnbd21607 2024-09-11T06:50:14.477 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:14.477 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:14.478 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:14.476+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:14.491 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:50:14.579 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:14.576+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:14.680 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:14.680+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:14.690 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:14.690 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:14.704 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:50:14.739 INFO:tasks.workunit.client.0.smithi176.stderr:+ for ns in '' ${NS} 2024-09-11T06:50:14.739 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd --dest-pool rbd --dest-namespace ns --no-progress import /tmp/tmp.IGWYJYR7sz/data testrbdnbd21607 2024-09-11T06:50:14.781 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:14.780+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:14.881 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:14.881 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:14.882 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:14.880+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:14.896 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:50:14.983 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:14.980+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:15.031 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:15.031 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:15.040 INFO:tasks.workunit.client.0.smithi176.stderr:+ ceph osd pool create new_default_pool21607 8 2024-09-11T06:50:15.045 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:50:15.087 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:15.088+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:15.187 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:15.184+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:15.209 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:15.209 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:15.223 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:50:15.311 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:15.312+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:15.389 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:15.384+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:15.390 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:15.390 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:15.404 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:50:15.490 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:15.488+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:15.496 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:50:15.572 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:15.572 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:15.586 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:50:15.631 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:15.632+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:15.691 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:15.692+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:15.707 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:15.708 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:15.723 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:50:15.792 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:15.792+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:15.799 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:50:15.799 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:50:15.818 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2024-09-11T06:50:15.027575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1771161,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":171032,"kb_used_data":4696,"kb_used_omap":9,"kb_used_meta":166198,"kb_avail":566060008,"statfs":{"total":579820584960,"available":579645448192,"internally_reserved":0,"allocated":4808704,"data_stored":2487214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9565,"internal_metadata":170187427},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001371"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T06:50:11.042298+0000","last_change":"2024-09-11T06:49:13.393788+0000","last_active":"2024-09-11T06:50:11.042298+0000","last_peered":"2024-09-11T06:50:11.042298+0000","last_clean":"2024-09-11T06:50:11.042298+0000","last_became_active":"2024-09-11T06:49:11.433437+0000","last_became_peered":"2024-09-11T06:49:11.433437+0000","last_unstale":"2024-09-11T06:50:11.042298+0000","last_undegraded":"2024-09-11T06:50:11.042298+0000","last_fullsized":"2024-09-11T06:50:11.042298+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:20:04.339208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001403348,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":34,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T06:50:14.050783+0000","last_change":"2024-09-11T06:49:55.948281+0000","last_active":"2024-09-11T06:50:14.050783+0000","last_peered":"2024-09-11T06:50:14.050783+0000","last_clean":"2024-09-11T06:50:14.050783+0000","last_became_active":"2024-09-11T06:49:55.948006+0000","last_became_peered":"2024-09-11T06:49:55.948006+0000","last_unstale":"2024-09-11T06:50:14.050783+0000","last_undegraded":"2024-09-11T06:50:14.050783+0000","last_fullsized":"2024-09-11T06:50:14.050783+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:10:41.665226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T06:50:11.042217+0000","last_change":"2024-09-11T06:50:06.933758+0000","last_active":"2024-09-11T06:50:11.042217+0000","last_peered":"2024-09-11T06:50:11.042217+0000","last_clean":"2024-09-11T06:50:11.042217+0000","last_became_active":"2024-09-11T06:50:06.932797+0000","last_became_peered":"2024-09-11T06:50:06.932797+0000","last_unstale":"2024-09-11T06:50:11.042217+0000","last_undegraded":"2024-09-11T06:50:11.042217+0000","last_fullsized":"2024-09-11T06:50:11.042217+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:22:15.086364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095172900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T06:50:14.050690+0000","last_change":"2024-09-11T06:49:56.122331+0000","last_active":"2024-09-11T06:50:14.050690+0000","last_peered":"2024-09-11T06:50:14.050690+0000","last_clean":"2024-09-11T06:50:14.050690+0000","last_became_active":"2024-09-11T06:49:56.121998+0000","last_became_peered":"2024-09-11T06:49:56.121998+0000","last_unstale":"2024-09-11T06:50:14.050690+0000","last_undegraded":"2024-09-11T06:50:14.050690+0000","last_fullsized":"2024-09-11T06:50:14.050690+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:24.336670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T06:50:11.840223+0000","last_change":"2024-09-11T06:49:55.961845+0000","last_active":"2024-09-11T06:50:11.840223+0000","last_peered":"2024-09-11T06:50:11.840223+0000","last_clean":"2024-09-11T06:50:11.840223+0000","last_became_active":"2024-09-11T06:49:55.955837+0000","last_became_peered":"2024-09-11T06:49:55.955837+0000","last_unstale":"2024-09-11T06:50:11.840223+0000","last_undegraded":"2024-09-11T06:50:11.840223+0000","last_fullsized":"2024-09-11T06:50:11.840223+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:05:32.670233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001910244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":101,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T06:50:14.049715+0000","last_change":"2024-09-11T06:49:55.948809+0000","last_active":"2024-09-11T06:50:14.049715+0000","last_peered":"2024-09-11T06:50:14.049715+0000","last_clean":"2024-09-11T06:50:14.049715+0000","last_became_active":"2024-09-11T06:49:55.948499+0000","last_became_peered":"2024-09-11T06:49:55.948499+0000","last_unstale":"2024-09-11T06:50:14.049715+0000","last_undegraded":"2024-09-11T06:50:14.049715+0000","last_fullsized":"2024-09-11T06:50:14.049715+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:43:31.563009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010163780000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T06:50:11.042104+0000","last_change":"2024-09-11T06:49:55.951569+0000","last_active":"2024-09-11T06:50:11.042104+0000","last_peered":"2024-09-11T06:50:11.042104+0000","last_clean":"2024-09-11T06:50:11.042104+0000","last_became_active":"2024-09-11T06:49:55.950989+0000","last_became_peered":"2024-09-11T06:49:55.950989+0000","last_unstale":"2024-09-11T06:50:11.042104+0000","last_undegraded":"2024-09-11T06:50:11.042104+0000","last_fullsized":"2024-09-11T06:50:11.042104+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:07.743579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077615599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T06:50:11.840248+0000","last_change":"2024-09-11T06:49:13.251115+0000","last_active":"2024-09-11T06:50:11.840248+0000","last_peered":"2024-09-11T06:50:11.840248+0000","last_clean":"2024-09-11T06:50:11.840248+0000","last_became_active":"2024-09-11T06:49:11.235231+0000","last_became_peered":"2024-09-11T06:49:11.235231+0000","last_unstale":"2024-09-11T06:50:11.840248+0000","last_undegraded":"2024-09-11T06:50:11.840248+0000","last_fullsized":"2024-09-11T06:50:11.840248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:01:21.028466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011212469999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":279,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T06:50:11.042211+0000","last_change":"2024-09-11T06:50:09.941308+0000","last_active":"2024-09-11T06:50:11.042211+0000","last_peered":"2024-09-11T06:50:11.042211+0000","last_clean":"2024-09-11T06:50:11.042211+0000","last_became_active":"2024-09-11T06:50:09.940161+0000","last_became_peered":"2024-09-11T06:50:09.940161+0000","last_unstale":"2024-09-11T06:50:11.042211+0000","last_undegraded":"2024-09-11T06:50:11.042211+0000","last_fullsized":"2024-09-11T06:50:11.042211+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_clean_scrub_stamp":"2024-09-11T06:49:05.088839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:19:19.991300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640272,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":606208,"data_stored":217759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640273,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27616,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":602112,"data_stored":217740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640273,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28196,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343644,"statfs":{"total":96636764160,"available":96607891456,"internally_reserved":0,"allocated":1196032,"data_stored":808108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640274,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":602112,"data_stored":217740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640274,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":606208,"data_stored":217759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640275,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32356,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339484,"statfs":{"total":96636764160,"available":96603631616,"internally_reserved":0,"allocated":1196032,"data_stored":808108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:50:15.819 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-11T06:50:15.819 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T06:50:15.819 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:50:15.847 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:15.847 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:15.861 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:50:15.893 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:15.892+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:15.995 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:15.992+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:16.022 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:16.022 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:16.037 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:50:16.039 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:50:16.077 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T06:50:16.125 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:16.124+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:16.127 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:50:16.127 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:50:16.149 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2024-09-11T06:50:16.076060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1771161,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":171032,"kb_used_data":4696,"kb_used_omap":9,"kb_used_meta":166198,"kb_avail":566060008,"statfs":{"total":579820584960,"available":579645448192,"internally_reserved":0,"allocated":4808704,"data_stored":2487214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9565,"internal_metadata":170187427},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.142337"},"pg_stats":[{"pgid":"1.0","version":"14'117","reported_seq":279,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T06:50:11.042211+0000","last_change":"2024-09-11T06:50:09.941308+0000","last_active":"2024-09-11T06:50:11.042211+0000","last_peered":"2024-09-11T06:50:11.042211+0000","last_clean":"2024-09-11T06:50:11.042211+0000","last_became_active":"2024-09-11T06:50:09.940161+0000","last_became_peered":"2024-09-11T06:50:09.940161+0000","last_unstale":"2024-09-11T06:50:11.042211+0000","last_undegraded":"2024-09-11T06:50:11.042211+0000","last_fullsized":"2024-09-11T06:50:11.042211+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_clean_scrub_stamp":"2024-09-11T06:49:05.088839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:19:19.991300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T06:50:11.840248+0000","last_change":"2024-09-11T06:49:13.251115+0000","last_active":"2024-09-11T06:50:11.840248+0000","last_peered":"2024-09-11T06:50:11.840248+0000","last_clean":"2024-09-11T06:50:11.840248+0000","last_became_active":"2024-09-11T06:49:11.235231+0000","last_became_peered":"2024-09-11T06:49:11.235231+0000","last_unstale":"2024-09-11T06:50:11.840248+0000","last_undegraded":"2024-09-11T06:50:11.840248+0000","last_fullsized":"2024-09-11T06:50:11.840248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:01:21.028466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011212469999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T06:50:16.062872+0000","last_change":"2024-09-11T06:50:16.062872+0000","last_active":"2024-09-11T06:50:16.062872+0000","last_peered":"2024-09-11T06:50:16.062872+0000","last_clean":"2024-09-11T06:50:16.062872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T06:50:16.062872+0000","last_undegraded":"2024-09-11T06:50:16.062872+0000","last_fullsized":"2024-09-11T06:50:16.062872+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T06:50:11.042104+0000","last_change":"2024-09-11T06:49:55.951569+0000","last_active":"2024-09-11T06:50:11.042104+0000","last_peered":"2024-09-11T06:50:11.042104+0000","last_clean":"2024-09-11T06:50:11.042104+0000","last_became_active":"2024-09-11T06:49:55.950989+0000","last_became_peered":"2024-09-11T06:49:55.950989+0000","last_unstale":"2024-09-11T06:50:11.042104+0000","last_undegraded":"2024-09-11T06:50:11.042104+0000","last_fullsized":"2024-09-11T06:50:11.042104+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:07.743579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077615599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T06:50:16.062872+0000","last_change":"2024-09-11T06:50:16.062872+0000","last_active":"2024-09-11T06:50:16.062872+0000","last_peered":"2024-09-11T06:50:16.062872+0000","last_clean":"2024-09-11T06:50:16.062872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T06:50:16.062872+0000","last_undegraded":"2024-09-11T06:50:16.062872+0000","last_fullsized":"2024-09-11T06:50:16.062872+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":101,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T06:50:14.049715+0000","last_change":"2024-09-11T06:49:55.948809+0000","last_active":"2024-09-11T06:50:14.049715+0000","last_peered":"2024-09-11T06:50:14.049715+0000","last_clean":"2024-09-11T06:50:14.049715+0000","last_became_active":"2024-09-11T06:49:55.948499+0000","last_became_peered":"2024-09-11T06:49:55.948499+0000","last_unstale":"2024-09-11T06:50:14.049715+0000","last_undegraded":"2024-09-11T06:50:14.049715+0000","last_fullsized":"2024-09-11T06:50:14.049715+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:43:31.563009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010163780000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T06:50:16.062872+0000","last_change":"2024-09-11T06:50:16.062872+0000","last_active":"2024-09-11T06:50:16.062872+0000","last_peered":"2024-09-11T06:50:16.062872+0000","last_clean":"2024-09-11T06:50:16.062872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T06:50:16.062872+0000","last_undegraded":"2024-09-11T06:50:16.062872+0000","last_fullsized":"2024-09-11T06:50:16.062872+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T06:50:11.840223+0000","last_change":"2024-09-11T06:49:55.961845+0000","last_active":"2024-09-11T06:50:11.840223+0000","last_peered":"2024-09-11T06:50:11.840223+0000","last_clean":"2024-09-11T06:50:11.840223+0000","last_became_active":"2024-09-11T06:49:55.955837+0000","last_became_peered":"2024-09-11T06:49:55.955837+0000","last_unstale":"2024-09-11T06:50:11.840223+0000","last_undegraded":"2024-09-11T06:50:11.840223+0000","last_fullsized":"2024-09-11T06:50:11.840223+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:05:32.670233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001910244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T06:50:16.062872+0000","last_change":"2024-09-11T06:50:16.062872+0000","last_active":"2024-09-11T06:50:16.062872+0000","last_peered":"2024-09-11T06:50:16.062872+0000","last_clean":"2024-09-11T06:50:16.062872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T06:50:16.062872+0000","last_undegraded":"2024-09-11T06:50:16.062872+0000","last_fullsized":"2024-09-11T06:50:16.062872+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T06:50:16.062872+0000","last_change":"2024-09-11T06:50:16.062872+0000","last_active":"2024-09-11T06:50:16.062872+0000","last_peered":"2024-09-11T06:50:16.062872+0000","last_clean":"2024-09-11T06:50:16.062872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T06:50:16.062872+0000","last_undegraded":"2024-09-11T06:50:16.062872+0000","last_fullsized":"2024-09-11T06:50:16.062872+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T06:50:14.050690+0000","last_change":"2024-09-11T06:49:56.122331+0000","last_active":"2024-09-11T06:50:14.050690+0000","last_peered":"2024-09-11T06:50:14.050690+0000","last_clean":"2024-09-11T06:50:14.050690+0000","last_became_active":"2024-09-11T06:49:56.121998+0000","last_became_peered":"2024-09-11T06:49:56.121998+0000","last_unstale":"2024-09-11T06:50:14.050690+0000","last_undegraded":"2024-09-11T06:50:14.050690+0000","last_fullsized":"2024-09-11T06:50:14.050690+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:24.336670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T06:50:16.062872+0000","last_change":"2024-09-11T06:50:16.062872+0000","last_active":"2024-09-11T06:50:16.062872+0000","last_peered":"2024-09-11T06:50:16.062872+0000","last_clean":"2024-09-11T06:50:16.062872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T06:50:16.062872+0000","last_undegraded":"2024-09-11T06:50:16.062872+0000","last_fullsized":"2024-09-11T06:50:16.062872+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T06:50:11.042217+0000","last_change":"2024-09-11T06:50:06.933758+0000","last_active":"2024-09-11T06:50:11.042217+0000","last_peered":"2024-09-11T06:50:11.042217+0000","last_clean":"2024-09-11T06:50:11.042217+0000","last_became_active":"2024-09-11T06:50:06.932797+0000","last_became_peered":"2024-09-11T06:50:06.932797+0000","last_unstale":"2024-09-11T06:50:11.042217+0000","last_undegraded":"2024-09-11T06:50:11.042217+0000","last_fullsized":"2024-09-11T06:50:11.042217+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:22:15.086364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095172900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T06:50:16.062872+0000","last_change":"2024-09-11T06:50:16.062872+0000","last_active":"2024-09-11T06:50:16.062872+0000","last_peered":"2024-09-11T06:50:16.062872+0000","last_clean":"2024-09-11T06:50:16.062872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T06:50:16.062872+0000","last_undegraded":"2024-09-11T06:50:16.062872+0000","last_fullsized":"2024-09-11T06:50:16.062872+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":34,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T06:50:14.050783+0000","last_change":"2024-09-11T06:49:55.948281+0000","last_active":"2024-09-11T06:50:14.050783+0000","last_peered":"2024-09-11T06:50:14.050783+0000","last_clean":"2024-09-11T06:50:14.050783+0000","last_became_active":"2024-09-11T06:49:55.948006+0000","last_became_peered":"2024-09-11T06:49:55.948006+0000","last_unstale":"2024-09-11T06:50:14.050783+0000","last_undegraded":"2024-09-11T06:50:14.050783+0000","last_fullsized":"2024-09-11T06:50:14.050783+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:10:41.665226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T06:50:16.062872+0000","last_change":"2024-09-11T06:50:16.062872+0000","last_active":"2024-09-11T06:50:16.062872+0000","last_peered":"2024-09-11T06:50:16.062872+0000","last_clean":"2024-09-11T06:50:16.062872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T06:50:16.062872+0000","last_undegraded":"2024-09-11T06:50:16.062872+0000","last_fullsized":"2024-09-11T06:50:16.062872+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T06:50:11.042298+0000","last_change":"2024-09-11T06:49:13.393788+0000","last_active":"2024-09-11T06:50:11.042298+0000","last_peered":"2024-09-11T06:50:11.042298+0000","last_clean":"2024-09-11T06:50:11.042298+0000","last_became_active":"2024-09-11T06:49:11.433437+0000","last_became_peered":"2024-09-11T06:49:11.433437+0000","last_unstale":"2024-09-11T06:50:11.042298+0000","last_undegraded":"2024-09-11T06:50:11.042298+0000","last_fullsized":"2024-09-11T06:50:11.042298+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:20:04.339208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001403348,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640272,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":606208,"data_stored":217759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640273,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27616,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":602112,"data_stored":217740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640273,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28196,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343644,"statfs":{"total":96636764160,"available":96607891456,"internally_reserved":0,"allocated":1196032,"data_stored":808108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640274,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":602112,"data_stored":217740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640274,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":606208,"data_stored":217759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640275,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32356,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339484,"statfs":{"total":96636764160,"available":96603631616,"internally_reserved":0,"allocated":1196032,"data_stored":808108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:50:16.166 INFO:tasks.workunit.client.0.smithi176.stderr:pool 'new_default_pool21607' already exists 2024-09-11T06:50:16.167 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:16.167 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:16.183 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:50:16.184 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd pool init new_default_pool21607 2024-09-11T06:50:16.216 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:50:16.217 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:50:16.217 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:16.215+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:16.234 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:50:16.234 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:50:16.253 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:50:16.253 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:50:16.273 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:50:16.274 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:50:16.294 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:50:16.295 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:50:16.297 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:16.296+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:16.316 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:50:16.317 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:50:16.333 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:16.333 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:16.333 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:50:16.334 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:16.348 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:50:16.398 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:16.396+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:16.499 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:16.496+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:16.502 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:16.503 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:16.503 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:50:16.503 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:50:16.503 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:16.517 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:50:16.600 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:16.600+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:16.644 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:16.644 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:50:16.644 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:50:16.644 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:50:16.644 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:16.644 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:5 2.2 e17/15 2:5cea7035:::rbd_info:head v 17'1, mlcod=0'0)", 2024-09-11T06:50:16.645 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:50:16.645 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 64.407750182000001, 2024-09-11T06:50:16.645 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0019446719999999999, 2024-09-11T06:50:16.645 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:16.645 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:16.645 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:16.645 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:16.645 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:16.645 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:16.645 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:50:16.645 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:16.645 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:16.645 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:16.646 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:16.646 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:50:16.646 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:16.646 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:16.646 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:16.646 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:16.646 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236085+0000", 2024-09-11T06:50:16.646 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6000000000000003e-07 2024-09-11T06:50:16.646 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:16.646 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:16.646 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:16.646 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236108+0000", 2024-09-11T06:50:16.646 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3113e-05 2024-09-11T06:50:16.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:16.647 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:16.647 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:16.647 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236133+0000", 2024-09-11T06:50:16.647 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5723000000000001e-05 2024-09-11T06:50:16.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:16.647 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:16.647 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:16.647 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236154+0000", 2024-09-11T06:50:16.647 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0333999999999999e-05 2024-09-11T06:50:16.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:16.647 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:16.647 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:16.647 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236297+0000", 2024-09-11T06:50:16.648 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014278500000000001 2024-09-11T06:50:16.648 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:16.648 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:16.648 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:16.648 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236383+0000", 2024-09-11T06:50:16.648 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5927999999999995e-05 2024-09-11T06:50:16.648 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:16.648 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:16.648 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:16.648 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.237950+0000", 2024-09-11T06:50:16.648 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015673690000000001 2024-09-11T06:50:16.648 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:16.648 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:16.649 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:16.649 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.238029+0000", 2024-09-11T06:50:16.649 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8659999999999996e-05 2024-09-11T06:50:16.649 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:16.649 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:16.649 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:16.649 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:16.649 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:16.649 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:7 2.2 e18/15 2:5cea7035:::rbd_info:head v 18'2, mlcod=0'0)", 2024-09-11T06:50:16.649 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:50:16.649 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.393424975000002, 2024-09-11T06:50:16.649 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001907192, 2024-09-11T06:50:16.649 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:16.649 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:16.649 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:16.650 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:16.650 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:16.650 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:16.650 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:50:16.650 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:16.650 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:16.650 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:16.650 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:16.650 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:50:16.650 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:16.650 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:16.650 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:16.650 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:16.650 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250410+0000", 2024-09-11T06:50:16.651 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7700000000000004e-07 2024-09-11T06:50:16.651 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:16.651 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:16.651 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:16.651 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250445+0000", 2024-09-11T06:50:16.651 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4749000000000002e-05 2024-09-11T06:50:16.651 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:16.651 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:16.651 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:16.651 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250478+0000", 2024-09-11T06:50:16.651 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3269999999999998e-05 2024-09-11T06:50:16.651 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:16.651 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:16.651 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:16.652 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250501+0000", 2024-09-11T06:50:16.652 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3632999999999999e-05 2024-09-11T06:50:16.652 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:16.652 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:16.652 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:16.652 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250633+0000", 2024-09-11T06:50:16.652 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013143200000000001 2024-09-11T06:50:16.652 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:16.652 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:16.652 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:16.652 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250700+0000", 2024-09-11T06:50:16.652 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7002999999999995e-05 2024-09-11T06:50:16.652 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:16.652 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:16.653 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:16.653 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.252172+0000", 2024-09-11T06:50:16.653 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014719550000000001 2024-09-11T06:50:16.653 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:16.653 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:16.653 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:16.653 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.252316+0000", 2024-09-11T06:50:16.653 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144473 2024-09-11T06:50:16.653 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:16.653 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:16.653 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:16.653 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:16.653 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:16.653 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:16.659 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:50:16.701 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:16.700+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:16.782 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:16.782 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:16.782 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:50:16.782 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:16.796 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:50:16.841 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:16.839+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:16.903 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:16.899+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:16.913 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:16.914 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:16.914 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:50:16.914 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:50:16.914 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:16.929 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:50:17.004 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:17.004+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:17.080 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is unset 2024-09-11T06:50:17.105 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:17.105 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:50:17.105 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:50:17.105 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:50:17.105 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:17.105 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:1 2.3 e16/15 2:c4c92e5a:::rbd_trash:head v 16'1, mlcod=0'0)", 2024-09-11T06:50:17.105 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:50:17.106 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 65.868669779000001, 2024-09-11T06:50:17.106 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001234761, 2024-09-11T06:50:17.106 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:17.106 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:17.106 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:17.106 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:17.106 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:17.106 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:17.106 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:50:17.106 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:17.106 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:17.106 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:17.106 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:17.107 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:50:17.107 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:17.107 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:17.107 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:17.107 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:17.107 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236279+0000", 2024-09-11T06:50:17.107 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6000000000000003e-07 2024-09-11T06:50:17.107 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:17.107 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:17.107 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:17.107 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236296+0000", 2024-09-11T06:50:17.107 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6393999999999999e-05 2024-09-11T06:50:17.107 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:17.107 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:17.108 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:17.108 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236317+0000", 2024-09-11T06:50:17.108 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1243e-05 2024-09-11T06:50:17.108 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:17.108 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:17.108 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:17.108 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236333+0000", 2024-09-11T06:50:17.108 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.558e-05 2024-09-11T06:50:17.108 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:17.108 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:17.108 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:17.108 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236435+0000", 2024-09-11T06:50:17.108 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000102803 2024-09-11T06:50:17.108 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:17.108 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:17.109 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:17.109 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.236474+0000", 2024-09-11T06:50:17.109 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8383999999999997e-05 2024-09-11T06:50:17.109 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:17.109 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:17.109 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:17.109 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.237432+0000", 2024-09-11T06:50:17.109 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00095849300000000004 2024-09-11T06:50:17.109 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:17.109 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:17.109 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:17.109 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:11.237513+0000", 2024-09-11T06:50:17.109 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1204000000000004e-05 2024-09-11T06:50:17.110 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:17.110 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:17.110 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:17.110 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:17.110 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:17.110 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "MOSDPGPush(2.2 37/36 [PushOp(2:5cea7035:::rbd_info:head, version: 18'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@18'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:50:17.110 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:50.554243+0000", 2024-09-11T06:50:17.110 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 26.550705317999999, 2024-09-11T06:50:17.110 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00051362700000000001, 2024-09-11T06:50:17.110 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:17.110 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:17.110 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:17.110 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:17.110 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:17.111 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:17.111 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:50.554243+0000", 2024-09-11T06:50:17.111 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:17.111 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:17.111 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:17.111 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:17.111 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:50.554243+0000", 2024-09-11T06:50:17.111 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:17.111 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:17.111 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:17.111 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:17.111 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:50.554245+0000", 2024-09-11T06:50:17.111 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6759999999999999e-06 2024-09-11T06:50:17.112 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:17.112 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:17.112 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:17.112 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:50.554270+0000", 2024-09-11T06:50:17.112 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5244000000000001e-05 2024-09-11T06:50:17.112 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:17.112 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:17.112 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:17.112 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:50.554316+0000", 2024-09-11T06:50:17.112 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6164e-05 2024-09-11T06:50:17.112 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:17.112 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:17.112 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:17.113 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:50.554361+0000", 2024-09-11T06:50:17.113 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5102000000000003e-05 2024-09-11T06:50:17.113 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:17.113 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:17.113 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:17.113 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:50.554463+0000", 2024-09-11T06:50:17.113 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000101968 2024-09-11T06:50:17.113 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:17.113 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:17.113 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:17.113 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:50.554472+0000", 2024-09-11T06:50:17.113 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.816e-06 2024-09-11T06:50:17.113 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:17.114 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:17.114 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:17.114 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:50.554757+0000", 2024-09-11T06:50:17.114 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00028465700000000001 2024-09-11T06:50:17.114 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:17.114 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:17.114 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:17.114 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:17.114 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:17.114 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:17.114 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:17.103+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:17.120 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:50:17.206 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:17.203+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:17.250 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:17.251 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:17.251 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:50:17.251 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:17.265 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:50:17.311 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:17.312+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:17.383 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:17.383 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:17.383 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:50:17.383 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:50:17.384 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:17.399 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:50:17.409 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:17.407+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:17.510 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:17.507+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:17.572 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:17.572 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:17.572 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:17.573 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:50:17.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.573 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:17.573 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:50:17.573 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 71.398984861000002, 2024-09-11T06:50:17.573 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011420669, 2024-09-11T06:50:17.573 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:17.573 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:17.573 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:17.573 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:17.573 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:17.573 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:17.574 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 37 2024-09-11T06:50:17.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.574 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:17.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.574 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:17.574 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:50:17.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.574 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:17.574 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:50:17.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.575 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:17.575 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170509+0000", 2024-09-11T06:50:17.575 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.13e-06 2024-09-11T06:50:17.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.575 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:17.575 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170521+0000", 2024-09-11T06:50:17.575 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1776999999999999e-05 2024-09-11T06:50:17.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.575 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:17.575 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170538+0000", 2024-09-11T06:50:17.575 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7413000000000001e-05 2024-09-11T06:50:17.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.576 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:17.576 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170570+0000", 2024-09-11T06:50:17.576 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2313e-05 2024-09-11T06:50:17.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.576 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:17.576 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179768+0000", 2024-09-11T06:50:17.576 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0091978010000000002 2024-09-11T06:50:17.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.576 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:17.576 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179951+0000", 2024-09-11T06:50:17.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018227900000000001 2024-09-11T06:50:17.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:17.577 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.180175+0000", 2024-09-11T06:50:17.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022432800000000001 2024-09-11T06:50:17.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:17.577 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181112+0000", 2024-09-11T06:50:17.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00093712500000000002 2024-09-11T06:50:17.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:17.577 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181789+0000", 2024-09-11T06:50:17.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00067705100000000002 2024-09-11T06:50:17.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.578 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:17.578 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181871+0000", 2024-09-11T06:50:17.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1816000000000004e-05 2024-09-11T06:50:17.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.578 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:17.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181929+0000", 2024-09-11T06:50:17.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7636000000000002e-05 2024-09-11T06:50:17.579 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.579 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:17.579 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.579 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:17.579 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:17.579 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 71.297725900000003, 2024-09-11T06:50:17.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283336, 2024-09-11T06:50:17.579 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:17.579 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:17.580 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:17.580 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:17.580 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:17.580 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:17.580 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 87 2024-09-11T06:50:17.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.580 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:17.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.580 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:17.580 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:17.580 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.580 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:17.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:17.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.581 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:17.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271768+0000", 2024-09-11T06:50:17.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7599999999999996e-07 2024-09-11T06:50:17.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.581 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:17.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271860+0000", 2024-09-11T06:50:17.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2639999999999994e-05 2024-09-11T06:50:17.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:17.582 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271873+0000", 2024-09-11T06:50:17.582 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3139000000000001e-05 2024-09-11T06:50:17.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:17.582 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271882+0000", 2024-09-11T06:50:17.582 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2369999999999992e-06 2024-09-11T06:50:17.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:17.582 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277150+0000", 2024-09-11T06:50:17.582 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052684419999999999 2024-09-11T06:50:17.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:17.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277273+0000", 2024-09-11T06:50:17.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012278899999999999 2024-09-11T06:50:17.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:17.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277466+0000", 2024-09-11T06:50:17.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193578 2024-09-11T06:50:17.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:17.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.280808+0000", 2024-09-11T06:50:17.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003341971 2024-09-11T06:50:17.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.584 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:17.584 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284419+0000", 2024-09-11T06:50:17.584 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003610982 2024-09-11T06:50:17.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.584 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:17.584 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284531+0000", 2024-09-11T06:50:17.584 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011121299999999999 2024-09-11T06:50:17.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.584 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:17.584 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284600+0000", 2024-09-11T06:50:17.584 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9592999999999996e-05 2024-09-11T06:50:17.584 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.585 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:17.585 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.585 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:17.585 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:17.585 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 71.297595647999998, 2024-09-11T06:50:17.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283314, 2024-09-11T06:50:17.585 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:17.585 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:17.585 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:17.585 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:17.585 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:17.585 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:17.585 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 88 2024-09-11T06:50:17.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.586 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:17.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:17.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:17.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:17.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:17.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.587 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:17.587 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271898+0000", 2024-09-11T06:50:17.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2999999999999999e-07 2024-09-11T06:50:17.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.587 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:17.587 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271904+0000", 2024-09-11T06:50:17.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.767e-06 2024-09-11T06:50:17.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.587 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:17.587 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271913+0000", 2024-09-11T06:50:17.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7200000000000001e-06 2024-09-11T06:50:17.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.588 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:17.588 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271920+0000", 2024-09-11T06:50:17.588 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7100000000000001e-06 2024-09-11T06:50:17.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.588 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:17.588 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278684+0000", 2024-09-11T06:50:17.588 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0067642650000000002 2024-09-11T06:50:17.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.589 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:17.589 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278838+0000", 2024-09-11T06:50:17.589 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015371499999999999 2024-09-11T06:50:17.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.589 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:17.589 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.279040+0000", 2024-09-11T06:50:17.590 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020207499999999999 2024-09-11T06:50:17.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:17.590 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.281448+0000", 2024-09-11T06:50:17.590 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002407686 2024-09-11T06:50:17.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:17.590 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284615+0000", 2024-09-11T06:50:17.590 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031675499999999999 2024-09-11T06:50:17.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:17.591 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284682+0000", 2024-09-11T06:50:17.591 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7032000000000005e-05 2024-09-11T06:50:17.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.591 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:17.591 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284730+0000", 2024-09-11T06:50:17.591 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7790000000000002e-05 2024-09-11T06:50:17.591 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.591 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:17.591 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.591 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:17.591 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:50:17.592 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 71.129694426, 2024-09-11T06:50:17.592 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011114877, 2024-09-11T06:50:17.592 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:17.592 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:17.592 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:17.592 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:17.592 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:17.592 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:17.592 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 107 2024-09-11T06:50:17.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.592 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:17.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.592 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:17.593 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:50:17.593 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.593 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:17.593 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:50:17.593 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.593 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:17.593 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439799+0000", 2024-09-11T06:50:17.593 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.16e-07 2024-09-11T06:50:17.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.594 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:17.594 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439806+0000", 2024-09-11T06:50:17.594 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3640000000000004e-06 2024-09-11T06:50:17.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.594 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:17.594 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439816+0000", 2024-09-11T06:50:17.594 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0485999999999999e-05 2024-09-11T06:50:17.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.594 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:17.594 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439824+0000", 2024-09-11T06:50:17.594 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.6699999999999994e-06 2024-09-11T06:50:17.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.595 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:17.595 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443328+0000", 2024-09-11T06:50:17.595 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035044329999999999 2024-09-11T06:50:17.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.595 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:17.595 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443461+0000", 2024-09-11T06:50:17.595 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013272199999999999 2024-09-11T06:50:17.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.595 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:17.596 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443678+0000", 2024-09-11T06:50:17.596 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021674500000000001 2024-09-11T06:50:17.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.596 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:17.596 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449635+0000", 2024-09-11T06:50:17.596 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0059572119999999999 2024-09-11T06:50:17.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.596 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:17.596 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450750+0000", 2024-09-11T06:50:17.596 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001115654 2024-09-11T06:50:17.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.597 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:17.597 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450832+0000", 2024-09-11T06:50:17.597 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1188999999999998e-05 2024-09-11T06:50:17.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.597 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:17.597 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450913+0000", 2024-09-11T06:50:17.597 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1485999999999999e-05 2024-09-11T06:50:17.597 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.597 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:17.597 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.597 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:17.598 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:50:17.598 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 71.129103451000006, 2024-09-11T06:50:17.598 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011236556, 2024-09-11T06:50:17.598 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:17.598 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:17.598 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:17.598 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:17.598 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:17.598 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:17.598 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 109 2024-09-11T06:50:17.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.598 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:17.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.598 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:17.598 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:50:17.599 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.599 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:17.599 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:50:17.599 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.599 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:17.599 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440391+0000", 2024-09-11T06:50:17.599 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8360000000000001e-06 2024-09-11T06:50:17.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.599 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:17.600 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440560+0000", 2024-09-11T06:50:17.600 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000169085 2024-09-11T06:50:17.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.600 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:17.600 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440585+0000", 2024-09-11T06:50:17.600 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4530000000000001e-05 2024-09-11T06:50:17.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.600 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:17.600 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440606+0000", 2024-09-11T06:50:17.600 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1277e-05 2024-09-11T06:50:17.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.600 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:17.601 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446479+0000", 2024-09-11T06:50:17.601 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0058734989999999999 2024-09-11T06:50:17.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.601 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:17.601 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446649+0000", 2024-09-11T06:50:17.601 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016948199999999999 2024-09-11T06:50:17.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.601 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:17.601 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446854+0000", 2024-09-11T06:50:17.601 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020491500000000001 2024-09-11T06:50:17.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.602 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:17.602 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451380+0000", 2024-09-11T06:50:17.602 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0045264110000000001 2024-09-11T06:50:17.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.602 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:17.602 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451479+0000", 2024-09-11T06:50:17.602 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.8975999999999996e-05 2024-09-11T06:50:17.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.602 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:17.602 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451562+0000", 2024-09-11T06:50:17.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3213000000000004e-05 2024-09-11T06:50:17.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.603 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:17.603 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451626+0000", 2024-09-11T06:50:17.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3331999999999996e-05 2024-09-11T06:50:17.603 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.603 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:17.603 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.604 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:17.604 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:50:17.604 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 71.128854322999999, 2024-09-11T06:50:17.604 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011252866, 2024-09-11T06:50:17.604 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:17.604 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:17.604 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:17.604 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:17.604 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:17.604 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:17.604 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 110 2024-09-11T06:50:17.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.605 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:17.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.605 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:17.605 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:50:17.605 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.605 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:17.605 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:50:17.606 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.606 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.606 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:17.606 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440640+0000", 2024-09-11T06:50:17.606 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5430000000000001e-06 2024-09-11T06:50:17.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.606 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.606 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:17.606 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440651+0000", 2024-09-11T06:50:17.607 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0672999999999999e-05 2024-09-11T06:50:17.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.607 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:17.607 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440670+0000", 2024-09-11T06:50:17.607 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9726999999999999e-05 2024-09-11T06:50:17.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.607 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:17.607 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440691+0000", 2024-09-11T06:50:17.607 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0526e-05 2024-09-11T06:50:17.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.608 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:17.608 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448191+0000", 2024-09-11T06:50:17.608 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0075005990000000002 2024-09-11T06:50:17.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.608 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:17.608 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448390+0000", 2024-09-11T06:50:17.608 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000198815 2024-09-11T06:50:17.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.608 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:17.608 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448735+0000", 2024-09-11T06:50:17.609 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034424999999999999 2024-09-11T06:50:17.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.609 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:17.609 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451742+0000", 2024-09-11T06:50:17.609 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030077440000000001 2024-09-11T06:50:17.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.609 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:17.609 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451761+0000", 2024-09-11T06:50:17.609 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9196999999999999e-05 2024-09-11T06:50:17.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.610 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:17.610 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451834+0000", 2024-09-11T06:50:17.610 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2071999999999997e-05 2024-09-11T06:50:17.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.610 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:17.610 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451891+0000", 2024-09-11T06:50:17.610 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7720000000000003e-05 2024-09-11T06:50:17.610 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.610 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:17.610 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.610 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:17.611 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:17.611 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 71.128746448000001, 2024-09-11T06:50:17.611 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014944941, 2024-09-11T06:50:17.611 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:17.611 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:17.611 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:17.611 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:17.611 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:17.611 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:17.611 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 111 2024-09-11T06:50:17.611 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.611 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:17.612 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.612 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:17.612 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:17.612 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.612 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.612 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.612 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:17.612 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:17.612 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.612 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.612 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.612 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:17.612 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440748+0000", 2024-09-11T06:50:17.612 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:50:17.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.613 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.613 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:17.613 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440760+0000", 2024-09-11T06:50:17.613 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1897e-05 2024-09-11T06:50:17.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.613 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.613 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:17.613 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440782+0000", 2024-09-11T06:50:17.613 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2552999999999999e-05 2024-09-11T06:50:17.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.613 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.613 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:17.614 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440793+0000", 2024-09-11T06:50:17.614 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0643e-05 2024-09-11T06:50:17.614 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.614 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.614 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:17.614 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449755+0000", 2024-09-11T06:50:17.614 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089622620000000004 2024-09-11T06:50:17.614 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.614 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.614 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:17.614 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449918+0000", 2024-09-11T06:50:17.614 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163086 2024-09-11T06:50:17.614 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.614 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.615 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:17.615 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450150+0000", 2024-09-11T06:50:17.615 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023204500000000001 2024-09-11T06:50:17.615 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.615 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.615 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:17.615 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454533+0000", 2024-09-11T06:50:17.615 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0043831349999999998 2024-09-11T06:50:17.615 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.615 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.615 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:17.615 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455511+0000", 2024-09-11T06:50:17.615 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0009776050000000001 2024-09-11T06:50:17.615 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.616 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.616 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:17.616 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455597+0000", 2024-09-11T06:50:17.616 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6446000000000005e-05 2024-09-11T06:50:17.616 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.616 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.616 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:17.616 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455691+0000", 2024-09-11T06:50:17.616 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3832000000000006e-05 2024-09-11T06:50:17.616 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.616 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:17.616 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.616 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.616 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.617 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:17.617 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:17.617 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 71.128676685000002, 2024-09-11T06:50:17.617 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018885411000000001, 2024-09-11T06:50:17.617 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:17.617 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:17.617 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:17.617 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:17.617 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:17.617 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:17.617 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 112 2024-09-11T06:50:17.617 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.617 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:17.617 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.618 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:17.618 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:17.618 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.618 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.618 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.618 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:17.618 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:17.618 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.618 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.618 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.618 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:17.618 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440817+0000", 2024-09-11T06:50:17.618 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:50:17.618 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.619 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.619 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:17.619 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441003+0000", 2024-09-11T06:50:17.619 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018661200000000001 2024-09-11T06:50:17.619 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.619 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.619 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:17.619 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441027+0000", 2024-09-11T06:50:17.619 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3920000000000001e-05 2024-09-11T06:50:17.619 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.619 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.619 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:17.619 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441062+0000", 2024-09-11T06:50:17.619 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4663000000000003e-05 2024-09-11T06:50:17.619 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.620 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.620 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:17.620 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451981+0000", 2024-09-11T06:50:17.620 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010918926000000001 2024-09-11T06:50:17.620 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.620 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.620 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:17.620 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452111+0000", 2024-09-11T06:50:17.620 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129529 2024-09-11T06:50:17.620 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.620 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.620 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:17.620 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452311+0000", 2024-09-11T06:50:17.621 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020013799999999999 2024-09-11T06:50:17.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.621 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.621 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:17.621 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455809+0000", 2024-09-11T06:50:17.621 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003498443 2024-09-11T06:50:17.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.621 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.621 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:17.621 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459566+0000", 2024-09-11T06:50:17.621 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003757271 2024-09-11T06:50:17.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.621 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.621 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:17.622 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459640+0000", 2024-09-11T06:50:17.622 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3949000000000006e-05 2024-09-11T06:50:17.622 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.622 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.622 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:17.622 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459701+0000", 2024-09-11T06:50:17.622 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1117000000000005e-05 2024-09-11T06:50:17.622 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.622 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:17.622 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.622 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.622 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.622 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:17.622 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:17.623 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 71.128397833999998, 2024-09-11T06:50:17.623 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018760915, 2024-09-11T06:50:17.623 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:17.623 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:17.623 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:17.623 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:17.623 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:17.623 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:17.623 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 113 2024-09-11T06:50:17.623 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.623 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:17.623 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.623 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:17.623 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:17.624 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.624 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.624 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.624 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:17.624 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:17.624 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.624 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.624 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.624 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:17.624 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441096+0000", 2024-09-11T06:50:17.624 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4130000000000001e-06 2024-09-11T06:50:17.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.625 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:17.625 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441110+0000", 2024-09-11T06:50:17.625 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3900000000000001e-05 2024-09-11T06:50:17.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.625 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:17.625 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441131+0000", 2024-09-11T06:50:17.625 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0636999999999999e-05 2024-09-11T06:50:17.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.625 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:17.625 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441147+0000", 2024-09-11T06:50:17.626 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6512999999999999e-05 2024-09-11T06:50:17.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.626 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.626 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:17.626 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453274+0000", 2024-09-11T06:50:17.626 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012126764999999999 2024-09-11T06:50:17.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.626 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.626 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:17.626 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453498+0000", 2024-09-11T06:50:17.626 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000223568 2024-09-11T06:50:17.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.626 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.626 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:17.627 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453872+0000", 2024-09-11T06:50:17.627 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037467099999999998 2024-09-11T06:50:17.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.627 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:17.627 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458709+0000", 2024-09-11T06:50:17.627 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0048365680000000003 2024-09-11T06:50:17.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.627 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:17.627 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459716+0000", 2024-09-11T06:50:17.627 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010075240000000001 2024-09-11T06:50:17.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.627 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:17.628 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459779+0000", 2024-09-11T06:50:17.628 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2336999999999996e-05 2024-09-11T06:50:17.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.628 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.628 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:17.628 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459856+0000", 2024-09-11T06:50:17.628 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7019000000000002e-05 2024-09-11T06:50:17.628 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.628 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:17.628 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.628 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.629 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:17.629 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:17.629 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 71.128321095000004, 2024-09-11T06:50:17.629 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018963114, 2024-09-11T06:50:17.629 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:17.629 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:17.629 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:17.629 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:17.629 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:17.629 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:17.629 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 114 2024-09-11T06:50:17.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.629 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:17.629 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.630 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:17.630 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:17.630 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.630 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.630 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.630 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:17.630 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:17.630 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.630 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.630 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.630 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:17.630 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:17.630 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9299999999999996e-07 2024-09-11T06:50:17.630 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.631 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:17.631 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441180+0000", 2024-09-11T06:50:17.631 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7200000000000006e-06 2024-09-11T06:50:17.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.631 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:17.631 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441265+0000", 2024-09-11T06:50:17.631 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4753000000000007e-05 2024-09-11T06:50:17.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.631 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:17.631 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441288+0000", 2024-09-11T06:50:17.631 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2860000000000001e-05 2024-09-11T06:50:17.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.632 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:17.632 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454585+0000", 2024-09-11T06:50:17.632 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013297286 2024-09-11T06:50:17.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.632 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:17.632 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454714+0000", 2024-09-11T06:50:17.632 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000128465 2024-09-11T06:50:17.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.632 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:17.632 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454909+0000", 2024-09-11T06:50:17.633 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019522499999999999 2024-09-11T06:50:17.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.633 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:17.633 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459868+0000", 2024-09-11T06:50:17.633 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0049591239999999997 2024-09-11T06:50:17.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.633 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:17.633 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459988+0000", 2024-09-11T06:50:17.633 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000120532 2024-09-11T06:50:17.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.634 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.634 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:17.634 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460071+0000", 2024-09-11T06:50:17.634 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2256000000000007e-05 2024-09-11T06:50:17.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.634 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.634 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:17.634 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460135+0000", 2024-09-11T06:50:17.634 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3999999999999997e-05 2024-09-11T06:50:17.634 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.634 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:17.634 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.635 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.635 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:17.635 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:17.635 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 71.128163466000004, 2024-09-11T06:50:17.635 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.019036849000000002, 2024-09-11T06:50:17.635 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:17.635 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:17.635 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:17.635 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:17.635 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:17.635 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:17.635 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 115 2024-09-11T06:50:17.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.635 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:17.636 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.636 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:17.636 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:17.636 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.636 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.636 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:17.636 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:17.636 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.636 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.636 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:17.636 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441330+0000", 2024-09-11T06:50:17.636 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1930000000000001e-06 2024-09-11T06:50:17.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.637 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:17.637 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441544+0000", 2024-09-11T06:50:17.637 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021405799999999999 2024-09-11T06:50:17.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.637 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:17.637 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441572+0000", 2024-09-11T06:50:17.637 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7317e-05 2024-09-11T06:50:17.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.637 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:17.638 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441593+0000", 2024-09-11T06:50:17.638 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1702999999999999e-05 2024-09-11T06:50:17.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.638 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:17.638 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455891+0000", 2024-09-11T06:50:17.638 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01429708 2024-09-11T06:50:17.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.638 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:17.638 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456046+0000", 2024-09-11T06:50:17.638 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000155682 2024-09-11T06:50:17.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.638 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:17.639 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456242+0000", 2024-09-11T06:50:17.639 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000195692 2024-09-11T06:50:17.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.639 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:17.639 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459876+0000", 2024-09-11T06:50:17.639 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036336480000000002 2024-09-11T06:50:17.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.639 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:17.639 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460245+0000", 2024-09-11T06:50:17.639 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036901700000000002 2024-09-11T06:50:17.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.640 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:17.640 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460316+0000", 2024-09-11T06:50:17.640 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1846000000000003e-05 2024-09-11T06:50:17.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.640 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.640 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:17.640 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460366+0000", 2024-09-11T06:50:17.640 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9613000000000001e-05 2024-09-11T06:50:17.640 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.640 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:17.640 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.640 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.640 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:17.641 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:17.641 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 71.127841386, 2024-09-11T06:50:17.641 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01904432, 2024-09-11T06:50:17.641 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:17.641 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:17.641 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:17.641 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:17.641 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:17.641 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:17.641 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 116 2024-09-11T06:50:17.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.641 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:17.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.641 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:17.642 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:17.642 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.642 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:17.642 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:17.642 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.642 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:17.642 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441653+0000", 2024-09-11T06:50:17.642 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:50:17.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.643 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:17.643 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441665+0000", 2024-09-11T06:50:17.643 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1653e-05 2024-09-11T06:50:17.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.643 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:17.643 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441684+0000", 2024-09-11T06:50:17.643 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9490000000000001e-05 2024-09-11T06:50:17.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.643 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:17.643 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441699+0000", 2024-09-11T06:50:17.643 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5044e-05 2024-09-11T06:50:17.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.644 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:17.644 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457297+0000", 2024-09-11T06:50:17.644 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015598241000000001 2024-09-11T06:50:17.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.644 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:17.644 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457577+0000", 2024-09-11T06:50:17.644 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027930799999999998 2024-09-11T06:50:17.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.644 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:17.644 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457957+0000", 2024-09-11T06:50:17.645 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00038088 2024-09-11T06:50:17.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:17.645 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460432+0000", 2024-09-11T06:50:17.645 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024743349999999998 2024-09-11T06:50:17.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:17.645 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460584+0000", 2024-09-11T06:50:17.645 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015232900000000001 2024-09-11T06:50:17.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:17.646 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460646+0000", 2024-09-11T06:50:17.646 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1883000000000004e-05 2024-09-11T06:50:17.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.646 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:17.646 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460696+0000", 2024-09-11T06:50:17.646 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9589999999999998e-05 2024-09-11T06:50:17.646 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.646 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:17.646 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.646 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:17.646 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:17.646 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 71.127769319999999, 2024-09-11T06:50:17.647 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.021834473, 2024-09-11T06:50:17.647 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:17.647 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:17.647 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:17.647 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:17.647 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:17.647 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:17.647 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 117 2024-09-11T06:50:17.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.647 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:17.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.647 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:17.647 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:17.647 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.648 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:17.648 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:17.648 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.648 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:17.648 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441725+0000", 2024-09-11T06:50:17.648 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5069999999999999e-06 2024-09-11T06:50:17.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.648 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:17.648 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441738+0000", 2024-09-11T06:50:17.649 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2813000000000001e-05 2024-09-11T06:50:17.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.649 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:17.649 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441759+0000", 2024-09-11T06:50:17.649 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1246999999999998e-05 2024-09-11T06:50:17.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.649 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:17.649 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441779+0000", 2024-09-11T06:50:17.649 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0392999999999999e-05 2024-09-11T06:50:17.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.650 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:17.650 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458804+0000", 2024-09-11T06:50:17.650 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.017024182999999998 2024-09-11T06:50:17.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.650 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:17.650 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458977+0000", 2024-09-11T06:50:17.650 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000173926 2024-09-11T06:50:17.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.650 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:17.650 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459170+0000", 2024-09-11T06:50:17.650 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000192184 2024-09-11T06:50:17.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.651 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:17.651 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.462300+0000", 2024-09-11T06:50:17.651 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031304969999999999 2024-09-11T06:50:17.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.651 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:17.651 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463408+0000", 2024-09-11T06:50:17.651 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011077839999999999 2024-09-11T06:50:17.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.651 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:17.651 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463497+0000", 2024-09-11T06:50:17.651 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8828999999999999e-05 2024-09-11T06:50:17.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.652 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:17.652 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463558+0000", 2024-09-11T06:50:17.652 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1110000000000003e-05 2024-09-11T06:50:17.652 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.652 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:17.652 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.652 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:17.652 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:50:17.652 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 70.751261631999995, 2024-09-11T06:50:17.652 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012216898, 2024-09-11T06:50:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 137 2024-09-11T06:50:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:50:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.654 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:17.654 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:50:17.654 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.654 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:17.654 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818232+0000", 2024-09-11T06:50:17.654 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2839999999999999e-06 2024-09-11T06:50:17.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.654 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:17.654 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818326+0000", 2024-09-11T06:50:17.654 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3531999999999998e-05 2024-09-11T06:50:17.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.655 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:17.655 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818337+0000", 2024-09-11T06:50:17.655 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0655999999999999e-05 2024-09-11T06:50:17.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.655 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:17.655 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818342+0000", 2024-09-11T06:50:17.655 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4600000000000002e-06 2024-09-11T06:50:17.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.655 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:17.655 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823890+0000", 2024-09-11T06:50:17.655 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0055480700000000004 2024-09-11T06:50:17.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.656 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:17.656 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823953+0000", 2024-09-11T06:50:17.656 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2848999999999994e-05 2024-09-11T06:50:17.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.656 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:17.656 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824074+0000", 2024-09-11T06:50:17.656 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121309 2024-09-11T06:50:17.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.656 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:17.656 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828043+0000", 2024-09-11T06:50:17.657 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0039685240000000002 2024-09-11T06:50:17.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.657 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:17.657 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830293+0000", 2024-09-11T06:50:17.657 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0022503179999999998 2024-09-11T06:50:17.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.657 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:17.657 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830366+0000", 2024-09-11T06:50:17.657 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2539999999999993e-05 2024-09-11T06:50:17.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.657 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:17.658 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830448+0000", 2024-09-11T06:50:17.658 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2355999999999996e-05 2024-09-11T06:50:17.658 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.658 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:17.658 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.658 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:17.658 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:17.658 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 70.751127818000001, 2024-09-11T06:50:17.658 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013136297, 2024-09-11T06:50:17.658 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:17.658 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:17.658 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 138 2024-09-11T06:50:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.660 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:17.660 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:17.660 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.89e-07 2024-09-11T06:50:17.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.660 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:17.660 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818369+0000", 2024-09-11T06:50:17.660 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4170000000000001e-06 2024-09-11T06:50:17.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.660 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:17.660 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818376+0000", 2024-09-11T06:50:17.661 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4270000000000002e-06 2024-09-11T06:50:17.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.661 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.661 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:17.661 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818383+0000", 2024-09-11T06:50:17.661 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3060000000000003e-06 2024-09-11T06:50:17.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.661 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.661 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:17.661 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824783+0000", 2024-09-11T06:50:17.661 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064003020000000001 2024-09-11T06:50:17.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.661 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.662 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:17.662 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825013+0000", 2024-09-11T06:50:17.662 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023039499999999999 2024-09-11T06:50:17.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.662 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.662 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:17.663 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825484+0000", 2024-09-11T06:50:17.663 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000470562 2024-09-11T06:50:17.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.663 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:17.663 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828633+0000", 2024-09-11T06:50:17.663 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031496689999999999 2024-09-11T06:50:17.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831413+0000", 2024-09-11T06:50:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027793219999999999 2024-09-11T06:50:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831450+0000", 2024-09-11T06:50:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7370999999999998e-05 2024-09-11T06:50:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831501+0000", 2024-09-11T06:50:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1036999999999999e-05 2024-09-11T06:50:17.665 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.665 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:17.665 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.665 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.665 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.665 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:17.665 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:17.665 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 70.751103337999993, 2024-09-11T06:50:17.665 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013223011, 2024-09-11T06:50:17.665 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:17.665 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:17.665 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:17.665 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:17.666 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:17.666 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:17.666 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 139 2024-09-11T06:50:17.666 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.666 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:17.666 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.666 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:17.666 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:17.666 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.666 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.666 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.666 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:17.666 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:17.666 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.666 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.667 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.667 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:17.667 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818390+0000", 2024-09-11T06:50:17.667 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5600000000000001e-07 2024-09-11T06:50:17.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.667 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.667 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:17.667 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818394+0000", 2024-09-11T06:50:17.667 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5929999999999999e-06 2024-09-11T06:50:17.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.667 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.667 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:17.667 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818399+0000", 2024-09-11T06:50:17.668 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7740000000000004e-06 2024-09-11T06:50:17.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.668 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:17.668 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818412+0000", 2024-09-11T06:50:17.668 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2625e-05 2024-09-11T06:50:17.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.668 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:17.668 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826205+0000", 2024-09-11T06:50:17.668 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0077930730000000002 2024-09-11T06:50:17.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.668 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:17.669 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826281+0000", 2024-09-11T06:50:17.669 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5876000000000003e-05 2024-09-11T06:50:17.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.669 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.669 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:17.669 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826384+0000", 2024-09-11T06:50:17.669 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103838 2024-09-11T06:50:17.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.669 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.669 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:17.669 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830642+0000", 2024-09-11T06:50:17.669 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004257083 2024-09-11T06:50:17.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.670 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:17.670 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831509+0000", 2024-09-11T06:50:17.670 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086730900000000003 2024-09-11T06:50:17.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.670 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:17.670 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831547+0000", 2024-09-11T06:50:17.670 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7793000000000002e-05 2024-09-11T06:50:17.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.671 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:17.671 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831612+0000", 2024-09-11T06:50:17.671 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5691000000000005e-05 2024-09-11T06:50:17.671 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.671 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:17.671 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.671 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.671 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:17.671 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:17.671 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 70.751074643999999, 2024-09-11T06:50:17.671 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016098024999999998, 2024-09-11T06:50:17.671 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:17.671 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:17.672 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:17.672 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:17.672 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:17.672 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:17.672 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 140 2024-09-11T06:50:17.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.672 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:17.672 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.672 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:17.672 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:17.672 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.672 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.673 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:17.673 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:17.673 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.673 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:17.673 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:17.673 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5199999999999998e-07 2024-09-11T06:50:17.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.673 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:17.673 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818479+0000", 2024-09-11T06:50:17.673 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0368000000000002e-05 2024-09-11T06:50:17.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.674 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.674 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:17.674 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818487+0000", 2024-09-11T06:50:17.674 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2339999999999994e-06 2024-09-11T06:50:17.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.674 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.674 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:17.674 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818491+0000", 2024-09-11T06:50:17.674 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4329999999999997e-06 2024-09-11T06:50:17.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.674 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.674 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:17.674 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828080+0000", 2024-09-11T06:50:17.674 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0095890449999999992 2024-09-11T06:50:17.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.675 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.675 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:17.675 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828130+0000", 2024-09-11T06:50:17.675 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9239000000000001e-05 2024-09-11T06:50:17.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.675 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.675 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:17.675 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828208+0000", 2024-09-11T06:50:17.675 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.852e-05 2024-09-11T06:50:17.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.675 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.675 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:17.675 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831625+0000", 2024-09-11T06:50:17.676 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034169410000000002 2024-09-11T06:50:17.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.676 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:17.676 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834411+0000", 2024-09-11T06:50:17.676 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002786111 2024-09-11T06:50:17.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.676 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:17.676 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834457+0000", 2024-09-11T06:50:17.676 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6041000000000002e-05 2024-09-11T06:50:17.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.676 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:17.677 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834516+0000", 2024-09-11T06:50:17.677 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8740999999999999e-05 2024-09-11T06:50:17.677 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.677 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:17.677 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.677 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:17.677 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:17.677 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 70.750993577000003, 2024-09-11T06:50:17.677 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016588772000000002, 2024-09-11T06:50:17.677 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:17.677 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:17.677 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:17.678 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:17.678 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:17.678 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:17.678 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 141 2024-09-11T06:50:17.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.678 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:17.678 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.678 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:17.678 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:17.678 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.678 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.678 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:17.679 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:17.679 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.679 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:17.679 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818500+0000", 2024-09-11T06:50:17.679 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0800000000000005e-07 2024-09-11T06:50:17.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.679 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:17.679 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818505+0000", 2024-09-11T06:50:17.679 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2429999999999996e-06 2024-09-11T06:50:17.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.680 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:17.680 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818510+0000", 2024-09-11T06:50:17.680 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9339999999999997e-06 2024-09-11T06:50:17.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.680 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:17.680 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818514+0000", 2024-09-11T06:50:17.680 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7270000000000001e-06 2024-09-11T06:50:17.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.680 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:17.680 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828711+0000", 2024-09-11T06:50:17.681 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010197031 2024-09-11T06:50:17.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.681 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:17.681 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828853+0000", 2024-09-11T06:50:17.681 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014201200000000001 2024-09-11T06:50:17.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.681 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:17.681 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.829115+0000", 2024-09-11T06:50:17.681 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026257500000000003 2024-09-11T06:50:17.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.682 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:17.682 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831629+0000", 2024-09-11T06:50:17.682 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025138169999999998 2024-09-11T06:50:17.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.682 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:17.682 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834991+0000", 2024-09-11T06:50:17.682 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033624459999999998 2024-09-11T06:50:17.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.682 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:17.682 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835032+0000", 2024-09-11T06:50:17.682 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1001000000000003e-05 2024-09-11T06:50:17.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.683 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.683 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:17.683 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835088+0000", 2024-09-11T06:50:17.683 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5478e-05 2024-09-11T06:50:17.683 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.683 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:17.683 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.683 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.683 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:50:17.683 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:50:17.683 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 70.127358674000007, 2024-09-11T06:50:17.683 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011161912, 2024-09-11T06:50:17.683 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:17.684 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:17.684 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:17.684 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:17.684 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:17.684 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:17.684 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 158 2024-09-11T06:50:17.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.684 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:17.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.684 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:17.684 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:50:17.684 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.685 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:17.685 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:50:17.685 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.685 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:17.685 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442135+0000", 2024-09-11T06:50:17.685 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3e-07 2024-09-11T06:50:17.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.685 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:17.685 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442259+0000", 2024-09-11T06:50:17.685 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000123903 2024-09-11T06:50:17.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.686 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:17.686 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442289+0000", 2024-09-11T06:50:17.686 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0833000000000001e-05 2024-09-11T06:50:17.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.686 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:17.686 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442301+0000", 2024-09-11T06:50:17.686 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1433e-05 2024-09-11T06:50:17.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.686 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:17.686 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449796+0000", 2024-09-11T06:50:17.686 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0074954959999999999 2024-09-11T06:50:17.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.687 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:17.687 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449917+0000", 2024-09-11T06:50:17.687 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121033 2024-09-11T06:50:17.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.687 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:17.687 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.450166+0000", 2024-09-11T06:50:17.687 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00024832400000000002 2024-09-11T06:50:17.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.688 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:17.688 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.452630+0000", 2024-09-11T06:50:17.688 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002464188 2024-09-11T06:50:17.688 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.688 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:17.688 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453149+0000", 2024-09-11T06:50:17.688 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00051860999999999997 2024-09-11T06:50:17.688 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.688 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:17.688 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453234+0000", 2024-09-11T06:50:17.688 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.5865000000000005e-05 2024-09-11T06:50:17.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.689 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:17.689 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453296+0000", 2024-09-11T06:50:17.689 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1496999999999998e-05 2024-09-11T06:50:17.689 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.689 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:17.689 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.689 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:50:17.689 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:50:17.689 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 70.127172572000006, 2024-09-11T06:50:17.689 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011561900999999999, 2024-09-11T06:50:17.690 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:17.690 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:17.690 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:17.690 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:17.690 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:17.690 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:17.690 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 159 2024-09-11T06:50:17.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.690 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:17.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.690 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:17.690 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:50:17.690 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.691 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:17.691 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:50:17.691 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:17.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.691 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:17.691 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442321+0000", 2024-09-11T06:50:17.691 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8400000000000003e-07 2024-09-11T06:50:17.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.691 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:17.691 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442328+0000", 2024-09-11T06:50:17.691 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5130000000000002e-06 2024-09-11T06:50:17.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.692 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:17.692 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442341+0000", 2024-09-11T06:50:17.692 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.363e-05 2024-09-11T06:50:17.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.692 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:17.692 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442349+0000", 2024-09-11T06:50:17.692 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4760000000000001e-06 2024-09-11T06:50:17.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.692 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:17.692 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451257+0000", 2024-09-11T06:50:17.693 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089088569999999992 2024-09-11T06:50:17.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.693 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:17.693 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451389+0000", 2024-09-11T06:50:17.693 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013110899999999999 2024-09-11T06:50:17.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.693 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:17.693 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451608+0000", 2024-09-11T06:50:17.693 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000219588 2024-09-11T06:50:17.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.693 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:17.693 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453517+0000", 2024-09-11T06:50:17.694 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0019088779999999999 2024-09-11T06:50:17.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.694 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:17.694 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453727+0000", 2024-09-11T06:50:17.694 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020955100000000001 2024-09-11T06:50:17.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.694 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:17.694 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453815+0000", 2024-09-11T06:50:17.694 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8858999999999997e-05 2024-09-11T06:50:17.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:17.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:17.694 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:17.695 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453882+0000", 2024-09-11T06:50:17.695 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.6556000000000004e-05 2024-09-11T06:50:17.695 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.695 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:17.695 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.695 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:17.695 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:17.695 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:17.697 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:50:17.697 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:17.696+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:17.798 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:17.795+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:17.868 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:17.868 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:17.868 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:50:17.868 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:17.882 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:50:17.899 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:17.895+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:18.000 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:18.000+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:18.004 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:18.004 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:18.005 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:50:18.005 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:50:18.005 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:18.019 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:50:18.100 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:18.095+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:18.181 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:18.181 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:18.182 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:18.182 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:50:18.182 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:18.182 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4558.0:1 3.3 3:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-11T06:50:18.182 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:50:18.182 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 1.0922131770000001, 2024-09-11T06:50:18.182 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0085520600000000002, 2024-09-11T06:50:18.182 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:18.182 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:18.182 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:18.182 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:18.182 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4558", 2024-09-11T06:50:18.182 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1478388898", 2024-09-11T06:50:18.183 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:50:18.183 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:18.183 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:18.183 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:18.183 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:18.183 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:50:18.183 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:18.183 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:18.183 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:18.183 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:18.183 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:50:18.183 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:18.183 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:18.183 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:18.184 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:18.184 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090533+0000", 2024-09-11T06:50:18.184 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2629999999999999e-06 2024-09-11T06:50:18.184 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:18.184 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:18.184 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:18.184 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090546+0000", 2024-09-11T06:50:18.184 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2918999999999999e-05 2024-09-11T06:50:18.184 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:18.184 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:18.184 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:18.184 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090572+0000", 2024-09-11T06:50:18.184 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6412000000000001e-05 2024-09-11T06:50:18.185 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:18.185 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:18.185 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:18.185 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090594+0000", 2024-09-11T06:50:18.185 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2476e-05 2024-09-11T06:50:18.185 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:18.185 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:18.185 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:18.185 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090938+0000", 2024-09-11T06:50:18.185 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034325099999999998 2024-09-11T06:50:18.185 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:18.185 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:18.185 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:18.185 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.091149+0000", 2024-09-11T06:50:18.185 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021151300000000001 2024-09-11T06:50:18.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:18.186 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:18.186 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:18.186 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.091377+0000", 2024-09-11T06:50:18.186 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022785399999999999 2024-09-11T06:50:18.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:18.186 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:18.186 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:18.186 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.092912+0000", 2024-09-11T06:50:18.186 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0015346089999999999 2024-09-11T06:50:18.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:18.186 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:18.186 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:18.186 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098919+0000", 2024-09-11T06:50:18.187 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0060076799999999996 2024-09-11T06:50:18.187 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:18.187 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:18.187 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:18.187 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.099009+0000", 2024-09-11T06:50:18.187 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0056000000000006e-05 2024-09-11T06:50:18.187 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:18.187 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:18.187 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:18.187 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.099081+0000", 2024-09-11T06:50:18.187 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2026999999999993e-05 2024-09-11T06:50:18.187 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:18.187 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:18.187 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:18.188 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:18.188 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:18.188 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop_reply(client.4558.0:1 3.3 e56/55 ondisk, result = 0)", 2024-09-11T06:50:18.188 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:17.098705+0000", 2024-09-11T06:50:18.188 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 1.084038023, 2024-09-11T06:50:18.188 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00039016499999999999, 2024-09-11T06:50:18.188 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:18.188 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:18.188 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:18.188 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:18.188 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:18.188 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:18.188 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098705+0000", 2024-09-11T06:50:18.188 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:18.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:18.189 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:18.189 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:18.189 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098705+0000", 2024-09-11T06:50:18.189 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:18.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:18.189 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:18.189 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:18.189 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098706+0000", 2024-09-11T06:50:18.189 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.626e-06 2024-09-11T06:50:18.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:18.189 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:18.189 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:18.189 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098717+0000", 2024-09-11T06:50:18.189 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0699e-05 2024-09-11T06:50:18.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:18.190 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:18.190 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:18.190 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098743+0000", 2024-09-11T06:50:18.190 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6449e-05 2024-09-11T06:50:18.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:18.190 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:18.190 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:18.190 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098759+0000", 2024-09-11T06:50:18.190 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5815999999999999e-05 2024-09-11T06:50:18.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:18.190 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:18.190 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:18.190 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098894+0000", 2024-09-11T06:50:18.191 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013442099999999999 2024-09-11T06:50:18.191 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:18.191 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:18.191 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:18.191 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098910+0000", 2024-09-11T06:50:18.191 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6696000000000001e-05 2024-09-11T06:50:18.191 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:18.191 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:18.191 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:18.191 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.099095+0000", 2024-09-11T06:50:18.191 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000184458 2024-09-11T06:50:18.191 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:18.191 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:18.191 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:18.192 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:18.192 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:18.192 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:18.195 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:50:18.239 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:18.240+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:18.302 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:18.299+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:18.313 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:18.313 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:18.314 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:50:18.314 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:18.327 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:50:18.333 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:50:18.403 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:18.399+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:18.483 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:50:18.484 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:50:18.489 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:18.489 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:18.489 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:50:18.490 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:50:18.490 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:18.499 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:50:18.500 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:50:18.503 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:50:18.504 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:18.499+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:18.515 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:50:18.516 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:50:18.534 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:50:18.534 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:50:18.551 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:50:18.552 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:50:18.571 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:50:18.572 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:50:18.624 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:18.624 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:18.637 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:18.635+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:18.638 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:50:18.705 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:18.704+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:18.764 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:18.765 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:18.778 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:50:18.805 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:18.804+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:18.899 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:18.900 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:18.906 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:18.903+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:18.915 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:50:19.007 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:19.004+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:19.079 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:19.079 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:19.095 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:50:19.110 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false rbd-nbd 2024-09-11T06:50:19.110 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd-nbd 2024-09-11T06:50:19.115 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T06:50:19.141 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:19.139+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:19.150 INFO:tasks.workunit.client.0.smithi176.stderr:rbd-nbd: must specify command 2024-09-11T06:50:19.152 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:19.152 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false rbd-nbd INVALIDCMD 2024-09-11T06:50:19.152 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd-nbd INVALIDCMD 2024-09-11T06:50:19.178 INFO:tasks.workunit.client.0.smithi176.stderr:rbd-nbd: unknown command: INVALIDCMD 2024-09-11T06:50:19.179 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:19.180 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:19.181 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -ne 0 ']' 2024-09-11T06:50:19.181 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false rbd device --device-type nbd map testrbdnbd21607 2024-09-11T06:50:19.182 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd map testrbdnbd21607 2024-09-11T06:50:19.209 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:19.208+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:19.214 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:19.214 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:19.230 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:50:19.289 INFO:tasks.workunit.client.0.smithi176.stderr:modprobe: ERROR: could not insert 'nbd': Operation not permitted 2024-09-11T06:50:19.292 INFO:tasks.workunit.client.0.smithi176.stderr:rbd-nbd: Netlink interface not supported. Using ioctl interface. 2024-09-11T06:50:19.292 INFO:tasks.workunit.client.0.smithi176.stderr:rbd-nbd: failed to find unused device 2024-09-11T06:50:19.300 INFO:tasks.workunit.client.0.smithi176.stderr:rbd: rbd-nbd failed with error: /usr/bin/rbd-nbd: exit status: 1 2024-09-11T06:50:19.304 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:19.304 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false _sudo rbd device --device-type nbd map INVALIDIMAGE 2024-09-11T06:50:19.304 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device --device-type nbd map INVALIDIMAGE 2024-09-11T06:50:19.304 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:19.304 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:19.305 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:19.305 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:50:19.306 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:50:19.306 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:19.307 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd map INVALIDIMAGE 2024-09-11T06:50:19.310 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:19.307+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:19.355 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:19.355 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:19.370 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:50:19.397 INFO:tasks.workunit.client.0.smithi176.stderr:rbd: rbd-nbd failed with error: /usr/bin/rbd-nbd: exit status: 1 2024-09-11T06:50:19.402 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:19.403 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false _sudo rbd-nbd --device INVALIDDEV map testrbdnbd21607 2024-09-11T06:50:19.403 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd-nbd --device INVALIDDEV map testrbdnbd21607 2024-09-11T06:50:19.403 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:19.403 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:19.403 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:19.403 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd-nbd 2024-09-11T06:50:19.405 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd-nbd 2024-09-11T06:50:19.405 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:19.405 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd-nbd --device INVALIDDEV map testrbdnbd21607 2024-09-11T06:50:19.411 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:19.408+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:19.478 INFO:tasks.workunit.client.0.smithi176.stderr:rbd-nbd: invalid device path: INVALIDDEV (expected /dev/nbd{num}) 2024-09-11T06:50:19.489 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:19.489 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false rbd device --device-type nbd --format INVALID list 2024-09-11T06:50:19.489 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd --format INVALID list 2024-09-11T06:50:19.511 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:19.507+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:19.514 INFO:tasks.workunit.client.0.smithi176.stderr:rbd: the argument for option '--format' is invalid 2024-09-11T06:50:19.515 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:19.516 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd --format json --pretty-format list 2024-09-11T06:50:19.540 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:19.541 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:19.555 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:50:19.563 INFO:tasks.workunit.client.0.smithi176.stdout:[] 2024-09-11T06:50:19.566 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd --format xml list 2024-09-11T06:50:19.615 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo rbd device --device-type nbd map rbd/testrbdnbd21607 2024-09-11T06:50:19.615 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:50:19.615 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:50:19.615 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:50:19.616 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which rbd 2024-09-11T06:50:19.618 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/rbd 2024-09-11T06:50:19.618 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:50:19.618 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map rbd/testrbdnbd21607 2024-09-11T06:50:19.641 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:19.639+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:19.713 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:19.712+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:19.725 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:19.726 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:19.739 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:50:19.744 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV=/dev/nbd0 2024-09-11T06:50:19.744 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd 2024-09-11T06:50:19.744 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:50:19.744 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:50:19.745 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:50:19.745 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd0'\'']/id' 2024-09-11T06:50:19.814 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=21912 2024-09-11T06:50:19.814 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 21912 2024-09-11T06:50:19.814 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 21912 -C rbd-nbd 2024-09-11T06:50:19.819 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:50:19.819 INFO:tasks.workunit.client.0.smithi176.stdout: 21912 ? 00:00:00 rbd-nbd 2024-09-11T06:50:19.820 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false _sudo rbd-nbd --device /dev/nbd0 map rbd/testrbdnbd21607 2024-09-11T06:50:19.820 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd-nbd --device /dev/nbd0 map rbd/testrbdnbd21607 2024-09-11T06:50:19.820 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:19.820 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:19.821 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:19.821 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd-nbd 2024-09-11T06:50:19.822 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd-nbd 2024-09-11T06:50:19.822 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:19.822 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd-nbd --device /dev/nbd0 map rbd/testrbdnbd21607 2024-09-11T06:50:19.825 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:19.823+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:19.899 INFO:tasks.workunit.client.0.smithi176.stderr:rbd-nbd: netlink connect failed: Object busy 2024-09-11T06:50:19.906 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:19.906 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:19.908 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:19.908 INFO:tasks.workunit.client.0.smithi176.stderr:+ dev1=/dev/nbd0 2024-09-11T06:50:19.908 INFO:tasks.workunit.client.0.smithi176.stderr:+ unmap_device /dev/nbd0 21912 2024-09-11T06:50:19.909 INFO:tasks.workunit.client.0.smithi176.stderr:+ local args=/dev/nbd0 2024-09-11T06:50:19.909 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pid=21912 2024-09-11T06:50:19.909 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd0 2024-09-11T06:50:19.909 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:19.909 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:19.909 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:19.909 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:50:19.910 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:50:19.910 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:19.910 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd0 2024-09-11T06:50:19.915 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:19.911+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:19.921 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:50:19.979 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^21912\b' 2024-09-11T06:50:19.980 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^21912\b' 2024-09-11T06:50:19.980 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:50:20.015 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:20.011+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:20.030 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:20.031 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -C rbd-nbd 2024-09-11T06:50:20.031 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^ *21912\b' 2024-09-11T06:50:20.031 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^ *21912\b' 2024-09-11T06:50:20.036 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:20.037 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 0.5 2024-09-11T06:50:20.089 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:20.089 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:20.094 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is set 2024-09-11T06:50:20.107 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:50:20.118 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:20.115+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:20.216 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:20.216+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:20.237 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:20.237 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:20.251 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:50:20.318 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:20.315+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:20.419 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:20.415+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:20.420 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:20.420 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:20.434 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:50:20.520 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:20.515+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:20.538 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV= 2024-09-11T06:50:20.538 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo rbd-nbd --device /dev/nbd0 map rbd/testrbdnbd21607 2024-09-11T06:50:20.538 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:50:20.539 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:50:20.539 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:50:20.540 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which rbd-nbd 2024-09-11T06:50:20.543 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/rbd-nbd 2024-09-11T06:50:20.543 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:50:20.543 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/rbd-nbd --device /dev/nbd0 map rbd/testrbdnbd21607 2024-09-11T06:50:20.556 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:20.556 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:20.570 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:50:20.590 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:50:20.621 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:20.620+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:20.662 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV=/dev/nbd0 2024-09-11T06:50:20.662 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' /dev/nbd0 = /dev/nbd0 ']' 2024-09-11T06:50:20.663 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:50:20.663 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep testrbdnbd21607 2024-09-11T06:50:20.696 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:20.697 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:20.711 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:50:20.721 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:20.719+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:20.735 INFO:tasks.workunit.client.0.smithi176.stdout:22008 rbd testrbdnbd21607 - /dev/nbd0 58b57f4c-b3da-4b34-bab6-d60441aaa504 2024-09-11T06:50:20.736 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd 2024-09-11T06:50:20.736 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:50:20.736 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:50:20.736 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:50:20.736 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd0'\'']/id' 2024-09-11T06:50:20.746 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:50:20.747 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:50:20.763 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:50:20.763 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:50:20.779 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:50:20.779 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:50:20.798 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:50:20.798 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:50:20.805 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=22008 2024-09-11T06:50:20.805 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 22008 2024-09-11T06:50:20.805 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 22008 -C rbd-nbd 2024-09-11T06:50:20.811 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:50:20.811 INFO:tasks.workunit.client.0.smithi176.stdout: 22008 ? 00:00:00 rbd-nbd 2024-09-11T06:50:20.812 INFO:tasks.workunit.client.0.smithi176.stderr:++ dd if=/tmp/tmp.IGWYJYR7sz/data bs=1M 2024-09-11T06:50:20.812 INFO:tasks.workunit.client.0.smithi176.stderr:++ md5sum 2024-09-11T06:50:20.816 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:50:20.817 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:50:20.825 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:20.824+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:20.836 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:50:20.837 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:50:20.877 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:20.877 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:20.877 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:50:20.877 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:50:20.877 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:20.891 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:50:20.934 INFO:tasks.workunit.client.0.smithi176.stderr:64+0 records in 2024-09-11T06:50:20.934 INFO:tasks.workunit.client.0.smithi176.stderr:64+0 records out 2024-09-11T06:50:20.934 INFO:tasks.workunit.client.0.smithi176.stderr:67108864 bytes (67 MB, 64 MiB) copied, 0.121593 s, 552 MB/s 2024-09-11T06:50:20.935 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo dd if=/dev/nbd0 bs=1M 2024-09-11T06:50:20.935 INFO:tasks.workunit.client.0.smithi176.stderr:++ md5sum 2024-09-11T06:50:20.935 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:50:20.936 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:50:20.936 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:20.936+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:20.937 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:50:20.937 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which dd 2024-09-11T06:50:20.938 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/dd 2024-09-11T06:50:20.938 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:50:20.938 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/dd if=/dev/nbd0 bs=1M 2024-09-11T06:50:21.027 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:21.024+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:21.073 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:21.073 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:21.073 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:21.073 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:50:21.073 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.073 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4558.0:1 3.3 3:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-11T06:50:21.073 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:50:21.073 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 3.9835059799999999, 2024-09-11T06:50:21.073 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0085520600000000002, 2024-09-11T06:50:21.073 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:21.074 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:21.074 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:21.074 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:21.074 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4558", 2024-09-11T06:50:21.074 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1478388898", 2024-09-11T06:50:21.074 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:50:21.074 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.074 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:21.074 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.074 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:21.074 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:50:21.074 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.074 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.075 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.075 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:21.075 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:50:21.075 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.075 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.075 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.075 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:21.075 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090533+0000", 2024-09-11T06:50:21.075 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2629999999999999e-06 2024-09-11T06:50:21.075 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.075 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.075 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:21.075 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090546+0000", 2024-09-11T06:50:21.076 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2918999999999999e-05 2024-09-11T06:50:21.076 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.076 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.076 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:21.076 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090572+0000", 2024-09-11T06:50:21.076 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6412000000000001e-05 2024-09-11T06:50:21.076 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.076 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.076 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:21.076 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090594+0000", 2024-09-11T06:50:21.076 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2476e-05 2024-09-11T06:50:21.076 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.076 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.076 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:21.077 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090938+0000", 2024-09-11T06:50:21.077 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034325099999999998 2024-09-11T06:50:21.077 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.077 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.077 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:21.077 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.091149+0000", 2024-09-11T06:50:21.077 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021151300000000001 2024-09-11T06:50:21.077 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.077 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.077 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:21.077 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.091377+0000", 2024-09-11T06:50:21.077 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022785399999999999 2024-09-11T06:50:21.078 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.078 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.078 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:21.078 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.092912+0000", 2024-09-11T06:50:21.078 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0015346089999999999 2024-09-11T06:50:21.078 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.078 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.078 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:21.078 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098919+0000", 2024-09-11T06:50:21.078 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0060076799999999996 2024-09-11T06:50:21.078 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.078 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.078 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:21.078 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.099009+0000", 2024-09-11T06:50:21.079 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0056000000000006e-05 2024-09-11T06:50:21.079 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.079 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.079 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:21.079 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.099081+0000", 2024-09-11T06:50:21.079 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2026999999999993e-05 2024-09-11T06:50:21.079 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.079 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:21.079 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.079 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.079 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.079 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop_reply(client.4558.0:1 3.3 e56/55 ondisk, result = 0)", 2024-09-11T06:50:21.079 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:17.098705+0000", 2024-09-11T06:50:21.079 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 3.975330826, 2024-09-11T06:50:21.080 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00039016499999999999, 2024-09-11T06:50:21.080 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:21.080 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:21.080 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:21.080 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:21.080 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.080 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:21.080 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098705+0000", 2024-09-11T06:50:21.080 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.080 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.080 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.080 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:21.080 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098705+0000", 2024-09-11T06:50:21.081 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.081 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.081 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.081 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:21.081 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098706+0000", 2024-09-11T06:50:21.081 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.626e-06 2024-09-11T06:50:21.081 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.081 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.081 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:21.082 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098717+0000", 2024-09-11T06:50:21.082 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0699e-05 2024-09-11T06:50:21.082 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.082 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.082 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:21.082 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098743+0000", 2024-09-11T06:50:21.082 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6449e-05 2024-09-11T06:50:21.082 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.082 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.082 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:21.082 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098759+0000", 2024-09-11T06:50:21.082 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5815999999999999e-05 2024-09-11T06:50:21.082 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.082 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.083 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:21.083 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098894+0000", 2024-09-11T06:50:21.083 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013442099999999999 2024-09-11T06:50:21.083 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.083 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.083 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:21.084 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098910+0000", 2024-09-11T06:50:21.084 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6696000000000001e-05 2024-09-11T06:50:21.084 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.084 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.084 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:21.084 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.099095+0000", 2024-09-11T06:50:21.084 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000184458 2024-09-11T06:50:21.084 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.084 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:21.084 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.084 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.084 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:21.084 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:21.088 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:50:21.128 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:21.123+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:21.151 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T06:50:21.151 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T06:50:21.151 INFO:tasks.thrashosds.thrasher:Removing osd 3, in_osds are: [0, 2, 3, 4, 5, 1] 2024-09-11T06:50:21.152 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 3 2024-09-11T06:50:21.229 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:21.228+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:21.253 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:21.253 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:21.253 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:50:21.254 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:21.268 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:50:21.356 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:21.356+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:21.434 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:21.431+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:21.441 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:21.441 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:21.441 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:50:21.441 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:50:21.441 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:21.456 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:50:21.472 INFO:tasks.ceph.mon.a.smithi071.stderr:2024-09-11T06:50:21.467+0000 7f1296554640 -1 mon.a@0(leader).osd e59 definitely_dead 0 2024-09-11T06:50:21.536 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:21.531+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:21.622 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:21.622 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:21.622 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:21.622 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:50:21.622 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.622 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:50:21.622 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:50:21.622 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 70.389027335999998, 2024-09-11T06:50:21.622 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010553425999999999, 2024-09-11T06:50:21.622 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:21.622 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:21.623 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:21.623 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:21.623 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:21.623 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:21.623 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:50:21.623 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.623 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:21.623 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.623 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:21.623 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:50:21.623 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.623 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.623 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.624 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:21.624 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:50:21.624 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.624 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.624 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.624 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:21.624 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230274+0000", 2024-09-11T06:50:21.624 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.533e-06 2024-09-11T06:50:21.624 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.624 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.624 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:21.624 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230285+0000", 2024-09-11T06:50:21.624 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0540000000000001e-05 2024-09-11T06:50:21.624 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.625 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:21.625 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230309+0000", 2024-09-11T06:50:21.625 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4223000000000002e-05 2024-09-11T06:50:21.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.625 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:21.625 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230327+0000", 2024-09-11T06:50:21.625 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:50:21.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.625 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:21.625 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230478+0000", 2024-09-11T06:50:21.626 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000151459 2024-09-11T06:50:21.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.626 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.626 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:21.626 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230490+0000", 2024-09-11T06:50:21.626 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.147e-05 2024-09-11T06:50:21.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.626 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.626 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:21.626 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231803+0000", 2024-09-11T06:50:21.626 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001313158 2024-09-11T06:50:21.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.626 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.626 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:50:21.626 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231817+0000", 2024-09-11T06:50:21.626 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4197e-05 2024-09-11T06:50:21.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.627 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:21.627 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.235945+0000", 2024-09-11T06:50:21.627 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0041282480000000002 2024-09-11T06:50:21.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.627 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:21.627 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236312+0000", 2024-09-11T06:50:21.627 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036652700000000001 2024-09-11T06:50:21.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.628 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:50:21.628 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236689+0000", 2024-09-11T06:50:21.628 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000377117 2024-09-11T06:50:21.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.628 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.628 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:21.628 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238460+0000", 2024-09-11T06:50:21.628 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001770918 2024-09-11T06:50:21.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.628 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.628 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:21.628 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240627+0000", 2024-09-11T06:50:21.628 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021670779999999998 2024-09-11T06:50:21.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.628 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.629 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:21.629 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240751+0000", 2024-09-11T06:50:21.629 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012384900000000001 2024-09-11T06:50:21.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.629 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.629 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:21.629 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240821+0000", 2024-09-11T06:50:21.629 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0516e-05 2024-09-11T06:50:21.629 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.629 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:21.629 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.629 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.629 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-11T06:50:21.630 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:50:21.630 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 69.386856492999996, 2024-09-11T06:50:21.630 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069904770000000001, 2024-09-11T06:50:21.630 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:21.630 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:21.630 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:21.630 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:21.630 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:21.630 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:21.630 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 5 2024-09-11T06:50:21.630 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.630 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:21.630 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.631 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:21.631 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:50:21.631 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.631 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:21.631 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:50:21.631 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.631 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:21.631 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232442+0000", 2024-09-11T06:50:21.631 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.044e-06 2024-09-11T06:50:21.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.632 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:21.632 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232460+0000", 2024-09-11T06:50:21.632 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7863000000000002e-05 2024-09-11T06:50:21.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.632 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:21.632 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232492+0000", 2024-09-11T06:50:21.632 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2812999999999999e-05 2024-09-11T06:50:21.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.632 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:21.632 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232510+0000", 2024-09-11T06:50:21.632 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7552999999999999e-05 2024-09-11T06:50:21.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.633 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:21.633 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232640+0000", 2024-09-11T06:50:21.633 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013029900000000001 2024-09-11T06:50:21.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.633 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:21.633 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232650+0000", 2024-09-11T06:50:21.633 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0023999999999999e-05 2024-09-11T06:50:21.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.636 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.637 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:21.637 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235641+0000", 2024-09-11T06:50:21.637 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00299089 2024-09-11T06:50:21.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.637 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:21.637 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235980+0000", 2024-09-11T06:50:21.637 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000338907 2024-09-11T06:50:21.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.637 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:21.637 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.236351+0000", 2024-09-11T06:50:21.637 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037087699999999998 2024-09-11T06:50:21.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.638 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:21.638 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238074+0000", 2024-09-11T06:50:21.638 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001722859 2024-09-11T06:50:21.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.638 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:21.638 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239160+0000", 2024-09-11T06:50:21.638 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001086074 2024-09-11T06:50:21.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.638 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:21.638 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239299+0000", 2024-09-11T06:50:21.638 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013933500000000001 2024-09-11T06:50:21.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.639 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:21.639 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239429+0000", 2024-09-11T06:50:21.639 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129939 2024-09-11T06:50:21.639 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.639 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:21.639 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.639 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-11T06:50:21.639 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:50:21.639 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 68.372768540999999, 2024-09-11T06:50:21.639 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069253250000000004, 2024-09-11T06:50:21.640 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:21.640 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:21.640 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:21.640 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:21.640 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:21.640 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:21.640 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 7 2024-09-11T06:50:21.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.640 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:21.640 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.640 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:21.640 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:50:21.640 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.641 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:21.641 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:50:21.641 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.641 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:21.641 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246530+0000", 2024-09-11T06:50:21.641 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9469999999999999e-06 2024-09-11T06:50:21.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.642 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:21.642 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246545+0000", 2024-09-11T06:50:21.642 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5973000000000001e-05 2024-09-11T06:50:21.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.642 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:21.642 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246578+0000", 2024-09-11T06:50:21.642 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2759999999999998e-05 2024-09-11T06:50:21.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.642 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:21.643 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246595+0000", 2024-09-11T06:50:21.643 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6939000000000002e-05 2024-09-11T06:50:21.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.643 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:21.643 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246770+0000", 2024-09-11T06:50:21.643 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000174723 2024-09-11T06:50:21.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.644 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:21.644 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246789+0000", 2024-09-11T06:50:21.644 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9159000000000001e-05 2024-09-11T06:50:21.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.644 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:21.644 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250337+0000", 2024-09-11T06:50:21.644 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035480759999999998 2024-09-11T06:50:21.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.644 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:21.644 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250549+0000", 2024-09-11T06:50:21.645 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021192199999999999 2024-09-11T06:50:21.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:21.645 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250750+0000", 2024-09-11T06:50:21.645 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020104199999999999 2024-09-11T06:50:21.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:21.645 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.252455+0000", 2024-09-11T06:50:21.645 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001705249 2024-09-11T06:50:21.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:21.646 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253305+0000", 2024-09-11T06:50:21.646 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00084941200000000002 2024-09-11T06:50:21.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.646 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:21.646 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253388+0000", 2024-09-11T06:50:21.646 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3642999999999999e-05 2024-09-11T06:50:21.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.646 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:21.646 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253452+0000", 2024-09-11T06:50:21.646 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3479999999999995e-05 2024-09-11T06:50:21.646 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.646 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:21.647 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.647 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(client.4558.0:5 3.2 e57/55 3:5cea7035:::rbd_info:head v 57'1, mlcod=0'0)", 2024-09-11T06:50:21.647 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:50:21.647 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 3.5246641790000002, 2024-09-11T06:50:21.647 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011016419999999999, 2024-09-11T06:50:21.647 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:21.647 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:21.647 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:21.647 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:21.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.647 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:21.648 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:50:21.648 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.648 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:21.648 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:50:21.648 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.648 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:21.648 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:50:21.648 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4000000000000002e-07 2024-09-11T06:50:21.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.649 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:21.649 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094640+0000", 2024-09-11T06:50:21.649 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6230000000000003e-06 2024-09-11T06:50:21.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.649 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:21.649 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094669+0000", 2024-09-11T06:50:21.649 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8462e-05 2024-09-11T06:50:21.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.649 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:21.649 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094684+0000", 2024-09-11T06:50:21.649 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5536000000000002e-05 2024-09-11T06:50:21.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.650 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:21.650 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094798+0000", 2024-09-11T06:50:21.650 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011439599999999999 2024-09-11T06:50:21.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.650 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:21.650 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094819+0000", 2024-09-11T06:50:21.650 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0296e-05 2024-09-11T06:50:21.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.650 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:21.650 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.095687+0000", 2024-09-11T06:50:21.650 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086851799999999998 2024-09-11T06:50:21.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.651 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:21.651 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.095733+0000", 2024-09-11T06:50:21.651 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5271000000000001e-05 2024-09-11T06:50:21.651 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.651 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:21.651 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.651 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:160 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 4096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:21.651 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:50:21.651 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.58554917200000001, 2024-09-11T06:50:21.651 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001122215, 2024-09-11T06:50:21.652 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:21.652 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:21.652 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:21.652 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:21.652 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:21.652 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:21.652 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 160 2024-09-11T06:50:21.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.652 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:21.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.652 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:21.652 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:50:21.652 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.653 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:21.653 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:50:21.653 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.653 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:21.653 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:50:21.653 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0999999999999999e-07 2024-09-11T06:50:21.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.653 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:21.653 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033749+0000", 2024-09-11T06:50:21.653 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8930000000000001e-06 2024-09-11T06:50:21.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.654 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:21.654 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033758+0000", 2024-09-11T06:50:21.654 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.242e-06 2024-09-11T06:50:21.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.654 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:21.654 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033764+0000", 2024-09-11T06:50:21.654 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0510000000000002e-06 2024-09-11T06:50:21.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.654 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:21.655 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.034207+0000", 2024-09-11T06:50:21.655 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00044320600000000003 2024-09-11T06:50:21.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.655 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:21.655 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.034280+0000", 2024-09-11T06:50:21.655 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3094000000000002e-05 2024-09-11T06:50:21.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.655 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:21.655 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.034868+0000", 2024-09-11T06:50:21.655 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00058821899999999996 2024-09-11T06:50:21.655 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.655 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:21.656 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.656 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:162 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:21.656 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.039519+0000", 2024-09-11T06:50:21.656 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.57977605399999999, 2024-09-11T06:50:21.656 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001298485, 2024-09-11T06:50:21.656 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:21.656 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:21.656 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:21.656 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:21.656 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:21.656 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:21.656 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 162 2024-09-11T06:50:21.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.657 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:21.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.657 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:21.657 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039519+0000", 2024-09-11T06:50:21.657 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.657 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:21.657 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039519+0000", 2024-09-11T06:50:21.657 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.657 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:21.658 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039520+0000", 2024-09-11T06:50:21.658 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9299999999999998e-07 2024-09-11T06:50:21.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.658 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:21.658 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039523+0000", 2024-09-11T06:50:21.658 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.247e-06 2024-09-11T06:50:21.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.658 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:21.658 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039529+0000", 2024-09-11T06:50:21.658 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8300000000000001e-06 2024-09-11T06:50:21.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.658 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:21.659 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039533+0000", 2024-09-11T06:50:21.659 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.2379999999999999e-06 2024-09-11T06:50:21.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.659 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:21.659 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.040236+0000", 2024-09-11T06:50:21.659 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00070326500000000001 2024-09-11T06:50:21.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.659 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:21.659 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.040299+0000", 2024-09-11T06:50:21.659 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3077999999999995e-05 2024-09-11T06:50:21.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.660 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:21.660 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.040818+0000", 2024-09-11T06:50:21.660 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00051833399999999996 2024-09-11T06:50:21.660 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.660 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:21.660 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.660 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:164 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 528384~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:21.660 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.041536+0000", 2024-09-11T06:50:21.660 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.57775933199999996, 2024-09-11T06:50:21.660 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00119962, 2024-09-11T06:50:21.660 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:21.661 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:21.661 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:21.661 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:21.661 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:21.661 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:21.661 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 164 2024-09-11T06:50:21.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.661 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:21.661 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.661 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:21.661 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041536+0000", 2024-09-11T06:50:21.661 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.661 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.662 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:21.662 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041536+0000", 2024-09-11T06:50:21.662 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.662 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.662 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:21.662 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041537+0000", 2024-09-11T06:50:21.662 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3e-06 2024-09-11T06:50:21.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.662 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.662 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:21.662 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041543+0000", 2024-09-11T06:50:21.662 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2360000000000001e-06 2024-09-11T06:50:21.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.663 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:21.663 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041558+0000", 2024-09-11T06:50:21.663 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4909000000000001e-05 2024-09-11T06:50:21.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.663 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:21.663 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041569+0000", 2024-09-11T06:50:21.663 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0946e-05 2024-09-11T06:50:21.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.663 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:21.663 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.042138+0000", 2024-09-11T06:50:21.663 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000568996 2024-09-11T06:50:21.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.664 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.664 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:21.664 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.042190+0000", 2024-09-11T06:50:21.664 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1609e-05 2024-09-11T06:50:21.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.664 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.664 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:21.664 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.042735+0000", 2024-09-11T06:50:21.664 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00054562399999999998 2024-09-11T06:50:21.664 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.664 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:21.664 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.664 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.665 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:166 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 790528~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:21.665 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.043564+0000", 2024-09-11T06:50:21.665 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.57573093500000005, 2024-09-11T06:50:21.665 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011042000000000001, 2024-09-11T06:50:21.665 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:21.665 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:21.665 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:21.665 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:21.665 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:21.665 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:21.665 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 166 2024-09-11T06:50:21.665 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.665 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:21.665 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.666 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:21.666 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043564+0000", 2024-09-11T06:50:21.666 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.666 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.666 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.666 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:21.666 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043564+0000", 2024-09-11T06:50:21.666 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.666 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.666 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.666 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:21.666 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043565+0000", 2024-09-11T06:50:21.666 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0449999999999999e-06 2024-09-11T06:50:21.666 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.667 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.667 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:21.667 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043571+0000", 2024-09-11T06:50:21.667 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.6849999999999998e-06 2024-09-11T06:50:21.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.667 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.667 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:21.667 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043586+0000", 2024-09-11T06:50:21.667 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4902000000000001e-05 2024-09-11T06:50:21.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.667 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.667 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:21.667 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043593+0000", 2024-09-11T06:50:21.667 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9240000000000001e-06 2024-09-11T06:50:21.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.668 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:21.668 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.044073+0000", 2024-09-11T06:50:21.668 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000479813 2024-09-11T06:50:21.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.668 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:21.668 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.044123+0000", 2024-09-11T06:50:21.668 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0908000000000001e-05 2024-09-11T06:50:21.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.668 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:21.668 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.044668+0000", 2024-09-11T06:50:21.669 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000544923 2024-09-11T06:50:21.669 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.669 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:21.669 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.669 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.669 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:178 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 2363392~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:21.669 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.061599+0000", 2024-09-11T06:50:21.669 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.55769621999999996, 2024-09-11T06:50:21.669 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012194529999999999, 2024-09-11T06:50:21.669 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:21.669 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:21.669 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:21.669 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:21.670 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:21.670 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:21.670 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 178 2024-09-11T06:50:21.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.670 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:21.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.670 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:21.670 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061599+0000", 2024-09-11T06:50:21.670 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.670 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:21.670 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061599+0000", 2024-09-11T06:50:21.670 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.671 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.671 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:21.671 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061600+0000", 2024-09-11T06:50:21.671 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3600000000000003e-07 2024-09-11T06:50:21.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.671 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.671 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:21.671 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061603+0000", 2024-09-11T06:50:21.671 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.6550000000000002e-06 2024-09-11T06:50:21.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.671 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.671 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:21.671 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061618+0000", 2024-09-11T06:50:21.672 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4377e-05 2024-09-11T06:50:21.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.672 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.672 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:21.672 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061626+0000", 2024-09-11T06:50:21.672 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.0679999999999997e-06 2024-09-11T06:50:21.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.672 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.672 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:21.672 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.062098+0000", 2024-09-11T06:50:21.672 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00047251299999999998 2024-09-11T06:50:21.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.672 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.672 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:21.673 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.062253+0000", 2024-09-11T06:50:21.673 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015533100000000001 2024-09-11T06:50:21.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.673 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:21.673 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.062818+0000", 2024-09-11T06:50:21.673 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00056487300000000002 2024-09-11T06:50:21.673 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.673 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:21.673 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.673 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:184 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 3149824~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:21.673 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:50:21.674 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.553086517, 2024-09-11T06:50:21.674 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010685969999999999, 2024-09-11T06:50:21.674 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:21.674 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:21.674 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:21.674 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:21.674 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:21.674 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:21.674 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 184 2024-09-11T06:50:21.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.674 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:21.674 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.674 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:21.674 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:50:21.674 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.675 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.675 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:21.675 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:50:21.675 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.675 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.675 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:21.675 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:50:21.675 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.6700000000000003e-07 2024-09-11T06:50:21.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.675 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.675 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:21.675 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066212+0000", 2024-09-11T06:50:21.676 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.1669999999999999e-06 2024-09-11T06:50:21.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.676 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:21.676 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066219+0000", 2024-09-11T06:50:21.676 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5699999999999998e-06 2024-09-11T06:50:21.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.676 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:21.676 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066223+0000", 2024-09-11T06:50:21.676 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3540000000000002e-06 2024-09-11T06:50:21.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.676 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:21.677 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066713+0000", 2024-09-11T06:50:21.677 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00048985399999999996 2024-09-11T06:50:21.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.677 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:21.677 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066813+0000", 2024-09-11T06:50:21.677 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00010003 2024-09-11T06:50:21.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.677 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:21.677 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.067277+0000", 2024-09-11T06:50:21.677 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000463955 2024-09-11T06:50:21.677 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.677 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:21.677 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.678 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.678 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:186 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 3411968~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:21.678 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.071198+0000", 2024-09-11T06:50:21.678 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.54809717300000005, 2024-09-11T06:50:21.678 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012515779999999999, 2024-09-11T06:50:21.678 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:21.678 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:21.678 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:21.678 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:21.678 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:21.678 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:21.678 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 186 2024-09-11T06:50:21.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.679 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:21.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.679 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:21.679 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071198+0000", 2024-09-11T06:50:21.679 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.679 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:21.679 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071198+0000", 2024-09-11T06:50:21.679 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.679 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:21.679 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071199+0000", 2024-09-11T06:50:21.679 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0299999999999999e-07 2024-09-11T06:50:21.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.680 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:21.680 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071201+0000", 2024-09-11T06:50:21.680 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8540000000000001e-06 2024-09-11T06:50:21.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.680 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:21.680 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071209+0000", 2024-09-11T06:50:21.680 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3629999999999996e-06 2024-09-11T06:50:21.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.680 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:21.680 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071214+0000", 2024-09-11T06:50:21.681 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2109999999999997e-06 2024-09-11T06:50:21.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.681 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:21.681 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071915+0000", 2024-09-11T06:50:21.681 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00070101799999999998 2024-09-11T06:50:21.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.681 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:21.681 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.072011+0000", 2024-09-11T06:50:21.681 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.5755999999999996e-05 2024-09-11T06:50:21.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.681 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:21.682 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.072450+0000", 2024-09-11T06:50:21.682 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000438773 2024-09-11T06:50:21.682 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.682 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:21.682 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.682 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:190 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 3936256~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:21.682 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.074875+0000", 2024-09-11T06:50:21.682 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.54442025800000005, 2024-09-11T06:50:21.682 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0013119620000000001, 2024-09-11T06:50:21.682 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:21.682 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:21.682 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:21.683 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:21.683 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:21.683 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:21.683 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 190 2024-09-11T06:50:21.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.683 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:21.683 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.683 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:21.683 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074875+0000", 2024-09-11T06:50:21.683 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.683 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.683 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:21.683 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074875+0000", 2024-09-11T06:50:21.684 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.684 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:21.684 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074876+0000", 2024-09-11T06:50:21.684 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7400000000000003e-07 2024-09-11T06:50:21.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.684 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:21.684 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074887+0000", 2024-09-11T06:50:21.684 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1817999999999999e-05 2024-09-11T06:50:21.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.685 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:21.685 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074896+0000", 2024-09-11T06:50:21.685 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9120000000000004e-06 2024-09-11T06:50:21.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.685 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:21.686 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074900+0000", 2024-09-11T06:50:21.686 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.2220000000000004e-06 2024-09-11T06:50:21.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.686 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:21.686 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.075198+0000", 2024-09-11T06:50:21.686 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00029750399999999998 2024-09-11T06:50:21.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.686 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:21.686 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.075430+0000", 2024-09-11T06:50:21.686 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023200400000000001 2024-09-11T06:50:21.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.687 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:21.687 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.076187+0000", 2024-09-11T06:50:21.687 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00075682799999999997 2024-09-11T06:50:21.687 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.687 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:21.687 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.687 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:359 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 135168~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:21.687 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.298832+0000", 2024-09-11T06:50:21.687 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.32046262399999997, 2024-09-11T06:50:21.687 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001078575, 2024-09-11T06:50:21.687 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:21.688 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:21.688 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:21.688 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:21.688 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:21.688 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:21.688 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 359 2024-09-11T06:50:21.688 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.688 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:21.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.688 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:21.688 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298832+0000", 2024-09-11T06:50:21.688 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.688 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.689 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:21.689 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298832+0000", 2024-09-11T06:50:21.689 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.689 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:21.689 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298834+0000", 2024-09-11T06:50:21.689 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.646e-06 2024-09-11T06:50:21.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.689 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:21.689 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298845+0000", 2024-09-11T06:50:21.689 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.081e-05 2024-09-11T06:50:21.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.690 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:21.690 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298872+0000", 2024-09-11T06:50:21.690 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7382e-05 2024-09-11T06:50:21.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.690 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:21.690 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298884+0000", 2024-09-11T06:50:21.690 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1956e-05 2024-09-11T06:50:21.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.690 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:21.690 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298959+0000", 2024-09-11T06:50:21.690 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5214000000000002e-05 2024-09-11T06:50:21.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.691 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:21.691 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.299090+0000", 2024-09-11T06:50:21.691 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000130247 2024-09-11T06:50:21.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.691 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:21.691 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.299911+0000", 2024-09-11T06:50:21.691 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00082131999999999995 2024-09-11T06:50:21.691 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.691 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:21.691 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.692 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:360 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:21.692 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.298914+0000", 2024-09-11T06:50:21.692 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.320380954, 2024-09-11T06:50:21.692 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0023571690000000001, 2024-09-11T06:50:21.692 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:21.692 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:21.692 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:21.692 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:21.692 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:21.692 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:21.692 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 360 2024-09-11T06:50:21.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.693 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:21.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.693 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:21.693 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298914+0000", 2024-09-11T06:50:21.693 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.693 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:21.693 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298914+0000", 2024-09-11T06:50:21.693 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.693 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:21.693 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298915+0000", 2024-09-11T06:50:21.694 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9300000000000006e-07 2024-09-11T06:50:21.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.694 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:21.694 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298921+0000", 2024-09-11T06:50:21.694 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.6030000000000004e-06 2024-09-11T06:50:21.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.694 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:21.694 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298932+0000", 2024-09-11T06:50:21.694 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.082e-05 2024-09-11T06:50:21.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.694 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:21.694 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298939+0000", 2024-09-11T06:50:21.695 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7919999999999993e-06 2024-09-11T06:50:21.695 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.695 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.695 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:21.695 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.300011+0000", 2024-09-11T06:50:21.695 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001071179 2024-09-11T06:50:21.695 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.695 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.695 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:21.695 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.300245+0000", 2024-09-11T06:50:21.695 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000234434 2024-09-11T06:50:21.695 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.695 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.695 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:21.695 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.301271+0000", 2024-09-11T06:50:21.696 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010263480000000001 2024-09-11T06:50:21.696 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.696 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:21.696 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.696 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.696 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:362 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 528384~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:21.696 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.302376+0000", 2024-09-11T06:50:21.696 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.31691863100000001, 2024-09-11T06:50:21.696 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014132909999999999, 2024-09-11T06:50:21.696 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:21.696 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:21.696 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:21.697 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:21.697 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:21.697 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:21.697 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 362 2024-09-11T06:50:21.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.697 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:21.697 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.697 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:21.697 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302376+0000", 2024-09-11T06:50:21.697 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.697 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.697 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:21.697 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302376+0000", 2024-09-11T06:50:21.698 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.698 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.698 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:21.698 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302378+0000", 2024-09-11T06:50:21.698 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1629999999999999e-06 2024-09-11T06:50:21.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.698 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.698 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:21.698 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302383+0000", 2024-09-11T06:50:21.698 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.463e-06 2024-09-11T06:50:21.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.698 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.698 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:21.698 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302401+0000", 2024-09-11T06:50:21.699 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8219e-05 2024-09-11T06:50:21.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.699 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:21.699 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302411+0000", 2024-09-11T06:50:21.699 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9000000000000001e-06 2024-09-11T06:50:21.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.699 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:21.699 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302524+0000", 2024-09-11T06:50:21.699 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000113199 2024-09-11T06:50:21.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.699 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:21.700 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302745+0000", 2024-09-11T06:50:21.700 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022057399999999999 2024-09-11T06:50:21.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.700 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.700 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:21.700 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.303790+0000", 2024-09-11T06:50:21.700 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010447729999999999 2024-09-11T06:50:21.700 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.700 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:21.700 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.700 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.700 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:363 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 659456~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:21.700 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.302938+0000", 2024-09-11T06:50:21.701 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.31635703799999998, 2024-09-11T06:50:21.701 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00185012, 2024-09-11T06:50:21.701 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:21.701 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:21.701 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:21.701 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:21.701 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:21.701 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:21.701 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 363 2024-09-11T06:50:21.701 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.701 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:21.701 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.701 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:21.701 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302938+0000", 2024-09-11T06:50:21.702 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.702 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.702 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.702 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:21.702 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302938+0000", 2024-09-11T06:50:21.702 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.702 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.702 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.702 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:21.702 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302939+0000", 2024-09-11T06:50:21.702 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2300000000000001e-06 2024-09-11T06:50:21.702 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.702 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.702 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:21.702 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302944+0000", 2024-09-11T06:50:21.703 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.493e-06 2024-09-11T06:50:21.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.703 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.703 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:21.703 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302960+0000", 2024-09-11T06:50:21.703 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.632e-05 2024-09-11T06:50:21.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.703 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.703 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:21.703 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302972+0000", 2024-09-11T06:50:21.703 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1885000000000001e-05 2024-09-11T06:50:21.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.703 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.703 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:21.704 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.303835+0000", 2024-09-11T06:50:21.704 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000863141 2024-09-11T06:50:21.704 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.704 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.704 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:21.704 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.303965+0000", 2024-09-11T06:50:21.704 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013032199999999999 2024-09-11T06:50:21.704 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.704 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.704 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:21.704 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.304788+0000", 2024-09-11T06:50:21.704 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00082272900000000002 2024-09-11T06:50:21.704 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.705 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:21.705 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.705 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.705 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:366 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 1052672~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:21.705 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.306960+0000", 2024-09-11T06:50:21.705 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.31233516900000002, 2024-09-11T06:50:21.705 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010286749999999999, 2024-09-11T06:50:21.705 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:21.705 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:21.705 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:21.705 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:21.705 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:21.705 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:21.705 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 366 2024-09-11T06:50:21.706 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.706 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:21.706 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.706 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:21.706 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.306960+0000", 2024-09-11T06:50:21.706 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.706 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.706 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.706 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:21.706 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.306960+0000", 2024-09-11T06:50:21.706 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.706 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.706 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.706 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:21.707 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.306961+0000", 2024-09-11T06:50:21.707 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3939999999999999e-06 2024-09-11T06:50:21.707 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.707 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.707 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:21.707 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.306970+0000", 2024-09-11T06:50:21.707 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6859999999999995e-06 2024-09-11T06:50:21.707 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.707 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.707 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:21.707 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.306991+0000", 2024-09-11T06:50:21.707 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0772e-05 2024-09-11T06:50:21.707 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.707 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.707 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:21.708 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.307002+0000", 2024-09-11T06:50:21.708 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0757000000000001e-05 2024-09-11T06:50:21.708 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.708 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.708 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:21.708 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.307077+0000", 2024-09-11T06:50:21.708 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5337000000000001e-05 2024-09-11T06:50:21.708 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.708 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.708 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:21.708 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.307185+0000", 2024-09-11T06:50:21.708 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000108335 2024-09-11T06:50:21.708 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.708 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.709 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:21.709 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.307989+0000", 2024-09-11T06:50:21.709 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00080339400000000003 2024-09-11T06:50:21.709 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.709 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:21.709 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.709 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.709 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:367 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 1183744~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:21.709 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.312392+0000", 2024-09-11T06:50:21.709 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.30690339799999999, 2024-09-11T06:50:21.709 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011343460000000001, 2024-09-11T06:50:21.709 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:21.709 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:21.710 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:21.710 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:21.710 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:21.710 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:21.710 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 367 2024-09-11T06:50:21.710 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.710 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:21.710 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.710 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:21.710 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312392+0000", 2024-09-11T06:50:21.710 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.710 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.710 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.710 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:21.711 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312392+0000", 2024-09-11T06:50:21.711 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.711 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.711 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.711 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:21.711 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312393+0000", 2024-09-11T06:50:21.711 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6199999999999999e-06 2024-09-11T06:50:21.711 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.711 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.711 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:21.711 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312400+0000", 2024-09-11T06:50:21.711 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0700000000000001e-06 2024-09-11T06:50:21.711 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.711 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.712 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:21.712 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312420+0000", 2024-09-11T06:50:21.712 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9375999999999999e-05 2024-09-11T06:50:21.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.712 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.712 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:21.712 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312430+0000", 2024-09-11T06:50:21.712 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0209e-05 2024-09-11T06:50:21.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.712 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.712 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:21.712 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312549+0000", 2024-09-11T06:50:21.712 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000118779 2024-09-11T06:50:21.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.713 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.713 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:21.713 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312767+0000", 2024-09-11T06:50:21.713 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021786400000000001 2024-09-11T06:50:21.713 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.713 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.713 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:21.713 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313526+0000", 2024-09-11T06:50:21.713 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00075942800000000003 2024-09-11T06:50:21.713 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.713 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:21.713 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.713 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.713 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.713 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:368 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 1314816~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:21.714 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.312454+0000", 2024-09-11T06:50:21.714 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.30684087399999999, 2024-09-11T06:50:21.714 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014704550000000001, 2024-09-11T06:50:21.714 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:21.714 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:21.714 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:21.714 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:21.714 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:21.714 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:21.714 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 368 2024-09-11T06:50:21.714 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.714 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:21.714 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.714 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:21.715 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312454+0000", 2024-09-11T06:50:21.715 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.715 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.715 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.715 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:21.715 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312454+0000", 2024-09-11T06:50:21.715 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.715 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.715 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.715 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:21.715 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312455+0000", 2024-09-11T06:50:21.715 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4399999999999999e-07 2024-09-11T06:50:21.715 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.715 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.716 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:21.716 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312474+0000", 2024-09-11T06:50:21.716 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8899e-05 2024-09-11T06:50:21.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.716 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.716 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:21.716 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312485+0000", 2024-09-11T06:50:21.716 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0793e-05 2024-09-11T06:50:21.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.716 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.716 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:21.716 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312492+0000", 2024-09-11T06:50:21.716 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.593e-06 2024-09-11T06:50:21.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.717 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.717 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:21.717 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313576+0000", 2024-09-11T06:50:21.717 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010837970000000001 2024-09-11T06:50:21.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.717 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.717 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:21.717 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313620+0000", 2024-09-11T06:50:21.717 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3640000000000002e-05 2024-09-11T06:50:21.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.717 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.717 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:21.717 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313925+0000", 2024-09-11T06:50:21.717 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00030498899999999998 2024-09-11T06:50:21.717 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.718 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:21.718 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.718 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.718 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.718 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:370 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 1576960~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:21.718 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.314817+0000", 2024-09-11T06:50:21.718 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.30447816500000002, 2024-09-11T06:50:21.718 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011606190000000001, 2024-09-11T06:50:21.718 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:21.718 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:21.718 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:21.718 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:21.718 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:21.718 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:21.719 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 370 2024-09-11T06:50:21.719 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.719 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:21.719 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.719 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:21.719 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314817+0000", 2024-09-11T06:50:21.719 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.719 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.719 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.719 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:21.719 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314817+0000", 2024-09-11T06:50:21.719 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:21.719 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.719 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.720 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:21.720 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314818+0000", 2024-09-11T06:50:21.720 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.463e-06 2024-09-11T06:50:21.720 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.720 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.720 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:21.720 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314826+0000", 2024-09-11T06:50:21.720 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3860000000000001e-06 2024-09-11T06:50:21.720 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.720 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.720 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:21.720 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314842+0000", 2024-09-11T06:50:21.720 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6220000000000001e-05 2024-09-11T06:50:21.720 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.720 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.721 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:21.721 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314854+0000", 2024-09-11T06:50:21.721 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1562e-05 2024-09-11T06:50:21.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.721 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.721 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:21.721 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.315212+0000", 2024-09-11T06:50:21.721 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000358012 2024-09-11T06:50:21.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.721 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.721 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:21.721 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.315433+0000", 2024-09-11T06:50:21.721 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022103800000000001 2024-09-11T06:50:21.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:21.722 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:21.722 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:21.722 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.315978+0000", 2024-09-11T06:50:21.722 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00054493800000000002 2024-09-11T06:50:21.722 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.722 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:21.722 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.722 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:21.722 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:21.722 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:21.724 INFO:tasks.workunit.client.0.smithi176.stderr:64+0 records in 2024-09-11T06:50:21.724 INFO:tasks.workunit.client.0.smithi176.stderr:64+0 records out 2024-09-11T06:50:21.724 INFO:tasks.workunit.client.0.smithi176.stderr:67108864 bytes (67 MB, 64 MiB) copied, 0.689874 s, 97.3 MB/s 2024-09-11T06:50:21.724 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' '09665990e195d409def76084dc1bac09 -' = '09665990e195d409def76084dc1bac09 -' ']' 2024-09-11T06:50:21.724 INFO:tasks.workunit.client.0.smithi176.stderr:+ dd if=/dev/urandom of=/tmp/tmp.IGWYJYR7sz/data bs=1M count=64 2024-09-11T06:50:21.724 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:50:21.725 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:21.719+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:21.825 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:21.824+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:21.855 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:21.855 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:21.855 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:50:21.855 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:21.870 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:50:21.926 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:21.924+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:21.992 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:21.992 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:21.992 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:50:21.992 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:50:21.993 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:22.006 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:50:22.053 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:22.051+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:22.113 INFO:teuthology.orchestra.run.smithi071.stderr:marked out osd.3. 2024-09-11T06:50:22.118 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T06:50:22.128 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:22.128+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:22.136 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:22.136 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:50:22.136 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:50:22.136 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:50:22.137 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.137 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:54 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'32, mlcod=13'29)", 2024-09-11T06:50:22.137 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:50:22.137 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 75.923772486000004, 2024-09-11T06:50:22.137 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034271670000000001, 2024-09-11T06:50:22.137 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.137 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.137 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:22.137 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.137 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.137 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.137 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:50:22.137 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.137 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.138 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.138 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.138 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:50:22.138 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.138 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.138 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.138 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.138 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210361+0000", 2024-09-11T06:50:22.138 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1260000000000001e-06 2024-09-11T06:50:22.138 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.138 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.138 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.138 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210370+0000", 2024-09-11T06:50:22.139 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8200000000000003e-06 2024-09-11T06:50:22.139 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.139 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.139 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.139 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210390+0000", 2024-09-11T06:50:22.139 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0126e-05 2024-09-11T06:50:22.139 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.139 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.139 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.139 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210404+0000", 2024-09-11T06:50:22.139 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4589999999999999e-05 2024-09-11T06:50:22.139 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.139 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.140 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.140 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210538+0000", 2024-09-11T06:50:22.140 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000133719 2024-09-11T06:50:22.140 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.140 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.140 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.140 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210593+0000", 2024-09-11T06:50:22.140 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5016999999999999e-05 2024-09-11T06:50:22.140 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.140 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.140 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:22.140 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213695+0000", 2024-09-11T06:50:22.140 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003101866 2024-09-11T06:50:22.141 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.141 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.141 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.141 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213787+0000", 2024-09-11T06:50:22.141 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1903000000000004e-05 2024-09-11T06:50:22.141 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.141 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.141 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.141 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.141 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.141 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:55 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'33, mlcod=13'29)", 2024-09-11T06:50:22.141 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:50:22.141 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 75.922753248000006, 2024-09-11T06:50:22.142 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.005086091, 2024-09-11T06:50:22.142 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.142 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.142 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:22.142 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.142 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.142 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.142 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:50:22.142 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.142 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.142 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.142 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.142 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:50:22.142 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.143 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.143 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.143 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.143 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211380+0000", 2024-09-11T06:50:22.143 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.097e-06 2024-09-11T06:50:22.143 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.143 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.143 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.143 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211537+0000", 2024-09-11T06:50:22.143 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000157369 2024-09-11T06:50:22.143 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.143 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.143 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.143 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211559+0000", 2024-09-11T06:50:22.144 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000001e-05 2024-09-11T06:50:22.144 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.144 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.144 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.144 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211573+0000", 2024-09-11T06:50:22.144 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4946000000000001e-05 2024-09-11T06:50:22.144 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.144 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.144 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.144 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211676+0000", 2024-09-11T06:50:22.144 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0001026 2024-09-11T06:50:22.144 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.144 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.144 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.145 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211709+0000", 2024-09-11T06:50:22.145 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3071999999999998e-05 2024-09-11T06:50:22.145 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.145 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.145 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:22.145 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216379+0000", 2024-09-11T06:50:22.145 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0046693780000000001 2024-09-11T06:50:22.145 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.145 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.145 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.145 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216465+0000", 2024-09-11T06:50:22.145 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6469000000000002e-05 2024-09-11T06:50:22.145 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.145 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.146 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.146 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.146 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.146 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:56 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'34, mlcod=13'29)", 2024-09-11T06:50:22.146 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:50:22.146 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 75.921319389000004, 2024-09-11T06:50:22.146 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0060310629999999997, 2024-09-11T06:50:22.146 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.146 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.146 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:22.146 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.146 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.146 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.146 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:50:22.146 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.147 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.147 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.147 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.147 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:50:22.147 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.147 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.147 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.147 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.147 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212814+0000", 2024-09-11T06:50:22.147 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0869999999999999e-06 2024-09-11T06:50:22.147 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.147 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.147 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.148 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212822+0000", 2024-09-11T06:50:22.148 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0870000000000007e-06 2024-09-11T06:50:22.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.148 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.148 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.148 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212840+0000", 2024-09-11T06:50:22.148 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:50:22.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.148 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.148 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.148 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212854+0000", 2024-09-11T06:50:22.148 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4003e-05 2024-09-11T06:50:22.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.148 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.148 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.149 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213026+0000", 2024-09-11T06:50:22.149 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017207499999999999 2024-09-11T06:50:22.149 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.149 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.149 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.149 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213063+0000", 2024-09-11T06:50:22.149 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7187000000000002e-05 2024-09-11T06:50:22.149 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.149 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.149 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:22.149 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218762+0000", 2024-09-11T06:50:22.149 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0056993290000000004 2024-09-11T06:50:22.150 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.150 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.150 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.150 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218844+0000", 2024-09-11T06:50:22.150 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1701999999999998e-05 2024-09-11T06:50:22.150 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.150 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.150 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.150 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.150 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.150 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:60 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'38, mlcod=13'36)", 2024-09-11T06:50:22.150 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:50:22.150 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 75.907378039999998, 2024-09-11T06:50:22.150 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039254270000000004, 2024-09-11T06:50:22.151 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.151 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.151 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:22.151 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.151 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.151 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.151 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:50:22.151 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.151 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.151 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.151 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.151 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:50:22.151 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.151 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.152 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.152 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.152 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226756+0000", 2024-09-11T06:50:22.152 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-11T06:50:22.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.152 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.152 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.152 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226768+0000", 2024-09-11T06:50:22.152 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1919999999999999e-05 2024-09-11T06:50:22.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.152 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.152 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.152 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226797+0000", 2024-09-11T06:50:22.152 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9573e-05 2024-09-11T06:50:22.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.153 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.153 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.153 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226818+0000", 2024-09-11T06:50:22.153 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0432999999999999e-05 2024-09-11T06:50:22.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.153 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.153 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.153 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226909+0000", 2024-09-11T06:50:22.153 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0920000000000004e-05 2024-09-11T06:50:22.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.153 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.153 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.153 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226936+0000", 2024-09-11T06:50:22.154 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7472999999999999e-05 2024-09-11T06:50:22.154 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.154 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.154 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:22.154 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230596+0000", 2024-09-11T06:50:22.154 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0036602520000000001 2024-09-11T06:50:22.154 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.154 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.154 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.154 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230680+0000", 2024-09-11T06:50:22.154 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3106e-05 2024-09-11T06:50:22.154 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.154 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.154 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.155 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.155 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:107 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'69, mlcod=13'64)", 2024-09-11T06:50:22.155 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:50:22.155 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 75.690930582999997, 2024-09-11T06:50:22.155 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040728830000000002, 2024-09-11T06:50:22.155 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.155 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.155 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:22.155 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.155 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.155 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.155 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:50:22.155 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.156 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.156 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.156 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.156 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:50:22.156 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.156 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.156 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.156 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.156 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443203+0000", 2024-09-11T06:50:22.156 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1480000000000001e-06 2024-09-11T06:50:22.156 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.156 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.156 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.156 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443222+0000", 2024-09-11T06:50:22.157 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9040000000000001e-05 2024-09-11T06:50:22.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.157 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.157 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.157 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443246+0000", 2024-09-11T06:50:22.157 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4606000000000001e-05 2024-09-11T06:50:22.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.157 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.157 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.157 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443277+0000", 2024-09-11T06:50:22.157 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0207999999999999e-05 2024-09-11T06:50:22.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.157 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.157 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.158 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443340+0000", 2024-09-11T06:50:22.158 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3602000000000005e-05 2024-09-11T06:50:22.158 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.158 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.158 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.158 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443363+0000", 2024-09-11T06:50:22.158 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2331e-05 2024-09-11T06:50:22.158 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.158 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.158 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:22.158 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447216+0000", 2024-09-11T06:50:22.158 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0038539690000000001 2024-09-11T06:50:22.158 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.159 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.159 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447274+0000", 2024-09-11T06:50:22.159 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7979000000000002e-05 2024-09-11T06:50:22.159 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.159 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.159 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.159 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:111 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'73, mlcod=13'67)", 2024-09-11T06:50:22.159 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:50:22.159 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 75.684418953000005, 2024-09-11T06:50:22.159 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034372360000000002, 2024-09-11T06:50:22.159 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.159 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.160 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:22.160 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.160 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.160 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.160 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:50:22.160 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.160 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.160 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.160 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:50:22.160 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.160 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.161 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.161 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449714+0000", 2024-09-11T06:50:22.161 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5000000000000001e-07 2024-09-11T06:50:22.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.161 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.161 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.161 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449721+0000", 2024-09-11T06:50:22.161 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8530000000000003e-06 2024-09-11T06:50:22.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.161 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.161 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.161 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449736+0000", 2024-09-11T06:50:22.161 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5114e-05 2024-09-11T06:50:22.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.162 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.162 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.162 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449743+0000", 2024-09-11T06:50:22.162 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0210000000000002e-06 2024-09-11T06:50:22.162 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.162 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.162 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.162 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449804+0000", 2024-09-11T06:50:22.162 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0479999999999997e-05 2024-09-11T06:50:22.162 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.162 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.162 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.162 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449826+0000", 2024-09-11T06:50:22.162 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1922000000000001e-05 2024-09-11T06:50:22.163 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.163 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.163 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:22.163 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453104+0000", 2024-09-11T06:50:22.163 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003278652 2024-09-11T06:50:22.163 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.163 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.163 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.163 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453150+0000", 2024-09-11T06:50:22.163 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6244e-05 2024-09-11T06:50:22.163 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.163 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.163 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.163 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.163 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.163 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:114 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'76, mlcod=13'71)", 2024-09-11T06:50:22.164 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:50:22.164 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 75.679740882000004, 2024-09-11T06:50:22.164 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0042576660000000002, 2024-09-11T06:50:22.164 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.164 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.164 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:22.164 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.164 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.164 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.164 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:50:22.164 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.164 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.164 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.164 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.165 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:50:22.165 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.165 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.165 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.165 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454392+0000", 2024-09-11T06:50:22.165 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5999999999999991e-07 2024-09-11T06:50:22.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.165 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.165 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.165 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454399+0000", 2024-09-11T06:50:22.165 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.878e-06 2024-09-11T06:50:22.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.165 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.166 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.166 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454415+0000", 2024-09-11T06:50:22.166 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5659999999999999e-05 2024-09-11T06:50:22.166 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.166 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.166 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.166 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454422+0000", 2024-09-11T06:50:22.166 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:50:22.166 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.166 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.166 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.166 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454474+0000", 2024-09-11T06:50:22.166 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1860999999999997e-05 2024-09-11T06:50:22.166 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.166 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.167 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.167 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454497+0000", 2024-09-11T06:50:22.167 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3686e-05 2024-09-11T06:50:22.167 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.167 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.167 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:22.167 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458552+0000", 2024-09-11T06:50:22.167 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040544550000000002 2024-09-11T06:50:22.167 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.167 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.167 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.167 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458649+0000", 2024-09-11T06:50:22.167 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7005999999999999e-05 2024-09-11T06:50:22.167 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.168 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.168 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.168 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.168 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:115 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'77, mlcod=13'72)", 2024-09-11T06:50:22.168 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:22.168 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 75.678400269999997, 2024-09-11T06:50:22.168 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003355584, 2024-09-11T06:50:22.168 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.168 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.168 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:22.168 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.168 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.168 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.169 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:22.169 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.169 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.169 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.169 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:22.169 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.169 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.169 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.169 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:22.169 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5999999999999999e-07 2024-09-11T06:50:22.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.169 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.169 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.170 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455803+0000", 2024-09-11T06:50:22.170 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0310999999999998e-05 2024-09-11T06:50:22.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.170 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.170 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.170 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455815+0000", 2024-09-11T06:50:22.170 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2787e-05 2024-09-11T06:50:22.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.170 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.170 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.170 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455823+0000", 2024-09-11T06:50:22.170 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5959999999999998e-06 2024-09-11T06:50:22.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.170 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.171 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.171 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455875+0000", 2024-09-11T06:50:22.171 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2054e-05 2024-09-11T06:50:22.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.171 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.171 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.171 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455896+0000", 2024-09-11T06:50:22.171 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0554999999999999e-05 2024-09-11T06:50:22.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.171 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.171 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:22.171 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459000+0000", 2024-09-11T06:50:22.171 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003104842 2024-09-11T06:50:22.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.172 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.172 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.172 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459087+0000", 2024-09-11T06:50:22.172 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6978999999999995e-05 2024-09-11T06:50:22.172 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.172 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.172 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.172 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.172 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.172 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:120 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'82, mlcod=13'78)", 2024-09-11T06:50:22.172 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:50:22.172 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 75.667722984999997, 2024-09-11T06:50:22.172 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0064612549999999999, 2024-09-11T06:50:22.173 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.173 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.173 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:22.173 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.173 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.173 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.173 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:50:22.173 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.173 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.173 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.173 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.173 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:50:22.173 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.174 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.174 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.174 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466410+0000", 2024-09-11T06:50:22.174 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0389999999999999e-06 2024-09-11T06:50:22.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.174 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.174 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.174 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466496+0000", 2024-09-11T06:50:22.174 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5519000000000006e-05 2024-09-11T06:50:22.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.174 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.174 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.174 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466510+0000", 2024-09-11T06:50:22.174 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3842999999999999e-05 2024-09-11T06:50:22.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.175 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.175 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.175 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466517+0000", 2024-09-11T06:50:22.175 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5229999999999998e-06 2024-09-11T06:50:22.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.175 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.175 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.175 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466595+0000", 2024-09-11T06:50:22.175 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8019e-05 2024-09-11T06:50:22.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.175 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.175 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.175 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466612+0000", 2024-09-11T06:50:22.176 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7013000000000001e-05 2024-09-11T06:50:22.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.176 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.176 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:22.176 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472787+0000", 2024-09-11T06:50:22.176 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0061750269999999996 2024-09-11T06:50:22.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.176 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.176 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.176 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472870+0000", 2024-09-11T06:50:22.176 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3271999999999998e-05 2024-09-11T06:50:22.176 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.176 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.176 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.177 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.177 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:121 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'83, mlcod=13'78)", 2024-09-11T06:50:22.177 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:50:22.177 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 75.666498188999995, 2024-09-11T06:50:22.177 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055613069999999997, 2024-09-11T06:50:22.177 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.177 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.177 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:22.177 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.177 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.177 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.177 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:50:22.177 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.178 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.178 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:50:22.178 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.178 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.178 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467635+0000", 2024-09-11T06:50:22.178 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6899999999999996e-07 2024-09-11T06:50:22.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.178 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.178 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467727+0000", 2024-09-11T06:50:22.178 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1800999999999997e-05 2024-09-11T06:50:22.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.179 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.179 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.179 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467740+0000", 2024-09-11T06:50:22.179 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3747999999999999e-05 2024-09-11T06:50:22.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.179 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.179 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.179 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467753+0000", 2024-09-11T06:50:22.179 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2807e-05 2024-09-11T06:50:22.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.179 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.179 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.179 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467812+0000", 2024-09-11T06:50:22.179 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8381999999999998e-05 2024-09-11T06:50:22.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.180 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.180 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.180 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467828+0000", 2024-09-11T06:50:22.180 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6728e-05 2024-09-11T06:50:22.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.180 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.180 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:22.180 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473112+0000", 2024-09-11T06:50:22.180 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052833619999999998 2024-09-11T06:50:22.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.180 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.180 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.181 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473195+0000", 2024-09-11T06:50:22.181 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3609999999999994e-05 2024-09-11T06:50:22.181 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.181 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.181 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.181 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.181 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.181 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:131 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'89, mlcod=13'85)", 2024-09-11T06:50:22.181 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:50:22.181 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 75.315092473999997, 2024-09-11T06:50:22.181 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052492260000000001, 2024-09-11T06:50:22.181 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.181 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.181 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:22.182 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.182 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.182 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:50:22.182 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.182 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.182 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:50:22.182 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.182 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.182 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819041+0000", 2024-09-11T06:50:22.183 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4360000000000001e-06 2024-09-11T06:50:22.183 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.183 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.183 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.183 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819100+0000", 2024-09-11T06:50:22.183 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9225999999999998e-05 2024-09-11T06:50:22.183 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.183 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.183 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.183 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819128+0000", 2024-09-11T06:50:22.183 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.779e-05 2024-09-11T06:50:22.183 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.183 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.184 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.184 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819142+0000", 2024-09-11T06:50:22.184 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3573e-05 2024-09-11T06:50:22.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.184 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.184 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.184 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819263+0000", 2024-09-11T06:50:22.184 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000121086 2024-09-11T06:50:22.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.184 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.184 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.184 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819292+0000", 2024-09-11T06:50:22.184 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9363e-05 2024-09-11T06:50:22.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.185 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.185 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:22.185 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824185+0000", 2024-09-11T06:50:22.185 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0048924620000000002 2024-09-11T06:50:22.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.185 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.185 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.185 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824289+0000", 2024-09-11T06:50:22.185 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010429000000000001 2024-09-11T06:50:22.185 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.185 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.185 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.185 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.185 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:132 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'90, mlcod=13'85)", 2024-09-11T06:50:22.186 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:50:22.186 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 75.314317783999996, 2024-09-11T06:50:22.186 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.004532342, 2024-09-11T06:50:22.186 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.186 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.186 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:22.186 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.186 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.186 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:50:22.186 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.186 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.187 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:50:22.187 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.187 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.187 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.187 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819816+0000", 2024-09-11T06:50:22.187 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.61e-06 2024-09-11T06:50:22.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.187 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.187 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.188 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819827+0000", 2024-09-11T06:50:22.188 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1e-05 2024-09-11T06:50:22.188 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.188 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.188 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.188 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819851+0000", 2024-09-11T06:50:22.188 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4267e-05 2024-09-11T06:50:22.188 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.188 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.188 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.188 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819864+0000", 2024-09-11T06:50:22.188 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2767e-05 2024-09-11T06:50:22.188 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.188 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.189 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.189 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820071+0000", 2024-09-11T06:50:22.189 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000207061 2024-09-11T06:50:22.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.189 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.189 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.189 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820103+0000", 2024-09-11T06:50:22.189 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2326999999999997e-05 2024-09-11T06:50:22.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.189 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.189 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:22.189 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824302+0000", 2024-09-11T06:50:22.189 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0041986000000000002 2024-09-11T06:50:22.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.190 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.190 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.190 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824347+0000", 2024-09-11T06:50:22.190 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4709999999999997e-05 2024-09-11T06:50:22.190 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.190 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.190 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.190 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.190 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:133 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'91, mlcod=13'86)", 2024-09-11T06:50:22.190 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:50:22.190 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 75.313686125000004, 2024-09-11T06:50:22.190 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039421459999999997, 2024-09-11T06:50:22.190 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.191 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.191 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:22.191 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.191 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.191 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:50:22.191 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.191 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.191 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:50:22.191 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.191 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.192 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820448+0000", 2024-09-11T06:50:22.192 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.84e-06 2024-09-11T06:50:22.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.192 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.192 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820463+0000", 2024-09-11T06:50:22.192 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5356e-05 2024-09-11T06:50:22.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.192 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.192 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820499+0000", 2024-09-11T06:50:22.192 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.625e-05 2024-09-11T06:50:22.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.193 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.193 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820546+0000", 2024-09-11T06:50:22.193 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7017000000000001e-05 2024-09-11T06:50:22.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.193 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.193 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.193 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820882+0000", 2024-09-11T06:50:22.193 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033512 2024-09-11T06:50:22.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.193 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.193 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.193 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820941+0000", 2024-09-11T06:50:22.193 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8922999999999998e-05 2024-09-11T06:50:22.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.194 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.194 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:22.194 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824351+0000", 2024-09-11T06:50:22.194 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034101940000000001 2024-09-11T06:50:22.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.194 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.194 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.194 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824388+0000", 2024-09-11T06:50:22.194 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7446e-05 2024-09-11T06:50:22.194 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.194 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.194 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.194 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.194 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:134 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'92, mlcod=13'86)", 2024-09-11T06:50:22.194 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:50:22.195 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 75.313027007000002, 2024-09-11T06:50:22.195 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033248179999999998, 2024-09-11T06:50:22.195 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.195 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.195 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:22.195 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.195 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.195 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.195 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:50:22.195 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.195 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.195 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.195 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.195 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:50:22.196 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.196 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.196 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.196 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821107+0000", 2024-09-11T06:50:22.196 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7430000000000001e-06 2024-09-11T06:50:22.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.196 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.196 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.196 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821205+0000", 2024-09-11T06:50:22.196 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7666999999999998e-05 2024-09-11T06:50:22.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.196 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.196 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.197 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821234+0000", 2024-09-11T06:50:22.197 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9756000000000001e-05 2024-09-11T06:50:22.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.197 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.197 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821254+0000", 2024-09-11T06:50:22.197 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9446e-05 2024-09-11T06:50:22.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.197 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.197 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821846+0000", 2024-09-11T06:50:22.197 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00059187599999999997 2024-09-11T06:50:22.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821894+0000", 2024-09-11T06:50:22.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8766e-05 2024-09-11T06:50:22.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:22.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824392+0000", 2024-09-11T06:50:22.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0024977509999999999 2024-09-11T06:50:22.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824430+0000", 2024-09-11T06:50:22.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7812999999999999e-05 2024-09-11T06:50:22.198 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.199 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.199 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.199 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:139 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'97, mlcod=13'86)", 2024-09-11T06:50:22.199 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:50:22.199 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 75.308276147000001, 2024-09-11T06:50:22.199 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003732221, 2024-09-11T06:50:22.199 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.199 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.199 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:22.199 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.199 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.199 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:50:22.200 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.200 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.200 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.200 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:50:22.200 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.200 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.200 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.200 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825858+0000", 2024-09-11T06:50:22.200 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5829999999999999e-06 2024-09-11T06:50:22.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.200 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.201 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.201 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825874+0000", 2024-09-11T06:50:22.201 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5967000000000001e-05 2024-09-11T06:50:22.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.201 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.201 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.201 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825909+0000", 2024-09-11T06:50:22.201 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5692999999999999e-05 2024-09-11T06:50:22.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.201 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.201 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.201 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825923+0000", 2024-09-11T06:50:22.201 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4080000000000001e-05 2024-09-11T06:50:22.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.202 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.202 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.202 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826010+0000", 2024-09-11T06:50:22.202 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6248999999999994e-05 2024-09-11T06:50:22.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.202 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.202 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.202 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826038+0000", 2024-09-11T06:50:22.202 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8010000000000001e-05 2024-09-11T06:50:22.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.202 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.202 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:22.202 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829499+0000", 2024-09-11T06:50:22.202 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003461783 2024-09-11T06:50:22.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.203 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.203 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829588+0000", 2024-09-11T06:50:22.203 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8856000000000004e-05 2024-09-11T06:50:22.203 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.203 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.203 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.203 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:140 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'98, mlcod=13'93)", 2024-09-11T06:50:22.203 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:50:22.203 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 75.306425418000003, 2024-09-11T06:50:22.203 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0057995929999999996, 2024-09-11T06:50:22.203 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.204 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.204 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:22.204 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.204 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.204 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.204 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:50:22.204 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.204 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.204 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.204 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:50:22.204 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.204 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.204 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.205 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827708+0000", 2024-09-11T06:50:22.205 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.547e-06 2024-09-11T06:50:22.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.205 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.205 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827765+0000", 2024-09-11T06:50:22.205 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6266e-05 2024-09-11T06:50:22.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.205 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.205 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827791+0000", 2024-09-11T06:50:22.205 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6098999999999999e-05 2024-09-11T06:50:22.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.208 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.208 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827806+0000", 2024-09-11T06:50:22.208 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5537e-05 2024-09-11T06:50:22.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.208 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.208 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.208 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827928+0000", 2024-09-11T06:50:22.208 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000122012 2024-09-11T06:50:22.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.209 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827961+0000", 2024-09-11T06:50:22.209 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2663999999999998e-05 2024-09-11T06:50:22.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:22.209 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833409+0000", 2024-09-11T06:50:22.209 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054479350000000001 2024-09-11T06:50:22.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.209 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833506+0000", 2024-09-11T06:50:22.210 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7533000000000003e-05 2024-09-11T06:50:22.210 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.210 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.210 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.210 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.210 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:141 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'99, mlcod=13'93)", 2024-09-11T06:50:22.210 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:50:22.210 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 75.305532034999999, 2024-09-11T06:50:22.210 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055394709999999998, 2024-09-11T06:50:22.210 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.210 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.210 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:22.211 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.211 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.211 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.211 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:50:22.211 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.211 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.211 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.211 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:50:22.211 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.211 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.211 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.211 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828602+0000", 2024-09-11T06:50:22.212 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.46e-06 2024-09-11T06:50:22.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.212 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.212 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828618+0000", 2024-09-11T06:50:22.212 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6592999999999999e-05 2024-09-11T06:50:22.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.212 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.212 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828646+0000", 2024-09-11T06:50:22.212 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7552999999999999e-05 2024-09-11T06:50:22.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.212 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.213 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828663+0000", 2024-09-11T06:50:22.213 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.719e-05 2024-09-11T06:50:22.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.213 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.213 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829023+0000", 2024-09-11T06:50:22.213 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036054799999999998 2024-09-11T06:50:22.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.213 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.213 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829062+0000", 2024-09-11T06:50:22.213 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8865999999999997e-05 2024-09-11T06:50:22.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.214 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:22.214 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834078+0000", 2024-09-11T06:50:22.214 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050157480000000004 2024-09-11T06:50:22.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.214 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.214 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.214 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834140+0000", 2024-09-11T06:50:22.214 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1513000000000005e-05 2024-09-11T06:50:22.214 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.214 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.214 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.214 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.214 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:144 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'102, mlcod=13'98)", 2024-09-11T06:50:22.214 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:50:22.215 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 75.296371421999993, 2024-09-11T06:50:22.215 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003210348, 2024-09-11T06:50:22.215 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.215 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.215 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:22.215 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.215 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.215 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.215 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:50:22.215 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.215 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.215 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.215 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.215 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:50:22.216 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.216 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.216 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837768+0000", 2024-09-11T06:50:22.216 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9330000000000004e-06 2024-09-11T06:50:22.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.216 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.216 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837858+0000", 2024-09-11T06:50:22.216 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0185999999999993e-05 2024-09-11T06:50:22.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.217 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.217 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837890+0000", 2024-09-11T06:50:22.217 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2437000000000001e-05 2024-09-11T06:50:22.217 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.217 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.217 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.217 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837905+0000", 2024-09-11T06:50:22.217 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:50:22.217 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.217 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.217 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.217 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838019+0000", 2024-09-11T06:50:22.217 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000114336 2024-09-11T06:50:22.217 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.218 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.218 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838052+0000", 2024-09-11T06:50:22.218 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3139000000000002e-05 2024-09-11T06:50:22.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.218 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:22.218 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840914+0000", 2024-09-11T06:50:22.218 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028612780000000001 2024-09-11T06:50:22.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.218 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.218 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840971+0000", 2024-09-11T06:50:22.218 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7402999999999999e-05 2024-09-11T06:50:22.218 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.218 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.219 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.219 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.219 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.219 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:145 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'103, mlcod=13'98)", 2024-09-11T06:50:22.219 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:50:22.219 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 75.295579537999998, 2024-09-11T06:50:22.219 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054455409999999999, 2024-09-11T06:50:22.219 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.219 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.219 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:22.219 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.219 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.219 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.219 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:50:22.220 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.220 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.220 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.220 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:50:22.220 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.220 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.220 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.220 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838554+0000", 2024-09-11T06:50:22.220 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3430000000000001e-06 2024-09-11T06:50:22.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.220 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.220 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.221 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838640+0000", 2024-09-11T06:50:22.221 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5569999999999996e-05 2024-09-11T06:50:22.221 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.221 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.221 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.221 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838660+0000", 2024-09-11T06:50:22.221 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0576000000000001e-05 2024-09-11T06:50:22.221 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.221 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.221 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.221 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838674+0000", 2024-09-11T06:50:22.221 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3529999999999999e-05 2024-09-11T06:50:22.221 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.221 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.222 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.222 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838838+0000", 2024-09-11T06:50:22.222 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016468799999999999 2024-09-11T06:50:22.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.222 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.222 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838885+0000", 2024-09-11T06:50:22.222 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6884e-05 2024-09-11T06:50:22.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.222 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:22.222 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843901+0000", 2024-09-11T06:50:22.222 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050162310000000003 2024-09-11T06:50:22.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.223 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.223 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843998+0000", 2024-09-11T06:50:22.223 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.6718999999999993e-05 2024-09-11T06:50:22.223 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.223 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.223 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.223 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.223 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4558.0:1 3.3 e56/55 3:c4c92e5a:::rbd_trash:head v 56'1, mlcod=0'0)", 2024-09-11T06:50:22.223 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:50:22.223 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.0427764760000002, 2024-09-11T06:50:22.223 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0053791810000000002, 2024-09-11T06:50:22.223 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.223 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.223 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:22.224 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.224 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.224 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.224 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:50:22.224 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.224 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.224 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.224 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:50:22.224 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.224 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.224 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.224 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091357+0000", 2024-09-11T06:50:22.225 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.54e-07 2024-09-11T06:50:22.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.225 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.225 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091376+0000", 2024-09-11T06:50:22.225 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9769e-05 2024-09-11T06:50:22.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.225 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.225 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091396+0000", 2024-09-11T06:50:22.225 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0152999999999998e-05 2024-09-11T06:50:22.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.225 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.225 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091416+0000", 2024-09-11T06:50:22.226 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0006000000000001e-05 2024-09-11T06:50:22.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.226 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.226 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091506+0000", 2024-09-11T06:50:22.226 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9325000000000003e-05 2024-09-11T06:50:22.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.226 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.226 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091523+0000", 2024-09-11T06:50:22.226 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.77e-05 2024-09-11T06:50:22.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.226 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:22.226 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.096685+0000", 2024-09-11T06:50:22.227 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0051618569999999997 2024-09-11T06:50:22.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.227 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.227 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.227 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.096735+0000", 2024-09-11T06:50:22.227 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9517e-05 2024-09-11T06:50:22.227 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.227 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.227 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.227 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.227 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.227 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:22.230 INFO:tasks.workunit.client.0.smithi176.stderr:64+0 records in 2024-09-11T06:50:22.230 INFO:tasks.workunit.client.0.smithi176.stderr:64+0 records out 2024-09-11T06:50:22.230 INFO:tasks.workunit.client.0.smithi176.stderr:67108864 bytes (67 MB, 64 MiB) copied, 0.559812 s, 120 MB/s 2024-09-11T06:50:22.230 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo dd if=/tmp/tmp.IGWYJYR7sz/data of=/dev/nbd0 bs=1M oflag=direct 2024-09-11T06:50:22.230 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:22.230 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:22.230 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:22.230 INFO:tasks.workunit.client.0.smithi176.stderr:++ which dd 2024-09-11T06:50:22.230 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/dd 2024-09-11T06:50:22.230 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:22.230 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/dd if=/tmp/tmp.IGWYJYR7sz/data of=/dev/nbd0 bs=1M oflag=direct 2024-09-11T06:50:22.231 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:50:22.231 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:22.228+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:22.330 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:22.327+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:22.389 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:22.389 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:22.389 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:50:22.389 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:22.402 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:50:22.431 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:22.428+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:22.532 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:22.527+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:22.568 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:22.569 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:22.569 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:50:22.569 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:50:22.569 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:22.582 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:50:22.669 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:22.667+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:22.734 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:22.731+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:22.750 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:22.750 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:50:22.751 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:50:22.751 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:50:22.751 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.751 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:5 2.2 e17/15 2:5cea7035:::rbd_info:head v 17'1, mlcod=0'0)", 2024-09-11T06:50:22.751 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:50:22.751 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 70.510420723999999, 2024-09-11T06:50:22.751 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0019446719999999999, 2024-09-11T06:50:22.751 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.751 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.751 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:22.751 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.751 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.751 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.751 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:50:22.752 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.752 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.752 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.752 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:50:22.752 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.752 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.752 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.752 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236085+0000", 2024-09-11T06:50:22.752 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6000000000000003e-07 2024-09-11T06:50:22.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.753 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.753 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236108+0000", 2024-09-11T06:50:22.753 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3113e-05 2024-09-11T06:50:22.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.753 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.753 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236133+0000", 2024-09-11T06:50:22.753 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5723000000000001e-05 2024-09-11T06:50:22.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.754 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.754 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.754 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236154+0000", 2024-09-11T06:50:22.754 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0333999999999999e-05 2024-09-11T06:50:22.754 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.754 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.754 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.754 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236297+0000", 2024-09-11T06:50:22.754 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014278500000000001 2024-09-11T06:50:22.754 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.754 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.754 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.754 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236383+0000", 2024-09-11T06:50:22.754 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5927999999999995e-05 2024-09-11T06:50:22.754 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.755 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:22.755 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.237950+0000", 2024-09-11T06:50:22.755 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015673690000000001 2024-09-11T06:50:22.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.755 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.755 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.238029+0000", 2024-09-11T06:50:22.755 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8659999999999996e-05 2024-09-11T06:50:22.755 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.755 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.755 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.756 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:7 2.2 e18/15 2:5cea7035:::rbd_info:head v 18'2, mlcod=0'0)", 2024-09-11T06:50:22.756 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:50:22.756 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 69.496095517000001, 2024-09-11T06:50:22.756 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001907192, 2024-09-11T06:50:22.756 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.756 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.756 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:22.756 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.756 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.756 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.756 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:50:22.756 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.757 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.757 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.757 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:50:22.757 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.757 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.757 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.757 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250410+0000", 2024-09-11T06:50:22.757 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7700000000000004e-07 2024-09-11T06:50:22.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.758 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.758 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.758 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250445+0000", 2024-09-11T06:50:22.758 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4749000000000002e-05 2024-09-11T06:50:22.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.758 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.758 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.758 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250478+0000", 2024-09-11T06:50:22.758 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3269999999999998e-05 2024-09-11T06:50:22.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.758 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.758 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.758 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250501+0000", 2024-09-11T06:50:22.759 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3632999999999999e-05 2024-09-11T06:50:22.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.759 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.759 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250633+0000", 2024-09-11T06:50:22.759 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013143200000000001 2024-09-11T06:50:22.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.759 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.759 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250700+0000", 2024-09-11T06:50:22.759 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7002999999999995e-05 2024-09-11T06:50:22.760 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.760 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.760 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:22.760 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.252172+0000", 2024-09-11T06:50:22.760 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014719550000000001 2024-09-11T06:50:22.760 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.760 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.761 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.761 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.252316+0000", 2024-09-11T06:50:22.761 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144473 2024-09-11T06:50:22.761 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.761 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.761 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.761 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.761 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4558.0:5 3.2 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-11T06:50:22.761 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:50:22.761 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.6580862869999997, 2024-09-11T06:50:22.761 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0062353959999999998, 2024-09-11T06:50:22.761 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.762 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.762 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:22.762 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:22.762 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4558", 2024-09-11T06:50:22.762 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1478388898", 2024-09-11T06:50:22.762 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 5 2024-09-11T06:50:22.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.762 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.762 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.762 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:50:22.762 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.763 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.763 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:50:22.763 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.763 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.763 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.763 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088422+0000", 2024-09-11T06:50:22.763 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7859999999999998e-06 2024-09-11T06:50:22.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.764 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.764 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088440+0000", 2024-09-11T06:50:22.764 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8185999999999998e-05 2024-09-11T06:50:22.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.764 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.764 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088479+0000", 2024-09-11T06:50:22.765 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8248999999999999e-05 2024-09-11T06:50:22.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.765 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.765 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.765 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088497+0000", 2024-09-11T06:50:22.765 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8889000000000002e-05 2024-09-11T06:50:22.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.765 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.765 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.765 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088658+0000", 2024-09-11T06:50:22.765 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000160271 2024-09-11T06:50:22.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.766 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op must wait for map", 2024-09-11T06:50:22.766 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088668+0000", 2024-09-11T06:50:22.766 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0409e-05 2024-09-11T06:50:22.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.766 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.766 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092347+0000", 2024-09-11T06:50:22.766 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00367915 2024-09-11T06:50:22.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.766 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.766 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092541+0000", 2024-09-11T06:50:22.767 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000193833 2024-09-11T06:50:22.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.767 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-11T06:50:22.767 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092732+0000", 2024-09-11T06:50:22.767 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000190772 2024-09-11T06:50:22.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.767 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:22.767 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094136+0000", 2024-09-11T06:50:22.767 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014040809999999999 2024-09-11T06:50:22.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.768 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:22.768 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094506+0000", 2024-09-11T06:50:22.768 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000369959 2024-09-11T06:50:22.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.768 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.768 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:22.768 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094590+0000", 2024-09-11T06:50:22.768 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4033e-05 2024-09-11T06:50:22.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.768 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.768 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.768 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094654+0000", 2024-09-11T06:50:22.768 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3777999999999998e-05 2024-09-11T06:50:22.769 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.769 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.769 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.769 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.769 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:259 2.6 2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head [sparse-read 4096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:22.769 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.147544+0000", 2024-09-11T06:50:22.769 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 1.598960903, 2024-09-11T06:50:22.769 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0019824170000000002, 2024-09-11T06:50:22.769 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.769 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.769 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:22.770 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:22.770 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:22.770 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:22.770 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 259 2024-09-11T06:50:22.770 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.770 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.770 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.770 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.770 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.147544+0000", 2024-09-11T06:50:22.770 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.770 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.771 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.771 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.147544+0000", 2024-09-11T06:50:22.771 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.771 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.771 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.147545+0000", 2024-09-11T06:50:22.771 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4600000000000003e-07 2024-09-11T06:50:22.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.772 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.772 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.147549+0000", 2024-09-11T06:50:22.772 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8160000000000001e-06 2024-09-11T06:50:22.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.772 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.772 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.147565+0000", 2024-09-11T06:50:22.772 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5140000000000001e-05 2024-09-11T06:50:22.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.772 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.772 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.147574+0000", 2024-09-11T06:50:22.772 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2529999999999993e-06 2024-09-11T06:50:22.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.773 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.773 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.148551+0000", 2024-09-11T06:50:22.773 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0009775579999999999 2024-09-11T06:50:22.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.773 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.773 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.148694+0000", 2024-09-11T06:50:22.774 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014299399999999999 2024-09-11T06:50:22.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.774 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.774 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.149526+0000", 2024-09-11T06:50:22.774 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00083171000000000004 2024-09-11T06:50:22.774 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.774 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.774 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.774 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:261 2.6 2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:22.775 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.154411+0000", 2024-09-11T06:50:22.775 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 1.592093489, 2024-09-11T06:50:22.775 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002245024, 2024-09-11T06:50:22.775 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.775 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.775 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:22.775 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:22.775 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:22.775 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:22.775 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 261 2024-09-11T06:50:22.775 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.775 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.775 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154411+0000", 2024-09-11T06:50:22.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154411+0000", 2024-09-11T06:50:22.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154412+0000", 2024-09-11T06:50:22.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:50:22.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.777 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.777 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154416+0000", 2024-09-11T06:50:22.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3930000000000001e-06 2024-09-11T06:50:22.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.777 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.777 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154428+0000", 2024-09-11T06:50:22.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2099999999999999e-05 2024-09-11T06:50:22.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.777 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.777 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154438+0000", 2024-09-11T06:50:22.778 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2590000000000006e-06 2024-09-11T06:50:22.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.778 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.778 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.155643+0000", 2024-09-11T06:50:22.778 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012048689999999999 2024-09-11T06:50:22.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.778 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.778 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.155838+0000", 2024-09-11T06:50:22.778 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019531299999999999 2024-09-11T06:50:22.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.778 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.778 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.156656+0000", 2024-09-11T06:50:22.779 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00081824700000000003 2024-09-11T06:50:22.779 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.779 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.779 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.779 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.779 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.779 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:269 2.6 2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head [sparse-read 1314816~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:22.779 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.166906+0000", 2024-09-11T06:50:22.779 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 1.579598461, 2024-09-11T06:50:22.779 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001538906, 2024-09-11T06:50:22.779 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.779 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.779 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:22.779 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:22.779 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:22.780 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:22.780 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 269 2024-09-11T06:50:22.780 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.780 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.780 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.780 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.780 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.166906+0000", 2024-09-11T06:50:22.780 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.780 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.780 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.780 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.780 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.166906+0000", 2024-09-11T06:50:22.780 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.166907+0000", 2024-09-11T06:50:22.781 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2729999999999999e-06 2024-09-11T06:50:22.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.166914+0000", 2024-09-11T06:50:22.781 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3230000000000003e-06 2024-09-11T06:50:22.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.782 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.782 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.166934+0000", 2024-09-11T06:50:22.782 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.018e-05 2024-09-11T06:50:22.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.782 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.782 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.166951+0000", 2024-09-11T06:50:22.782 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7501999999999999e-05 2024-09-11T06:50:22.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.782 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.783 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.167677+0000", 2024-09-11T06:50:22.783 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00072579299999999997 2024-09-11T06:50:22.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.783 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.783 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.167865+0000", 2024-09-11T06:50:22.783 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000188023 2024-09-11T06:50:22.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.783 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.783 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.168445+0000", 2024-09-11T06:50:22.784 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00057981199999999997 2024-09-11T06:50:22.784 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.784 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.784 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.784 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.784 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:277 2.6 2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head [sparse-read 2363392~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:22.784 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.179016+0000", 2024-09-11T06:50:22.784 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 1.5674884069999999, 2024-09-11T06:50:22.785 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015587540000000001, 2024-09-11T06:50:22.785 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.785 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.785 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:22.785 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:22.785 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:22.785 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:22.785 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 277 2024-09-11T06:50:22.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.785 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.785 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.785 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.785 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.179016+0000", 2024-09-11T06:50:22.786 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.786 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.786 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.179016+0000", 2024-09-11T06:50:22.786 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.786 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.786 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.179017+0000", 2024-09-11T06:50:22.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.78e-07 2024-09-11T06:50:22.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.787 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.787 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.179021+0000", 2024-09-11T06:50:22.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9469999999999996e-06 2024-09-11T06:50:22.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.787 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.787 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.179032+0000", 2024-09-11T06:50:22.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0763e-05 2024-09-11T06:50:22.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.787 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.787 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.179040+0000", 2024-09-11T06:50:22.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7360000000000001e-06 2024-09-11T06:50:22.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.788 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.788 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.179817+0000", 2024-09-11T06:50:22.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00077775700000000001 2024-09-11T06:50:22.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.788 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.788 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.179958+0000", 2024-09-11T06:50:22.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014041800000000001 2024-09-11T06:50:22.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.789 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.789 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.180575+0000", 2024-09-11T06:50:22.789 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00061725500000000004 2024-09-11T06:50:22.789 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.789 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.789 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.789 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.789 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:285 2.6 2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head [sparse-read 3411968~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:22.789 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.190856+0000", 2024-09-11T06:50:22.789 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 1.555648758, 2024-09-11T06:50:22.790 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001510352, 2024-09-11T06:50:22.790 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.790 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.790 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:22.790 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:22.790 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:22.790 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:22.790 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 285 2024-09-11T06:50:22.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.790 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.790 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.790 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.790 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.190856+0000", 2024-09-11T06:50:22.791 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.791 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.791 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.791 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.190856+0000", 2024-09-11T06:50:22.791 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.791 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.791 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.791 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.190857+0000", 2024-09-11T06:50:22.791 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1200000000000002e-07 2024-09-11T06:50:22.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.791 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.792 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.792 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.190860+0000", 2024-09-11T06:50:22.792 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6839999999999998e-06 2024-09-11T06:50:22.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.792 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.792 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.190877+0000", 2024-09-11T06:50:22.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6378000000000001e-05 2024-09-11T06:50:22.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.793 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.793 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.793 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.190885+0000", 2024-09-11T06:50:22.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.551e-06 2024-09-11T06:50:22.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.793 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.793 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.793 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.191593+0000", 2024-09-11T06:50:22.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070753199999999997 2024-09-11T06:50:22.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.794 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.794 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.191731+0000", 2024-09-11T06:50:22.794 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013835099999999999 2024-09-11T06:50:22.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.794 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.794 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.192366+0000", 2024-09-11T06:50:22.794 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00063514400000000003 2024-09-11T06:50:22.795 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.795 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.795 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.795 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:524 2.4 2:2a17619e:::rbd_data.1314f0a2bfa2.000000000000000e:head [sparse-read 135168~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:22.795 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.530346+0000", 2024-09-11T06:50:22.795 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 1.2161588969999999, 2024-09-11T06:50:22.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012158220000000001, 2024-09-11T06:50:22.795 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.795 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.795 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:22.795 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:22.796 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:22.796 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:22.796 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 524 2024-09-11T06:50:22.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.796 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530346+0000", 2024-09-11T06:50:22.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530346+0000", 2024-09-11T06:50:22.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530348+0000", 2024-09-11T06:50:22.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0669999999999999e-06 2024-09-11T06:50:22.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530358+0000", 2024-09-11T06:50:22.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0246e-05 2024-09-11T06:50:22.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530389+0000", 2024-09-11T06:50:22.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0476000000000001e-05 2024-09-11T06:50:22.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530411+0000", 2024-09-11T06:50:22.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2952e-05 2024-09-11T06:50:22.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530581+0000", 2024-09-11T06:50:22.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016948700000000001 2024-09-11T06:50:22.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.799 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530706+0000", 2024-09-11T06:50:22.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012535199999999999 2024-09-11T06:50:22.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.799 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.799 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.531562+0000", 2024-09-11T06:50:22.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00085524199999999998 2024-09-11T06:50:22.799 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.799 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.799 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.799 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:525 2.4 2:2a17619e:::rbd_data.1314f0a2bfa2.000000000000000e:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:22.800 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.530454+0000", 2024-09-11T06:50:22.800 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 1.216050804, 2024-09-11T06:50:22.800 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0019464650000000001, 2024-09-11T06:50:22.800 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.800 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.800 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:22.800 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:22.800 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:22.800 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:22.800 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 525 2024-09-11T06:50:22.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.800 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.800 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.800 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.801 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530454+0000", 2024-09-11T06:50:22.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.801 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.801 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530454+0000", 2024-09-11T06:50:22.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.801 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.801 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530455+0000", 2024-09-11T06:50:22.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4100000000000001e-06 2024-09-11T06:50:22.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530462+0000", 2024-09-11T06:50:22.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2760000000000003e-06 2024-09-11T06:50:22.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530486+0000", 2024-09-11T06:50:22.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3082999999999999e-05 2024-09-11T06:50:22.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.803 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530511+0000", 2024-09-11T06:50:22.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5065000000000001e-05 2024-09-11T06:50:22.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.803 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.803 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.531637+0000", 2024-09-11T06:50:22.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011263530000000001 2024-09-11T06:50:22.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.803 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.803 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.531716+0000", 2024-09-11T06:50:22.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8565000000000004e-05 2024-09-11T06:50:22.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.804 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.804 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.532400+0000", 2024-09-11T06:50:22.804 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00068471299999999999 2024-09-11T06:50:22.804 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.804 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.804 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.804 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:527 2.4 2:2a17619e:::rbd_data.1314f0a2bfa2.000000000000000e:head [sparse-read 528384~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:22.804 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.533265+0000", 2024-09-11T06:50:22.804 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 1.2132398209999999, 2024-09-11T06:50:22.804 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001244071, 2024-09-11T06:50:22.805 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.805 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.805 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:22.805 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:22.805 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:22.805 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:22.805 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 527 2024-09-11T06:50:22.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.805 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.805 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.533265+0000", 2024-09-11T06:50:22.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.533265+0000", 2024-09-11T06:50:22.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.533266+0000", 2024-09-11T06:50:22.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5999999999999999e-06 2024-09-11T06:50:22.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.807 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.807 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.533274+0000", 2024-09-11T06:50:22.807 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4930000000000001e-06 2024-09-11T06:50:22.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.807 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.807 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.533294+0000", 2024-09-11T06:50:22.807 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0506e-05 2024-09-11T06:50:22.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.807 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.807 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.533312+0000", 2024-09-11T06:50:22.807 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7413000000000001e-05 2024-09-11T06:50:22.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.808 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.809 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.809 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.533774+0000", 2024-09-11T06:50:22.809 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00046220700000000001 2024-09-11T06:50:22.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.809 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.533855+0000", 2024-09-11T06:50:22.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1203000000000002e-05 2024-09-11T06:50:22.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.534509+0000", 2024-09-11T06:50:22.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00065364899999999996 2024-09-11T06:50:22.811 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.811 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.811 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.811 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:530 2.4 2:2a17619e:::rbd_data.1314f0a2bfa2.000000000000000e:head [sparse-read 921600~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:22.812 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.536052+0000", 2024-09-11T06:50:22.812 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 1.2104525530000001, 2024-09-11T06:50:22.812 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012798600000000001, 2024-09-11T06:50:22.812 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.812 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.813 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:22.813 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:22.813 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:22.813 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:22.813 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 530 2024-09-11T06:50:22.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.814 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.814 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.814 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.536052+0000", 2024-09-11T06:50:22.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.536052+0000", 2024-09-11T06:50:22.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.816 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.816 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.536054+0000", 2024-09-11T06:50:22.816 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6640000000000001e-06 2024-09-11T06:50:22.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.817 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.817 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.817 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.536062+0000", 2024-09-11T06:50:22.817 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3760000000000008e-06 2024-09-11T06:50:22.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.817 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.818 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.818 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.536084+0000", 2024-09-11T06:50:22.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1988999999999999e-05 2024-09-11T06:50:22.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.818 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.536102+0000", 2024-09-11T06:50:22.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8142999999999999e-05 2024-09-11T06:50:22.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.536435+0000", 2024-09-11T06:50:22.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000332548 2024-09-11T06:50:22.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.536611+0000", 2024-09-11T06:50:22.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017594299999999999 2024-09-11T06:50:22.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.821 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.821 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.537332+0000", 2024-09-11T06:50:22.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00072119700000000003 2024-09-11T06:50:22.821 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.821 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.821 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.822 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.822 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:533 2.4 2:2a17619e:::rbd_data.1314f0a2bfa2.000000000000000e:head [sparse-read 1314816~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:22.822 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.542984+0000", 2024-09-11T06:50:22.822 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 1.2035204610000001, 2024-09-11T06:50:22.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001565873, 2024-09-11T06:50:22.823 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.823 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.823 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:22.823 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:22.823 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:22.823 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:22.823 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 533 2024-09-11T06:50:22.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.824 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.824 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.824 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.542984+0000", 2024-09-11T06:50:22.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.824 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.824 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.542984+0000", 2024-09-11T06:50:22.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.824 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.542985+0000", 2024-09-11T06:50:22.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.217e-06 2024-09-11T06:50:22.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.542993+0000", 2024-09-11T06:50:22.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2799999999999998e-06 2024-09-11T06:50:22.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.543007+0000", 2024-09-11T06:50:22.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4046000000000001e-05 2024-09-11T06:50:22.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.826 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.826 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.543022+0000", 2024-09-11T06:50:22.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5206e-05 2024-09-11T06:50:22.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.826 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.826 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.543871+0000", 2024-09-11T06:50:22.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000849459 2024-09-11T06:50:22.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.826 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.826 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.544004+0000", 2024-09-11T06:50:22.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000132958 2024-09-11T06:50:22.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.827 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.827 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.827 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.544550+0000", 2024-09-11T06:50:22.827 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00054570699999999998 2024-09-11T06:50:22.827 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.827 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.827 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.827 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.827 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.827 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:535 2.4 2:2a17619e:::rbd_data.1314f0a2bfa2.000000000000000e:head [sparse-read 1576960~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:22.827 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.545515+0000", 2024-09-11T06:50:22.827 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 1.2009892870000001, 2024-09-11T06:50:22.827 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001475706, 2024-09-11T06:50:22.827 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.828 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.828 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:22.828 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:22.828 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:22.828 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:22.828 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 535 2024-09-11T06:50:22.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.828 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.545515+0000", 2024-09-11T06:50:22.828 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.545515+0000", 2024-09-11T06:50:22.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.545517+0000", 2024-09-11T06:50:22.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.536e-06 2024-09-11T06:50:22.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.545525+0000", 2024-09-11T06:50:22.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6769999999999995e-06 2024-09-11T06:50:22.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.830 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.830 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.545546+0000", 2024-09-11T06:50:22.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1676e-05 2024-09-11T06:50:22.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.830 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.830 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.545564+0000", 2024-09-11T06:50:22.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7915e-05 2024-09-11T06:50:22.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.830 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.830 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.546295+0000", 2024-09-11T06:50:22.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00073042099999999998 2024-09-11T06:50:22.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.546410+0000", 2024-09-11T06:50:22.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011579 2024-09-11T06:50:22.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.546991+0000", 2024-09-11T06:50:22.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00058069100000000004 2024-09-11T06:50:22.831 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.831 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.831 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.832 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:541 2.4 2:2a17619e:::rbd_data.1314f0a2bfa2.000000000000000e:head [sparse-read 2363392~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:22.832 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.555808+0000", 2024-09-11T06:50:22.832 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 1.190696148, 2024-09-11T06:50:22.832 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015018779999999999, 2024-09-11T06:50:22.832 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.832 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.832 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:22.832 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:22.832 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:22.832 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:22.832 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 541 2024-09-11T06:50:22.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.833 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.833 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.555808+0000", 2024-09-11T06:50:22.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.833 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.555808+0000", 2024-09-11T06:50:22.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.833 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.834 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.555810+0000", 2024-09-11T06:50:22.834 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8199999999999999e-06 2024-09-11T06:50:22.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.834 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.834 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.555818+0000", 2024-09-11T06:50:22.834 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9759999999999995e-06 2024-09-11T06:50:22.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.834 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.834 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.555842+0000", 2024-09-11T06:50:22.834 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3428999999999999e-05 2024-09-11T06:50:22.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.555858+0000", 2024-09-11T06:50:22.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5826000000000001e-05 2024-09-11T06:50:22.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.556619+0000", 2024-09-11T06:50:22.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00076185000000000001 2024-09-11T06:50:22.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.556732+0000", 2024-09-11T06:50:22.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000112705 2024-09-11T06:50:22.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.836 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.557310+0000", 2024-09-11T06:50:22.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00057827200000000001 2024-09-11T06:50:22.836 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.836 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.836 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.836 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:549 2.4 2:2a17619e:::rbd_data.1314f0a2bfa2.000000000000000e:head [sparse-read 3411968~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:22.836 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.567982+0000", 2024-09-11T06:50:22.836 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 1.178522158, 2024-09-11T06:50:22.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002000216, 2024-09-11T06:50:22.837 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.837 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.837 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:22.837 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:22.837 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:22.837 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:22.837 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 549 2024-09-11T06:50:22.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.837 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.837 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.837 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.837 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.567982+0000", 2024-09-11T06:50:22.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.567982+0000", 2024-09-11T06:50:22.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.567984+0000", 2024-09-11T06:50:22.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7290000000000001e-06 2024-09-11T06:50:22.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.839 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.839 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.567994+0000", 2024-09-11T06:50:22.839 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4229999999999994e-06 2024-09-11T06:50:22.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.839 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.839 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.568015+0000", 2024-09-11T06:50:22.840 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1596000000000001e-05 2024-09-11T06:50:22.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.840 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.840 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.568032+0000", 2024-09-11T06:50:22.840 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7067000000000001e-05 2024-09-11T06:50:22.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.568713+0000", 2024-09-11T06:50:22.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00068109799999999999 2024-09-11T06:50:22.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.568866+0000", 2024-09-11T06:50:22.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015232399999999999 2024-09-11T06:50:22.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.842 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.842 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.569983+0000", 2024-09-11T06:50:22.842 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011169789999999999 2024-09-11T06:50:22.842 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.842 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.842 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.843 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.843 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:550 2.4 2:2a17619e:::rbd_data.1314f0a2bfa2.000000000000000e:head [sparse-read 3543040~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:22.843 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.569595+0000", 2024-09-11T06:50:22.843 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 1.1769091309999999, 2024-09-11T06:50:22.843 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001587565, 2024-09-11T06:50:22.843 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.843 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.843 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:22.843 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:22.844 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:22.844 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:22.844 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 550 2024-09-11T06:50:22.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.844 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.845 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.569595+0000", 2024-09-11T06:50:22.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.845 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.569595+0000", 2024-09-11T06:50:22.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.569597+0000", 2024-09-11T06:50:22.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6130000000000001e-06 2024-09-11T06:50:22.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.847 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.569604+0000", 2024-09-11T06:50:22.847 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2429999999999999e-06 2024-09-11T06:50:22.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.847 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.847 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.569626+0000", 2024-09-11T06:50:22.847 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1478999999999999e-05 2024-09-11T06:50:22.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.848 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.848 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.569643+0000", 2024-09-11T06:50:22.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6973000000000002e-05 2024-09-11T06:50:22.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.848 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.570088+0000", 2024-09-11T06:50:22.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00044539999999999998 2024-09-11T06:50:22.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.849 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.570227+0000", 2024-09-11T06:50:22.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013836499999999999 2024-09-11T06:50:22.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.850 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.850 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.571183+0000", 2024-09-11T06:50:22.850 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00095649199999999998 2024-09-11T06:50:22.850 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.850 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.850 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.851 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:551 2.4 2:2a17619e:::rbd_data.1314f0a2bfa2.000000000000000e:head [sparse-read 3674112~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:22.851 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.570987+0000", 2024-09-11T06:50:22.851 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 1.1755177219999999, 2024-09-11T06:50:22.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001264453, 2024-09-11T06:50:22.851 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.851 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.851 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:22.851 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:22.852 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:22.852 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:22.852 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 551 2024-09-11T06:50:22.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.852 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.852 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.852 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.852 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.570987+0000", 2024-09-11T06:50:22.852 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.853 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.853 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.853 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.570987+0000", 2024-09-11T06:50:22.853 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.853 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.853 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.853 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.570989+0000", 2024-09-11T06:50:22.853 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.356e-06 2024-09-11T06:50:22.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.853 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.853 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.854 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.570998+0000", 2024-09-11T06:50:22.854 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3270000000000001e-06 2024-09-11T06:50:22.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.854 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.854 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.571035+0000", 2024-09-11T06:50:22.854 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7008000000000002e-05 2024-09-11T06:50:22.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.854 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.854 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.571052+0000", 2024-09-11T06:50:22.854 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7323e-05 2024-09-11T06:50:22.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.855 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.855 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.571222+0000", 2024-09-11T06:50:22.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016963699999999999 2024-09-11T06:50:22.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.855 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.855 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.571333+0000", 2024-09-11T06:50:22.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000111859 2024-09-11T06:50:22.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.855 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.855 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.572251+0000", 2024-09-11T06:50:22.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00091794299999999995 2024-09-11T06:50:22.855 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.856 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.856 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.856 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:554 2.4 2:2a17619e:::rbd_data.1314f0a2bfa2.000000000000000e:head [sparse-read 4067328~126976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:22.856 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.574581+0000", 2024-09-11T06:50:22.856 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 1.171924038, 2024-09-11T06:50:22.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001401228, 2024-09-11T06:50:22.856 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.856 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.856 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:22.856 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:22.857 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:22.857 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:22.857 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 554 2024-09-11T06:50:22.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.857 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.857 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.857 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.574581+0000", 2024-09-11T06:50:22.857 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.860 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.860 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.574581+0000", 2024-09-11T06:50:22.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.861 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.861 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.574582+0000", 2024-09-11T06:50:22.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.46e-06 2024-09-11T06:50:22.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.861 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.861 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.574588+0000", 2024-09-11T06:50:22.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.443e-06 2024-09-11T06:50:22.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.862 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.862 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.574609+0000", 2024-09-11T06:50:22.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0965999999999999e-05 2024-09-11T06:50:22.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.862 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.862 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.574626+0000", 2024-09-11T06:50:22.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6699000000000001e-05 2024-09-11T06:50:22.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.862 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.862 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.574786+0000", 2024-09-11T06:50:22.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016030100000000001 2024-09-11T06:50:22.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.575017+0000", 2024-09-11T06:50:22.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023037900000000001 2024-09-11T06:50:22.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.575982+0000", 2024-09-11T06:50:22.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00096498000000000005 2024-09-11T06:50:22.863 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.863 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.863 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.863 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:587 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-11T06:50:22.864 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:50:22.864 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.51725363199999996, 2024-09-11T06:50:22.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0023436289999999999, 2024-09-11T06:50:22.864 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:22.864 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:22.864 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:22.864 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:22.864 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:22.864 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:22.864 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 587 2024-09-11T06:50:22.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.864 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:22.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:22.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:50:22.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:22.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:50:22.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:22.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:22.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229252+0000", 2024-09-11T06:50:22.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3799999999999999e-06 2024-09-11T06:50:22.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.866 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:22.866 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229259+0000", 2024-09-11T06:50:22.866 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4760000000000003e-06 2024-09-11T06:50:22.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.866 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:22.866 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229271+0000", 2024-09-11T06:50:22.866 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2313000000000001e-05 2024-09-11T06:50:22.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.866 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:22.866 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229279+0000", 2024-09-11T06:50:22.866 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1610000000000003e-06 2024-09-11T06:50:22.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.867 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:22.867 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229344+0000", 2024-09-11T06:50:22.867 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4375000000000006e-05 2024-09-11T06:50:22.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.867 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:22.867 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229404+0000", 2024-09-11T06:50:22.867 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0517999999999999e-05 2024-09-11T06:50:22.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.868 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:22.868 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229516+0000", 2024-09-11T06:50:22.868 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011129900000000001 2024-09-11T06:50:22.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.868 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:22.868 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231502+0000", 2024-09-11T06:50:22.868 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001986238 2024-09-11T06:50:22.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.868 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:22.868 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231537+0000", 2024-09-11T06:50:22.869 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4968000000000001e-05 2024-09-11T06:50:22.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.869 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:22.869 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231569+0000", 2024-09-11T06:50:22.869 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2459000000000002e-05 2024-09-11T06:50:22.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:22.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:22.869 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:22.869 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231595+0000", 2024-09-11T06:50:22.869 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5442000000000001e-05 2024-09-11T06:50:22.869 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.869 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.869 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.869 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:22.870 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:22.870 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:22.870 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:50:22.871 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:22.872+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:22.872 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:50:22.973 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:22.972+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:23.049 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:23.049 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:23.049 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:50:23.050 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:23.068 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:50:23.071 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:50:23.072 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:50:23.072 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:23.071+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:23.090 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:50:23.090 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:50:23.108 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:50:23.109 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:50:23.114 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is set 2024-09-11T06:50:23.137 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:50:23.138 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:50:23.162 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:50:23.162 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:50:23.174 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:23.172+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:23.193 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:50:23.193 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:50:23.215 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:23.215 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:23.235 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:50:23.275 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:23.272+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:23.375 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:23.371+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:23.393 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:23.393 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:23.407 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:50:23.477 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:23.476+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:23.541 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:23.541 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:23.557 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:50:23.578 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:23.575+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:23.682 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:23.679+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:23.730 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:23.730 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:23.746 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:50:23.783 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:23.779+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:23.865 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:23.865 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:23.879 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:50:23.924 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:23.924+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:23.984 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:23.979+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:23.996 INFO:tasks.workunit.client.0.smithi176.stderr:64+0 records in 2024-09-11T06:50:23.996 INFO:tasks.workunit.client.0.smithi176.stderr:64+0 records out 2024-09-11T06:50:23.996 INFO:tasks.workunit.client.0.smithi176.stderr:67108864 bytes (67 MB, 64 MiB) copied, 1.77078 s, 37.9 MB/s 2024-09-11T06:50:23.998 INFO:tasks.workunit.client.0.smithi176.stderr:++ dd if=/tmp/tmp.IGWYJYR7sz/data bs=1M 2024-09-11T06:50:23.999 INFO:tasks.workunit.client.0.smithi176.stderr:++ md5sum 2024-09-11T06:50:24.002 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:24.003 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:24.016 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:50:24.108 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:24.108+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:24.140 INFO:tasks.workunit.client.0.smithi176.stderr:64+0 records in 2024-09-11T06:50:24.141 INFO:tasks.workunit.client.0.smithi176.stderr:64+0 records out 2024-09-11T06:50:24.141 INFO:tasks.workunit.client.0.smithi176.stderr:67108864 bytes (67 MB, 64 MiB) copied, 0.140326 s, 478 MB/s 2024-09-11T06:50:24.142 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd -p rbd --no-progress export testrbdnbd21607 - 2024-09-11T06:50:24.142 INFO:tasks.workunit.client.0.smithi176.stderr:++ md5sum 2024-09-11T06:50:24.185 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:24.185 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:24.186 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:24.183+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:24.199 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:50:24.237 INFO:tasks.workunit.client.0.smithi176.stderr:2024-09-11T06:50:24.231+0000 7f60ed03e640 0 -- 172.21.15.176:0/1249326177 >> [v2:172.21.15.71:6800/1158147777,v1:172.21.15.71:6801/1158147777] conn(0x564195f2ec80 msgr2=0x564195f31070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-11T06:50:24.248 INFO:tasks.workunit.client.0.smithi176.stderr:2024-09-11T06:50:24.243+0000 7f60ebddc640 0 -- 172.21.15.176:0/1249326177 >> [v2:172.21.15.71:6808/620364628,v1:172.21.15.71:6809/620364628] conn(0x7f60c40099f0 msgr2=0x7f60c400be80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-11T06:50:24.288 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:24.283+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:24.374 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:24.374 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:24.389 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:50:24.390 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:24.388+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:24.395 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 'a2eae9e74aaf7afa3e00abe5fbd7e358 -' = 'a2eae9e74aaf7afa3e00abe5fbd7e358 -' ']' 2024-09-11T06:50:24.395 INFO:tasks.workunit.client.0.smithi176.stderr:+ unmap_device /dev/nbd0 22008 2024-09-11T06:50:24.395 INFO:tasks.workunit.client.0.smithi176.stderr:+ local args=/dev/nbd0 2024-09-11T06:50:24.395 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pid=22008 2024-09-11T06:50:24.395 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd0 2024-09-11T06:50:24.395 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:24.395 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:24.396 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:24.397 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:50:24.399 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:50:24.399 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:24.399 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd0 2024-09-11T06:50:24.489 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:24.488+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:24.492 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:50:24.492 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^22008\b' 2024-09-11T06:50:24.492 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^22008\b' 2024-09-11T06:50:24.546 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:24.547 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -C rbd-nbd 2024-09-11T06:50:24.547 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^ *22008\b' 2024-09-11T06:50:24.547 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^ *22008\b' 2024-09-11T06:50:24.553 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:24.554 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 0.5 2024-09-11T06:50:24.556 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:24.556 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:24.570 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:50:24.612 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:24.612+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:24.691 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:24.687+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:24.736 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:24.737 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:24.750 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:50:24.795 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:24.796+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:24.868 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:24.868 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:24.882 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:50:24.929 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:24.927+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:24.993 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:24.991+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:25.015 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:25.015 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:25.029 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:50:25.056 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo rbd device --device-type nbd --options notrim map rbd/testrbdnbd21607 2024-09-11T06:50:25.056 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:50:25.057 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:50:25.058 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:50:25.058 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which rbd 2024-09-11T06:50:25.060 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/rbd 2024-09-11T06:50:25.060 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:50:25.060 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd --options notrim map rbd/testrbdnbd21607 2024-09-11T06:50:25.094 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:25.092+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:25.145 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T06:50:25.195 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:25.191+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:25.202 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:25.202 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:25.204 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV=/dev/nbd0 2024-09-11T06:50:25.204 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd 2024-09-11T06:50:25.204 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:50:25.205 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:50:25.205 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:50:25.205 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd0'\'']/id' 2024-09-11T06:50:25.218 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:50:25.219 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:50:25.269 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=22121 2024-09-11T06:50:25.269 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 22121 2024-09-11T06:50:25.270 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 22121 -C rbd-nbd 2024-09-11T06:50:25.275 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:50:25.275 INFO:tasks.workunit.client.0.smithi176.stdout: 22121 ? 00:00:00 rbd-nbd 2024-09-11T06:50:25.275 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd -p rbd --format xml du testrbdnbd21607 2024-09-11T06:50:25.276 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -m //stats/images/image/provisioned_size -v . 2024-09-11T06:50:25.305 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:25.303+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:25.379 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:25.379 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:25.395 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:50:25.401 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:50:25.401 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:50:25.402 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:25.399+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:25.419 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:50:25.420 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:50:25.438 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:50:25.438 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:50:25.459 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:50:25.459 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:50:25.479 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:50:25.480 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:50:25.497 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:25.495+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:25.500 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:50:25.500 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:50:25.516 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:25.516 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:25.517 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:50:25.517 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:25.530 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:50:25.547 INFO:tasks.workunit.client.0.smithi176.stderr:+ provisioned=67108864 2024-09-11T06:50:25.548 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd -p rbd --format xml du testrbdnbd21607 2024-09-11T06:50:25.548 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -m //stats/images/image/used_size -v . 2024-09-11T06:50:25.598 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:25.596+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:25.611 INFO:tasks.workunit.client.0.smithi176.stderr:+ used=67108864 2024-09-11T06:50:25.612 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 67108864 -eq 67108864 ']' 2024-09-11T06:50:25.612 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false _sudo blkdiscard /dev/nbd0 2024-09-11T06:50:25.612 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo blkdiscard /dev/nbd0 2024-09-11T06:50:25.612 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:25.612 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:25.612 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:25.613 INFO:tasks.workunit.client.0.smithi176.stderr:++ which blkdiscard 2024-09-11T06:50:25.614 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/sbin/blkdiscard 2024-09-11T06:50:25.614 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:25.615 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/sbin/blkdiscard /dev/nbd0 2024-09-11T06:50:25.631 INFO:tasks.workunit.client.0.smithi176.stderr:blkdiscard: /dev/nbd0: BLKDISCARD ioctl failed: Operation not supported 2024-09-11T06:50:25.632 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:25.632 INFO:tasks.workunit.client.0.smithi176.stderr:+ sync 2024-09-11T06:50:25.694 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:25.694 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:25.694 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:50:25.694 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:50:25.694 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:25.698 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:25.695+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:25.708 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:50:25.799 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:25.800+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:25.877 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:25.877 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:50:25.877 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:50:25.877 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:50:25.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.877 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:5 2.2 e17/15 2:5cea7035:::rbd_info:head v 17'1, mlcod=0'0)", 2024-09-11T06:50:25.877 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:50:25.877 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 73.637268234999993, 2024-09-11T06:50:25.878 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0019446719999999999, 2024-09-11T06:50:25.878 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:25.878 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:25.878 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:25.878 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:25.878 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.878 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:25.878 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:50:25.878 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.878 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.878 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:25.878 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:50:25.878 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.879 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.879 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:25.879 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236085+0000", 2024-09-11T06:50:25.879 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6000000000000003e-07 2024-09-11T06:50:25.879 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.879 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:25.879 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236108+0000", 2024-09-11T06:50:25.879 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3113e-05 2024-09-11T06:50:25.879 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.879 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:25.879 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236133+0000", 2024-09-11T06:50:25.879 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5723000000000001e-05 2024-09-11T06:50:25.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.880 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.880 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:25.880 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236154+0000", 2024-09-11T06:50:25.880 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0333999999999999e-05 2024-09-11T06:50:25.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.880 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.880 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:25.880 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236297+0000", 2024-09-11T06:50:25.880 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014278500000000001 2024-09-11T06:50:25.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.880 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.880 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:25.881 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236383+0000", 2024-09-11T06:50:25.881 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5927999999999995e-05 2024-09-11T06:50:25.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.881 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:25.881 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.237950+0000", 2024-09-11T06:50:25.881 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015673690000000001 2024-09-11T06:50:25.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.881 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:25.881 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.238029+0000", 2024-09-11T06:50:25.881 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8659999999999996e-05 2024-09-11T06:50:25.881 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.881 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:25.882 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.882 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:7 2.2 e18/15 2:5cea7035:::rbd_info:head v 18'2, mlcod=0'0)", 2024-09-11T06:50:25.882 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:50:25.882 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 72.622943027999995, 2024-09-11T06:50:25.882 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001907192, 2024-09-11T06:50:25.882 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:25.882 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:25.882 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:25.882 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:25.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.882 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:25.882 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:50:25.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.883 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:25.883 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:50:25.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.883 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:25.883 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250410+0000", 2024-09-11T06:50:25.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7700000000000004e-07 2024-09-11T06:50:25.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.883 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:25.883 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250445+0000", 2024-09-11T06:50:25.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4749000000000002e-05 2024-09-11T06:50:25.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.884 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:25.884 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250478+0000", 2024-09-11T06:50:25.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3269999999999998e-05 2024-09-11T06:50:25.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.884 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:25.884 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250501+0000", 2024-09-11T06:50:25.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3632999999999999e-05 2024-09-11T06:50:25.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:25.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250633+0000", 2024-09-11T06:50:25.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013143200000000001 2024-09-11T06:50:25.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:25.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250700+0000", 2024-09-11T06:50:25.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7002999999999995e-05 2024-09-11T06:50:25.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:25.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.252172+0000", 2024-09-11T06:50:25.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014719550000000001 2024-09-11T06:50:25.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:25.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.252316+0000", 2024-09-11T06:50:25.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144473 2024-09-11T06:50:25.886 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.886 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:25.886 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.886 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4558.0:5 3.2 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-11T06:50:25.886 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:50:25.886 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 7.784933798, 2024-09-11T06:50:25.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0062353959999999998, 2024-09-11T06:50:25.886 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:25.887 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:25.887 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:25.887 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:25.887 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4558", 2024-09-11T06:50:25.887 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1478388898", 2024-09-11T06:50:25.887 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 5 2024-09-11T06:50:25.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.887 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:25.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.887 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:25.887 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:50:25.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.888 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:25.888 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:50:25.888 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.888 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:25.888 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088422+0000", 2024-09-11T06:50:25.888 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7859999999999998e-06 2024-09-11T06:50:25.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.888 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:25.888 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088440+0000", 2024-09-11T06:50:25.888 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8185999999999998e-05 2024-09-11T06:50:25.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.889 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:25.889 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088479+0000", 2024-09-11T06:50:25.889 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8248999999999999e-05 2024-09-11T06:50:25.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.889 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:25.889 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088497+0000", 2024-09-11T06:50:25.889 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8889000000000002e-05 2024-09-11T06:50:25.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.889 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:25.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088658+0000", 2024-09-11T06:50:25.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000160271 2024-09-11T06:50:25.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op must wait for map", 2024-09-11T06:50:25.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088668+0000", 2024-09-11T06:50:25.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0409e-05 2024-09-11T06:50:25.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:25.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092347+0000", 2024-09-11T06:50:25.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00367915 2024-09-11T06:50:25.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:25.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092541+0000", 2024-09-11T06:50:25.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000193833 2024-09-11T06:50:25.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-11T06:50:25.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092732+0000", 2024-09-11T06:50:25.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000190772 2024-09-11T06:50:25.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:25.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094136+0000", 2024-09-11T06:50:25.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014040809999999999 2024-09-11T06:50:25.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:25.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094506+0000", 2024-09-11T06:50:25.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000369959 2024-09-11T06:50:25.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:25.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094590+0000", 2024-09-11T06:50:25.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4033e-05 2024-09-11T06:50:25.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.893 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:25.893 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094654+0000", 2024-09-11T06:50:25.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3777999999999998e-05 2024-09-11T06:50:25.893 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.893 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:25.893 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.893 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:259 2.6 2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head [sparse-read 4096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:25.893 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.147544+0000", 2024-09-11T06:50:25.893 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.7258084140000003, 2024-09-11T06:50:25.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0019824170000000002, 2024-09-11T06:50:25.893 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:25.894 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:25.894 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:25.894 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:25.894 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:25.894 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:25.894 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 259 2024-09-11T06:50:25.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.894 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:25.894 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.894 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:25.894 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.147544+0000", 2024-09-11T06:50:25.894 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.894 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:25.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.147544+0000", 2024-09-11T06:50:25.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:25.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.147545+0000", 2024-09-11T06:50:25.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4600000000000003e-07 2024-09-11T06:50:25.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.896 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:25.896 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.147549+0000", 2024-09-11T06:50:25.896 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8160000000000001e-06 2024-09-11T06:50:25.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.896 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:25.896 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.147565+0000", 2024-09-11T06:50:25.896 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5140000000000001e-05 2024-09-11T06:50:25.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.896 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:25.896 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.147574+0000", 2024-09-11T06:50:25.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2529999999999993e-06 2024-09-11T06:50:25.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:25.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.148551+0000", 2024-09-11T06:50:25.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0009775579999999999 2024-09-11T06:50:25.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:25.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.148694+0000", 2024-09-11T06:50:25.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014299399999999999 2024-09-11T06:50:25.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.898 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:25.898 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.149526+0000", 2024-09-11T06:50:25.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00083171000000000004 2024-09-11T06:50:25.898 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.898 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:25.898 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.898 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:261 2.6 2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:25.898 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.154411+0000", 2024-09-11T06:50:25.898 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.7189410000000001, 2024-09-11T06:50:25.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002245024, 2024-09-11T06:50:25.898 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:25.898 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:25.899 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:25.899 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:25.899 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:25.899 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:25.899 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 261 2024-09-11T06:50:25.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.899 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:25.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.899 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:25.899 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154411+0000", 2024-09-11T06:50:25.899 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.899 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:25.899 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154411+0000", 2024-09-11T06:50:25.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:25.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154412+0000", 2024-09-11T06:50:25.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:50:25.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:25.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154416+0000", 2024-09-11T06:50:25.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3930000000000001e-06 2024-09-11T06:50:25.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:25.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154428+0000", 2024-09-11T06:50:25.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2099999999999999e-05 2024-09-11T06:50:25.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:25.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154438+0000", 2024-09-11T06:50:25.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2590000000000006e-06 2024-09-11T06:50:25.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:25.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.155643+0000", 2024-09-11T06:50:25.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012048689999999999 2024-09-11T06:50:25.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.902 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:25.902 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.155838+0000", 2024-09-11T06:50:25.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019531299999999999 2024-09-11T06:50:25.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.902 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:25.902 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.156656+0000", 2024-09-11T06:50:25.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00081824700000000003 2024-09-11T06:50:25.902 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.902 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:25.902 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.903 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:269 2.6 2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head [sparse-read 1314816~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:25.903 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.166906+0000", 2024-09-11T06:50:25.903 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.706445972, 2024-09-11T06:50:25.903 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001538906, 2024-09-11T06:50:25.903 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:25.903 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:25.903 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:25.903 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:25.903 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:25.903 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:25.903 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 269 2024-09-11T06:50:25.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.903 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:25.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.904 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:25.904 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.166906+0000", 2024-09-11T06:50:25.904 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.904 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:25.904 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.166906+0000", 2024-09-11T06:50:25.904 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.904 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:25.904 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.166907+0000", 2024-09-11T06:50:25.904 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2729999999999999e-06 2024-09-11T06:50:25.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.905 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:25.905 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.166914+0000", 2024-09-11T06:50:25.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3230000000000003e-06 2024-09-11T06:50:25.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.905 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:25.905 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.166934+0000", 2024-09-11T06:50:25.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.018e-05 2024-09-11T06:50:25.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.905 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:25.905 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.166951+0000", 2024-09-11T06:50:25.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7501999999999999e-05 2024-09-11T06:50:25.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:25.906 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.167677+0000", 2024-09-11T06:50:25.906 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00072579299999999997 2024-09-11T06:50:25.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:25.906 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.167865+0000", 2024-09-11T06:50:25.906 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000188023 2024-09-11T06:50:25.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:25.907 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.168445+0000", 2024-09-11T06:50:25.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00057981199999999997 2024-09-11T06:50:25.907 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.907 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:25.907 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.907 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:277 2.6 2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head [sparse-read 2363392~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:25.907 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.179016+0000", 2024-09-11T06:50:25.907 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.6943359180000002, 2024-09-11T06:50:25.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015587540000000001, 2024-09-11T06:50:25.907 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:25.907 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:25.907 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:25.908 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:25.908 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:25.908 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:25.908 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 277 2024-09-11T06:50:25.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.908 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:25.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:25.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.179016+0000", 2024-09-11T06:50:25.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:25.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.179016+0000", 2024-09-11T06:50:25.909 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.909 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:25.909 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.179017+0000", 2024-09-11T06:50:25.909 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.78e-07 2024-09-11T06:50:25.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.909 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:25.909 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.179021+0000", 2024-09-11T06:50:25.909 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9469999999999996e-06 2024-09-11T06:50:25.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.909 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:25.910 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.179032+0000", 2024-09-11T06:50:25.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0763e-05 2024-09-11T06:50:25.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.910 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:25.910 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.179040+0000", 2024-09-11T06:50:25.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7360000000000001e-06 2024-09-11T06:50:25.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.910 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:25.910 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.179817+0000", 2024-09-11T06:50:25.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00077775700000000001 2024-09-11T06:50:25.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.911 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:25.911 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.179958+0000", 2024-09-11T06:50:25.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014041800000000001 2024-09-11T06:50:25.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.911 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:25.911 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.180575+0000", 2024-09-11T06:50:25.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00061725500000000004 2024-09-11T06:50:25.911 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.911 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:25.911 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.912 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.912 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:285 2.6 2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head [sparse-read 3411968~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:25.912 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.190856+0000", 2024-09-11T06:50:25.912 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.6824962689999996, 2024-09-11T06:50:25.912 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001510352, 2024-09-11T06:50:25.912 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:25.912 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:25.912 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:25.912 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:25.912 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:25.912 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:25.912 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 285 2024-09-11T06:50:25.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.913 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:25.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:25.913 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.190856+0000", 2024-09-11T06:50:25.913 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:25.913 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.190856+0000", 2024-09-11T06:50:25.913 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:25.913 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.190857+0000", 2024-09-11T06:50:25.914 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1200000000000002e-07 2024-09-11T06:50:25.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.914 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.914 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:25.914 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.190860+0000", 2024-09-11T06:50:25.914 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6839999999999998e-06 2024-09-11T06:50:25.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.914 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.914 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:25.914 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.190877+0000", 2024-09-11T06:50:25.914 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6378000000000001e-05 2024-09-11T06:50:25.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.914 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.914 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:25.915 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.190885+0000", 2024-09-11T06:50:25.915 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.551e-06 2024-09-11T06:50:25.915 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.915 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:25.915 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.191593+0000", 2024-09-11T06:50:25.915 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070753199999999997 2024-09-11T06:50:25.915 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.915 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:25.915 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.191731+0000", 2024-09-11T06:50:25.915 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013835099999999999 2024-09-11T06:50:25.915 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.916 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:25.916 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.192366+0000", 2024-09-11T06:50:25.916 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00063514400000000003 2024-09-11T06:50:25.916 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.916 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:25.916 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.916 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.916 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:524 2.4 2:2a17619e:::rbd_data.1314f0a2bfa2.000000000000000e:head [sparse-read 135168~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:25.916 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.530346+0000", 2024-09-11T06:50:25.916 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3430064079999999, 2024-09-11T06:50:25.916 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012158220000000001, 2024-09-11T06:50:25.916 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:25.916 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:25.917 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:25.917 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:25.917 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:25.917 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:25.917 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 524 2024-09-11T06:50:25.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.917 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:25.917 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.917 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:25.917 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530346+0000", 2024-09-11T06:50:25.917 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.917 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.918 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:25.918 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530346+0000", 2024-09-11T06:50:25.918 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.918 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:25.918 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530348+0000", 2024-09-11T06:50:25.918 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0669999999999999e-06 2024-09-11T06:50:25.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.918 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:25.918 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530358+0000", 2024-09-11T06:50:25.918 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0246e-05 2024-09-11T06:50:25.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.919 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.919 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:25.919 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530389+0000", 2024-09-11T06:50:25.919 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0476000000000001e-05 2024-09-11T06:50:25.919 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.919 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.919 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:25.919 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530411+0000", 2024-09-11T06:50:25.919 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2952e-05 2024-09-11T06:50:25.919 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.919 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.919 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:25.919 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530581+0000", 2024-09-11T06:50:25.919 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016948700000000001 2024-09-11T06:50:25.920 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.920 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.920 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:25.920 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530706+0000", 2024-09-11T06:50:25.920 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012535199999999999 2024-09-11T06:50:25.920 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.920 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.920 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:25.920 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.531562+0000", 2024-09-11T06:50:25.920 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00085524199999999998 2024-09-11T06:50:25.920 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.920 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:25.920 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.921 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.921 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:525 2.4 2:2a17619e:::rbd_data.1314f0a2bfa2.000000000000000e:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:25.921 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.530454+0000", 2024-09-11T06:50:25.921 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3428983150000002, 2024-09-11T06:50:25.921 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0019464650000000001, 2024-09-11T06:50:25.921 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:25.921 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:25.924 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:25.924 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:25.925 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:25.925 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:25.925 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 525 2024-09-11T06:50:25.925 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.925 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:25.925 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.925 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:25.925 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530454+0000", 2024-09-11T06:50:25.925 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.925 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.925 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.925 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:25.925 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530454+0000", 2024-09-11T06:50:25.926 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.926 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.926 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:25.926 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530455+0000", 2024-09-11T06:50:25.926 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4100000000000001e-06 2024-09-11T06:50:25.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.926 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.926 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:25.926 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530462+0000", 2024-09-11T06:50:25.926 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2760000000000003e-06 2024-09-11T06:50:25.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.926 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.927 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:25.927 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530486+0000", 2024-09-11T06:50:25.927 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3082999999999999e-05 2024-09-11T06:50:25.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.927 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.927 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:25.927 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530511+0000", 2024-09-11T06:50:25.927 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5065000000000001e-05 2024-09-11T06:50:25.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.927 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.927 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:25.927 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.531637+0000", 2024-09-11T06:50:25.927 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011263530000000001 2024-09-11T06:50:25.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.928 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:25.928 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.531716+0000", 2024-09-11T06:50:25.928 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8565000000000004e-05 2024-09-11T06:50:25.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.928 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:25.928 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.532400+0000", 2024-09-11T06:50:25.928 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00068471299999999999 2024-09-11T06:50:25.928 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.928 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:25.928 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.929 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:527 2.4 2:2a17619e:::rbd_data.1314f0a2bfa2.000000000000000e:head [sparse-read 528384~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:25.929 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.533265+0000", 2024-09-11T06:50:25.929 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3400873320000004, 2024-09-11T06:50:25.929 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001244071, 2024-09-11T06:50:25.929 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:25.929 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:25.929 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:25.929 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:25.929 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:25.929 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:25.929 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 527 2024-09-11T06:50:25.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.930 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:25.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.930 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:25.930 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.533265+0000", 2024-09-11T06:50:25.930 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.930 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:25.930 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.533265+0000", 2024-09-11T06:50:25.930 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.930 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:25.931 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.533266+0000", 2024-09-11T06:50:25.931 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5999999999999999e-06 2024-09-11T06:50:25.931 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.931 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.931 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:25.931 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.533274+0000", 2024-09-11T06:50:25.931 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4930000000000001e-06 2024-09-11T06:50:25.931 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.931 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.931 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:25.931 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.533294+0000", 2024-09-11T06:50:25.931 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0506e-05 2024-09-11T06:50:25.931 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.932 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:25.932 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.533312+0000", 2024-09-11T06:50:25.932 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7413000000000001e-05 2024-09-11T06:50:25.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.932 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:25.932 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.533774+0000", 2024-09-11T06:50:25.932 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00046220700000000001 2024-09-11T06:50:25.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.932 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:25.932 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.533855+0000", 2024-09-11T06:50:25.933 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1203000000000002e-05 2024-09-11T06:50:25.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.933 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:25.933 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.534509+0000", 2024-09-11T06:50:25.933 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00065364899999999996 2024-09-11T06:50:25.933 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.933 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:25.933 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.933 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:530 2.4 2:2a17619e:::rbd_data.1314f0a2bfa2.000000000000000e:head [sparse-read 921600~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:25.933 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.536052+0000", 2024-09-11T06:50:25.933 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3373000639999999, 2024-09-11T06:50:25.934 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012798600000000001, 2024-09-11T06:50:25.934 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:25.934 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:25.934 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:25.934 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:25.934 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:25.934 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:25.934 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 530 2024-09-11T06:50:25.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.934 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:25.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.934 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:25.934 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.536052+0000", 2024-09-11T06:50:25.934 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.935 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.935 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:25.935 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.536052+0000", 2024-09-11T06:50:25.935 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.935 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.935 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:25.935 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.536054+0000", 2024-09-11T06:50:25.935 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6640000000000001e-06 2024-09-11T06:50:25.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.935 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.935 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:25.935 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.536062+0000", 2024-09-11T06:50:25.935 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3760000000000008e-06 2024-09-11T06:50:25.936 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.936 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:25.936 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.536084+0000", 2024-09-11T06:50:25.936 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1988999999999999e-05 2024-09-11T06:50:25.936 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.936 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:25.936 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.536102+0000", 2024-09-11T06:50:25.936 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8142999999999999e-05 2024-09-11T06:50:25.936 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.936 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.536435+0000", 2024-09-11T06:50:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000332548 2024-09-11T06:50:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.536611+0000", 2024-09-11T06:50:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017594299999999999 2024-09-11T06:50:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.537332+0000", 2024-09-11T06:50:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00072119700000000003 2024-09-11T06:50:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.938 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:25.938 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.938 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:533 2.4 2:2a17619e:::rbd_data.1314f0a2bfa2.000000000000000e:head [sparse-read 1314816~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:25.938 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.542984+0000", 2024-09-11T06:50:25.938 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3303679720000003, 2024-09-11T06:50:25.938 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001565873, 2024-09-11T06:50:25.938 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:25.938 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:25.938 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:25.938 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:25.938 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:25.939 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:25.939 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 533 2024-09-11T06:50:25.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.939 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:25.939 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.939 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:25.939 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.542984+0000", 2024-09-11T06:50:25.939 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.939 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.939 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:25.939 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.542984+0000", 2024-09-11T06:50:25.939 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.940 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:25.940 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.542985+0000", 2024-09-11T06:50:25.940 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.217e-06 2024-09-11T06:50:25.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.940 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:25.940 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.542993+0000", 2024-09-11T06:50:25.940 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2799999999999998e-06 2024-09-11T06:50:25.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.940 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:25.940 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.543007+0000", 2024-09-11T06:50:25.940 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4046000000000001e-05 2024-09-11T06:50:25.941 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.941 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.941 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:25.941 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.543022+0000", 2024-09-11T06:50:25.941 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5206e-05 2024-09-11T06:50:25.941 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.941 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.941 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:25.941 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.543871+0000", 2024-09-11T06:50:25.941 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000849459 2024-09-11T06:50:25.941 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.941 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.941 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.544004+0000", 2024-09-11T06:50:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000132958 2024-09-11T06:50:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.544550+0000", 2024-09-11T06:50:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00054570699999999998 2024-09-11T06:50:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:535 2.4 2:2a17619e:::rbd_data.1314f0a2bfa2.000000000000000e:head [sparse-read 1576960~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.545515+0000", 2024-09-11T06:50:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3278367979999999, 2024-09-11T06:50:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001475706, 2024-09-11T06:50:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 535 2024-09-11T06:50:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.545515+0000", 2024-09-11T06:50:25.944 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.944 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:25.944 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.545515+0000", 2024-09-11T06:50:25.944 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.944 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:25.944 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.545517+0000", 2024-09-11T06:50:25.944 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.536e-06 2024-09-11T06:50:25.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.945 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:25.945 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.545525+0000", 2024-09-11T06:50:25.945 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6769999999999995e-06 2024-09-11T06:50:25.945 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.945 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.945 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:25.945 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.545546+0000", 2024-09-11T06:50:25.945 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1676e-05 2024-09-11T06:50:25.945 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.945 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.945 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:25.945 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.545564+0000", 2024-09-11T06:50:25.945 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7915e-05 2024-09-11T06:50:25.945 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.946 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:25.946 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.546295+0000", 2024-09-11T06:50:25.946 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00073042099999999998 2024-09-11T06:50:25.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.946 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:25.946 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.546410+0000", 2024-09-11T06:50:25.946 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011579 2024-09-11T06:50:25.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.946 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:25.946 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.546991+0000", 2024-09-11T06:50:25.946 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00058069100000000004 2024-09-11T06:50:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:541 2.4 2:2a17619e:::rbd_data.1314f0a2bfa2.000000000000000e:head [sparse-read 2363392~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.555808+0000", 2024-09-11T06:50:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.317543659, 2024-09-11T06:50:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015018779999999999, 2024-09-11T06:50:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:25.948 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 541 2024-09-11T06:50:25.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.948 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:25.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.948 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:25.948 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.555808+0000", 2024-09-11T06:50:25.948 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.948 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:25.948 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.555808+0000", 2024-09-11T06:50:25.948 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.949 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:25.949 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.555810+0000", 2024-09-11T06:50:25.949 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8199999999999999e-06 2024-09-11T06:50:25.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.949 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:25.949 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.555818+0000", 2024-09-11T06:50:25.949 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9759999999999995e-06 2024-09-11T06:50:25.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.949 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:25.949 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.555842+0000", 2024-09-11T06:50:25.949 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3428999999999999e-05 2024-09-11T06:50:25.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.950 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:25.950 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.555858+0000", 2024-09-11T06:50:25.950 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5826000000000001e-05 2024-09-11T06:50:25.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.950 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:25.950 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.556619+0000", 2024-09-11T06:50:25.950 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00076185000000000001 2024-09-11T06:50:25.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.950 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:25.950 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.556732+0000", 2024-09-11T06:50:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000112705 2024-09-11T06:50:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.557310+0000", 2024-09-11T06:50:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00057827200000000001 2024-09-11T06:50:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:549 2.4 2:2a17619e:::rbd_data.1314f0a2bfa2.000000000000000e:head [sparse-read 3411968~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.567982+0000", 2024-09-11T06:50:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3053696690000001, 2024-09-11T06:50:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002000216, 2024-09-11T06:50:25.952 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:25.952 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:25.952 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:25.952 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:25.952 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:25.952 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:25.952 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 549 2024-09-11T06:50:25.952 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.952 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:25.952 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.952 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:25.952 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.567982+0000", 2024-09-11T06:50:25.952 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.953 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:25.953 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.567982+0000", 2024-09-11T06:50:25.953 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.953 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:25.953 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.567984+0000", 2024-09-11T06:50:25.953 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7290000000000001e-06 2024-09-11T06:50:25.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.953 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:25.954 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.567994+0000", 2024-09-11T06:50:25.954 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4229999999999994e-06 2024-09-11T06:50:25.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.954 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.954 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:25.954 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.568015+0000", 2024-09-11T06:50:25.954 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1596000000000001e-05 2024-09-11T06:50:25.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.954 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.954 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:25.954 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.568032+0000", 2024-09-11T06:50:25.954 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7067000000000001e-05 2024-09-11T06:50:25.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.955 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:25.955 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.568713+0000", 2024-09-11T06:50:25.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00068109799999999999 2024-09-11T06:50:25.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.955 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:25.955 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.568866+0000", 2024-09-11T06:50:25.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015232399999999999 2024-09-11T06:50:25.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.955 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:25.955 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.569983+0000", 2024-09-11T06:50:25.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011169789999999999 2024-09-11T06:50:25.956 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.956 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:25.956 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.956 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:550 2.4 2:2a17619e:::rbd_data.1314f0a2bfa2.000000000000000e:head [sparse-read 3543040~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:25.956 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.569595+0000", 2024-09-11T06:50:25.956 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3037566419999997, 2024-09-11T06:50:25.956 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001587565, 2024-09-11T06:50:25.956 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:25.956 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:25.956 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:25.956 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:25.957 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:25.957 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:25.957 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 550 2024-09-11T06:50:25.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.957 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:25.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.957 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:25.957 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.569595+0000", 2024-09-11T06:50:25.957 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.957 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:25.957 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.569595+0000", 2024-09-11T06:50:25.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.958 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:25.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.569597+0000", 2024-09-11T06:50:25.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6130000000000001e-06 2024-09-11T06:50:25.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.958 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:25.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.569604+0000", 2024-09-11T06:50:25.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2429999999999999e-06 2024-09-11T06:50:25.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.958 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:25.959 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.569626+0000", 2024-09-11T06:50:25.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1478999999999999e-05 2024-09-11T06:50:25.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.959 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.959 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:25.959 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.569643+0000", 2024-09-11T06:50:25.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6973000000000002e-05 2024-09-11T06:50:25.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.959 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.959 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:25.959 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.570088+0000", 2024-09-11T06:50:25.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00044539999999999998 2024-09-11T06:50:25.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.959 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.960 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:25.960 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.570227+0000", 2024-09-11T06:50:25.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013836499999999999 2024-09-11T06:50:25.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.960 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:25.960 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.571183+0000", 2024-09-11T06:50:25.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00095649199999999998 2024-09-11T06:50:25.960 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.960 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:25.960 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.961 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:551 2.4 2:2a17619e:::rbd_data.1314f0a2bfa2.000000000000000e:head [sparse-read 3674112~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:25.961 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.570987+0000", 2024-09-11T06:50:25.961 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3023652329999997, 2024-09-11T06:50:25.961 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001264453, 2024-09-11T06:50:25.961 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:25.961 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:25.961 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:25.961 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:25.961 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:25.961 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:25.961 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 551 2024-09-11T06:50:25.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.961 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:25.961 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:25.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.570987+0000", 2024-09-11T06:50:25.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:25.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.570987+0000", 2024-09-11T06:50:25.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:25.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.570989+0000", 2024-09-11T06:50:25.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.356e-06 2024-09-11T06:50:25.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.570998+0000", 2024-09-11T06:50:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3270000000000001e-06 2024-09-11T06:50:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.571035+0000", 2024-09-11T06:50:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7008000000000002e-05 2024-09-11T06:50:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.571052+0000", 2024-09-11T06:50:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7323e-05 2024-09-11T06:50:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.964 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.964 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:25.964 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.571222+0000", 2024-09-11T06:50:25.964 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016963699999999999 2024-09-11T06:50:25.964 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.964 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.964 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:25.964 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.571333+0000", 2024-09-11T06:50:25.964 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000111859 2024-09-11T06:50:25.964 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.964 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.964 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:25.964 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.572251+0000", 2024-09-11T06:50:25.965 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00091794299999999995 2024-09-11T06:50:25.965 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.965 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:25.965 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.965 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.965 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:554 2.4 2:2a17619e:::rbd_data.1314f0a2bfa2.000000000000000e:head [sparse-read 4067328~126976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:25.965 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.574581+0000", 2024-09-11T06:50:25.965 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.2987715489999996, 2024-09-11T06:50:25.965 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001401228, 2024-09-11T06:50:25.965 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:25.965 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:25.965 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:25.965 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 554 2024-09-11T06:50:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.574581+0000", 2024-09-11T06:50:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.574581+0000", 2024-09-11T06:50:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.967 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:25.967 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.574582+0000", 2024-09-11T06:50:25.967 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.46e-06 2024-09-11T06:50:25.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.967 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:25.967 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.574588+0000", 2024-09-11T06:50:25.967 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.443e-06 2024-09-11T06:50:25.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.967 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:25.967 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.574609+0000", 2024-09-11T06:50:25.968 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0965999999999999e-05 2024-09-11T06:50:25.968 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.968 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.968 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:25.968 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.574626+0000", 2024-09-11T06:50:25.968 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6699000000000001e-05 2024-09-11T06:50:25.968 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.968 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.968 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:25.968 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.574786+0000", 2024-09-11T06:50:25.968 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016030100000000001 2024-09-11T06:50:25.968 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.969 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:25.969 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.575017+0000", 2024-09-11T06:50:25.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023037900000000001 2024-09-11T06:50:25.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.969 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:25.969 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.575982+0000", 2024-09-11T06:50:25.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00096498000000000005 2024-09-11T06:50:25.969 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.969 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:25.969 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.970 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:587 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-11T06:50:25.970 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:50:25.970 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 3.6441011429999999, 2024-09-11T06:50:25.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0023436289999999999, 2024-09-11T06:50:25.970 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:25.970 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:25.970 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:25.970 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:25.970 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:25.970 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:25.970 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 587 2024-09-11T06:50:25.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.970 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:25.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:25.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:50:25.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:25.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:50:25.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:25.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:25.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229252+0000", 2024-09-11T06:50:25.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3799999999999999e-06 2024-09-11T06:50:25.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:25.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229259+0000", 2024-09-11T06:50:25.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4760000000000003e-06 2024-09-11T06:50:25.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:25.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229271+0000", 2024-09-11T06:50:25.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2313000000000001e-05 2024-09-11T06:50:25.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:25.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229279+0000", 2024-09-11T06:50:25.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1610000000000003e-06 2024-09-11T06:50:25.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:25.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229344+0000", 2024-09-11T06:50:25.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4375000000000006e-05 2024-09-11T06:50:25.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:25.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229404+0000", 2024-09-11T06:50:25.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0517999999999999e-05 2024-09-11T06:50:25.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:25.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229516+0000", 2024-09-11T06:50:25.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011129900000000001 2024-09-11T06:50:25.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:25.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231502+0000", 2024-09-11T06:50:25.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001986238 2024-09-11T06:50:25.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:25.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231537+0000", 2024-09-11T06:50:25.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4968000000000001e-05 2024-09-11T06:50:25.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:25.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231569+0000", 2024-09-11T06:50:25.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2459000000000002e-05 2024-09-11T06:50:25.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:25.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:25.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:25.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231595+0000", 2024-09-11T06:50:25.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5442000000000001e-05 2024-09-11T06:50:25.975 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.975 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:25.975 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.976 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:25.976 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:25.976 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:25.977 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:50:25.978 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:25.975+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:26.078 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:26.076+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:26.147 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is unset 2024-09-11T06:50:26.148 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:26.149 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:26.149 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:50:26.149 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:26.170 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:50:26.217 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:26.215+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:26.280 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:26.275+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:26.285 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:26.285 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:26.286 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:50:26.286 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:50:26.286 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:26.300 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:50:26.381 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:26.380+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:26.463 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:26.463 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:50:26.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:50:26.463 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:50:26.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.464 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:490 2.1 2:97696759:::rbd_data.1314f0a2bfa2.000000000000000d:head [sparse-read 4096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:26.464 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.471687+0000", 2024-09-11T06:50:26.464 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.989688857, 2024-09-11T06:50:26.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0018255929999999999, 2024-09-11T06:50:26.464 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:26.464 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:26.464 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:26.464 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:26.464 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:26.464 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:26.464 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 490 2024-09-11T06:50:26.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.464 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:26.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.465 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:26.465 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.471687+0000", 2024-09-11T06:50:26.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.465 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:26.465 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.471687+0000", 2024-09-11T06:50:26.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.465 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:26.465 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.471689+0000", 2024-09-11T06:50:26.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5340000000000001e-06 2024-09-11T06:50:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.471696+0000", 2024-09-11T06:50:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.746e-06 2024-09-11T06:50:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.471711+0000", 2024-09-11T06:50:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4912999999999999e-05 2024-09-11T06:50:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.471721+0000", 2024-09-11T06:50:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8830000000000001e-06 2024-09-11T06:50:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.472699+0000", 2024-09-11T06:50:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0009781499999999999 2024-09-11T06:50:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.472800+0000", 2024-09-11T06:50:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010076 2024-09-11T06:50:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.473513+0000", 2024-09-11T06:50:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00071260699999999998 2024-09-11T06:50:26.468 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.468 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:26.468 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.468 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:492 2.1 2:97696759:::rbd_data.1314f0a2bfa2.000000000000000d:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:26.468 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.477730+0000", 2024-09-11T06:50:26.468 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.9836460489999999, 2024-09-11T06:50:26.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0018544119999999999, 2024-09-11T06:50:26.468 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:26.468 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:26.468 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:26.468 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:26.468 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 492 2024-09-11T06:50:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.477730+0000", 2024-09-11T06:50:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.477730+0000", 2024-09-11T06:50:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.470 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:26.470 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.477731+0000", 2024-09-11T06:50:26.470 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0470000000000001e-06 2024-09-11T06:50:26.470 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.470 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.470 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:26.470 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.477735+0000", 2024-09-11T06:50:26.470 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4700000000000004e-06 2024-09-11T06:50:26.470 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.470 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.470 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:26.470 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.477749+0000", 2024-09-11T06:50:26.470 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3276e-05 2024-09-11T06:50:26.470 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.470 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.471 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:26.471 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.477758+0000", 2024-09-11T06:50:26.471 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9700000000000005e-06 2024-09-11T06:50:26.471 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.471 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.471 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:26.471 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.478626+0000", 2024-09-11T06:50:26.471 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000868581 2024-09-11T06:50:26.471 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.471 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.471 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:26.471 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.478780+0000", 2024-09-11T06:50:26.471 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015381499999999999 2024-09-11T06:50:26.471 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.471 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.472 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:26.472 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.479584+0000", 2024-09-11T06:50:26.472 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000804253 2024-09-11T06:50:26.472 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.472 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:26.472 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.472 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.472 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.472 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:500 2.1 2:97696759:::rbd_data.1314f0a2bfa2.000000000000000d:head [sparse-read 1314816~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:26.472 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.490524+0000", 2024-09-11T06:50:26.472 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.9708517490000004, 2024-09-11T06:50:26.472 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0020214260000000002, 2024-09-11T06:50:26.472 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:26.472 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:26.473 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:26.473 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:26.473 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:26.473 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:26.473 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 500 2024-09-11T06:50:26.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.473 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:26.473 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.473 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:26.473 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.490524+0000", 2024-09-11T06:50:26.473 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.473 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.473 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:26.474 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.490524+0000", 2024-09-11T06:50:26.474 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.474 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.474 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:26.474 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.490525+0000", 2024-09-11T06:50:26.474 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3399999999999998e-07 2024-09-11T06:50:26.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.474 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.474 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:26.474 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.490529+0000", 2024-09-11T06:50:26.474 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7100000000000001e-06 2024-09-11T06:50:26.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.475 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:26.475 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.490542+0000", 2024-09-11T06:50:26.475 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3346000000000001e-05 2024-09-11T06:50:26.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.475 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:26.475 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.490554+0000", 2024-09-11T06:50:26.475 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1719000000000001e-05 2024-09-11T06:50:26.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.475 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:26.475 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.491455+0000", 2024-09-11T06:50:26.475 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00090156399999999999 2024-09-11T06:50:26.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.476 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.476 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:26.476 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.491684+0000", 2024-09-11T06:50:26.476 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022832500000000001 2024-09-11T06:50:26.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.476 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.476 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:26.476 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.492546+0000", 2024-09-11T06:50:26.476 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00086192799999999998 2024-09-11T06:50:26.476 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.476 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:26.476 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.476 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.477 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:503 2.1 2:97696759:::rbd_data.1314f0a2bfa2.000000000000000d:head [sparse-read 1708032~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:26.477 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.494210+0000", 2024-09-11T06:50:26.477 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.967166164, 2024-09-11T06:50:26.477 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00146407, 2024-09-11T06:50:26.477 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:26.477 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:26.477 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:26.477 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:26.477 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:26.477 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:26.477 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 503 2024-09-11T06:50:26.477 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.477 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:26.478 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.478 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:26.478 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.494210+0000", 2024-09-11T06:50:26.478 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.478 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.478 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:26.478 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.494210+0000", 2024-09-11T06:50:26.478 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.478 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.478 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:26.478 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.494212+0000", 2024-09-11T06:50:26.478 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9999999999999999e-06 2024-09-11T06:50:26.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.479 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.479 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:26.479 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.494222+0000", 2024-09-11T06:50:26.479 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8130000000000007e-06 2024-09-11T06:50:26.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.479 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.479 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:26.479 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.494251+0000", 2024-09-11T06:50:26.479 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9771999999999999e-05 2024-09-11T06:50:26.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.479 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.479 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:26.479 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.494271+0000", 2024-09-11T06:50:26.480 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9429999999999999e-05 2024-09-11T06:50:26.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.480 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.480 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:26.480 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.494668+0000", 2024-09-11T06:50:26.480 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00039718199999999999 2024-09-11T06:50:26.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.480 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.480 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:26.480 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.494838+0000", 2024-09-11T06:50:26.480 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016983699999999999 2024-09-11T06:50:26.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.480 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.481 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:26.481 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.495674+0000", 2024-09-11T06:50:26.481 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00083603599999999996 2024-09-11T06:50:26.481 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.481 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:26.481 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.481 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.481 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:508 2.1 2:97696759:::rbd_data.1314f0a2bfa2.000000000000000d:head [sparse-read 2363392~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:26.481 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.503367+0000", 2024-09-11T06:50:26.481 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.9580086459999997, 2024-09-11T06:50:26.481 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00206821, 2024-09-11T06:50:26.481 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:26.481 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:26.482 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:26.482 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:26.482 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:26.482 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:26.482 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 508 2024-09-11T06:50:26.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.482 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:26.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.482 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:26.482 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.503367+0000", 2024-09-11T06:50:26.482 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.482 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:26.483 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.503367+0000", 2024-09-11T06:50:26.483 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.483 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.483 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:26.483 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.503368+0000", 2024-09-11T06:50:26.483 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6000000000000002e-07 2024-09-11T06:50:26.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.483 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.483 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:26.483 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.503388+0000", 2024-09-11T06:50:26.483 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9375e-05 2024-09-11T06:50:26.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.483 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.484 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:26.484 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.503400+0000", 2024-09-11T06:50:26.484 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2707e-05 2024-09-11T06:50:26.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.484 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.484 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:26.484 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.503409+0000", 2024-09-11T06:50:26.484 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8959999999999992e-06 2024-09-11T06:50:26.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.484 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.484 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:26.484 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.504372+0000", 2024-09-11T06:50:26.484 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00096251500000000005 2024-09-11T06:50:26.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.485 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.485 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:26.485 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.504516+0000", 2024-09-11T06:50:26.485 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014421400000000001 2024-09-11T06:50:26.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.485 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.485 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:26.485 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.505436+0000", 2024-09-11T06:50:26.485 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00091964300000000004 2024-09-11T06:50:26.485 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.485 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:26.485 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.486 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.486 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:516 2.1 2:97696759:::rbd_data.1314f0a2bfa2.000000000000000d:head [sparse-read 3411968~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:26.486 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.516855+0000", 2024-09-11T06:50:26.486 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.9445210519999998, 2024-09-11T06:50:26.486 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0021435920000000002, 2024-09-11T06:50:26.486 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:26.486 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:26.486 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:26.486 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:26.486 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:26.486 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:26.486 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 516 2024-09-11T06:50:26.486 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.487 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:26.487 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.487 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:26.487 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.516855+0000", 2024-09-11T06:50:26.487 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.487 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.487 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.487 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:26.487 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.516855+0000", 2024-09-11T06:50:26.487 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.487 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.487 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.487 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:26.487 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.516856+0000", 2024-09-11T06:50:26.488 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1930000000000001e-06 2024-09-11T06:50:26.488 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.488 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.488 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:26.488 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.516861+0000", 2024-09-11T06:50:26.488 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6070000000000001e-06 2024-09-11T06:50:26.488 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.488 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.488 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:26.488 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.516875+0000", 2024-09-11T06:50:26.488 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3916e-05 2024-09-11T06:50:26.488 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.488 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.489 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:26.489 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.516884+0000", 2024-09-11T06:50:26.489 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5300000000000002e-06 2024-09-11T06:50:26.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.489 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.489 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:26.489 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.517883+0000", 2024-09-11T06:50:26.489 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00099854700000000002 2024-09-11T06:50:26.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.489 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.489 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:26.489 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.518070+0000", 2024-09-11T06:50:26.489 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00018680899999999999 2024-09-11T06:50:26.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.490 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.490 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:26.490 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.518999+0000", 2024-09-11T06:50:26.490 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00092898999999999998 2024-09-11T06:50:26.490 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.490 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:26.490 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.490 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.490 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:520 2.1 2:97696759:::rbd_data.1314f0a2bfa2.000000000000000d:head [sparse-read 3936256~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:26.490 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.522273+0000", 2024-09-11T06:50:26.490 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.9391028439999998, 2024-09-11T06:50:26.490 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014272639999999999, 2024-09-11T06:50:26.491 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:26.491 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:26.491 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:26.491 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:26.491 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:26.491 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:26.491 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 520 2024-09-11T06:50:26.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.491 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:26.492 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.492 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:26.492 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.522273+0000", 2024-09-11T06:50:26.492 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.492 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.492 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.492 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:26.492 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.522273+0000", 2024-09-11T06:50:26.492 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.492 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.492 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.492 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:26.492 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.522275+0000", 2024-09-11T06:50:26.493 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0729999999999999e-06 2024-09-11T06:50:26.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.493 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.493 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:26.493 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.522285+0000", 2024-09-11T06:50:26.493 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8730000000000002e-06 2024-09-11T06:50:26.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.493 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.493 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:26.494 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.522315+0000", 2024-09-11T06:50:26.494 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0249000000000001e-05 2024-09-11T06:50:26.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.494 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.494 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:26.494 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.522330+0000", 2024-09-11T06:50:26.494 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4839e-05 2024-09-11T06:50:26.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.494 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.494 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:26.494 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.522423+0000", 2024-09-11T06:50:26.494 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2769999999999994e-05 2024-09-11T06:50:26.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.494 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.495 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:26.495 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.522585+0000", 2024-09-11T06:50:26.495 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016196799999999999 2024-09-11T06:50:26.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.495 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.495 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:26.495 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.523700+0000", 2024-09-11T06:50:26.495 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001115492 2024-09-11T06:50:26.495 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.495 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:26.495 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.495 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.495 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:558 2.1 2:88b29bb7:::rbd_data.1314f0a2bfa2.000000000000000f:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:26.496 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.581983+0000", 2024-09-11T06:50:26.496 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.8793929760000001, 2024-09-11T06:50:26.496 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0021559639999999998, 2024-09-11T06:50:26.496 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:26.496 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:26.496 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:26.496 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:26.496 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:26.496 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:26.496 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 558 2024-09-11T06:50:26.496 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.496 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:26.496 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.496 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:26.497 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.581983+0000", 2024-09-11T06:50:26.497 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.497 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.497 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:26.497 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.581983+0000", 2024-09-11T06:50:26.497 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.497 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.497 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:26.497 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.581985+0000", 2024-09-11T06:50:26.497 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8530000000000001e-06 2024-09-11T06:50:26.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.497 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.498 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:26.498 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.581993+0000", 2024-09-11T06:50:26.498 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4999999999999999e-06 2024-09-11T06:50:26.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.498 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:26.498 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.582020+0000", 2024-09-11T06:50:26.498 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6954999999999999e-05 2024-09-11T06:50:26.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.498 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:26.498 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.582041+0000", 2024-09-11T06:50:26.498 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0919000000000001e-05 2024-09-11T06:50:26.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:26.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.583144+0000", 2024-09-11T06:50:26.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00110301 2024-09-11T06:50:26.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:26.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.583316+0000", 2024-09-11T06:50:26.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017154 2024-09-11T06:50:26.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:26.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.584139+0000", 2024-09-11T06:50:26.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00082318699999999996 2024-09-11T06:50:26.500 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.500 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:26.500 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.500 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:561 2.1 2:88b29bb7:::rbd_data.1314f0a2bfa2.000000000000000f:head [sparse-read 659456~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:26.500 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.586127+0000", 2024-09-11T06:50:26.500 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.8752491649999996, 2024-09-11T06:50:26.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001576912, 2024-09-11T06:50:26.500 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:26.500 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:26.500 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:26.500 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:26.501 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:26.501 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:26.501 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 561 2024-09-11T06:50:26.501 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.501 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:26.501 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.501 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:26.501 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.586127+0000", 2024-09-11T06:50:26.501 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.501 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.501 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.501 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:26.501 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.586127+0000", 2024-09-11T06:50:26.501 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.502 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.502 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:26.502 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.586129+0000", 2024-09-11T06:50:26.502 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2699999999999999e-06 2024-09-11T06:50:26.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.502 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.502 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:26.502 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.586140+0000", 2024-09-11T06:50:26.502 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0648999999999999e-05 2024-09-11T06:50:26.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.502 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.502 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:26.502 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.586171+0000", 2024-09-11T06:50:26.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1309000000000001e-05 2024-09-11T06:50:26.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:26.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.586188+0000", 2024-09-11T06:50:26.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.734e-05 2024-09-11T06:50:26.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:26.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.586555+0000", 2024-09-11T06:50:26.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036660600000000001 2024-09-11T06:50:26.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:26.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.586728+0000", 2024-09-11T06:50:26.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000172707 2024-09-11T06:50:26.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.504 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:26.504 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.587704+0000", 2024-09-11T06:50:26.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00097603100000000003 2024-09-11T06:50:26.504 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.504 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:26.504 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.504 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:564 2.1 2:88b29bb7:::rbd_data.1314f0a2bfa2.000000000000000f:head [sparse-read 1052672~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:26.504 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.589945+0000", 2024-09-11T06:50:26.505 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.8714310689999998, 2024-09-11T06:50:26.505 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001765605, 2024-09-11T06:50:26.505 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:26.505 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:26.505 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:26.505 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:26.505 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:26.505 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:26.505 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 564 2024-09-11T06:50:26.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.505 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:26.505 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.505 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:26.505 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.589945+0000", 2024-09-11T06:50:26.506 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.506 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.506 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:26.506 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.589945+0000", 2024-09-11T06:50:26.506 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.506 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.506 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:26.506 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.589947+0000", 2024-09-11T06:50:26.506 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.933e-06 2024-09-11T06:50:26.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.506 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.506 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:26.507 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.589958+0000", 2024-09-11T06:50:26.507 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0866000000000001e-05 2024-09-11T06:50:26.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.507 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:26.507 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.589984+0000", 2024-09-11T06:50:26.507 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6426000000000001e-05 2024-09-11T06:50:26.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.507 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:26.507 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.589997+0000", 2024-09-11T06:50:26.507 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2927e-05 2024-09-11T06:50:26.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:26.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.590631+0000", 2024-09-11T06:50:26.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00063381899999999998 2024-09-11T06:50:26.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:26.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.590791+0000", 2024-09-11T06:50:26.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016030499999999999 2024-09-11T06:50:26.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:26.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.591711+0000", 2024-09-11T06:50:26.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00091932899999999998 2024-09-11T06:50:26.508 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.509 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:26.509 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.509 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:566 2.1 2:88b29bb7:::rbd_data.1314f0a2bfa2.000000000000000f:head [sparse-read 1314816~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:26.509 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.596515+0000", 2024-09-11T06:50:26.509 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.8648610259999998, 2024-09-11T06:50:26.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0017403519999999999, 2024-09-11T06:50:26.509 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:26.509 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:26.509 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:26.509 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:26.509 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:26.509 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 566 2024-09-11T06:50:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.596515+0000", 2024-09-11T06:50:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.596515+0000", 2024-09-11T06:50:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.596516+0000", 2024-09-11T06:50:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.39e-06 2024-09-11T06:50:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.596523+0000", 2024-09-11T06:50:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9820000000000002e-06 2024-09-11T06:50:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.596561+0000", 2024-09-11T06:50:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7996e-05 2024-09-11T06:50:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.596596+0000", 2024-09-11T06:50:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4752000000000002e-05 2024-09-11T06:50:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.597445+0000", 2024-09-11T06:50:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00084871899999999995 2024-09-11T06:50:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.597629+0000", 2024-09-11T06:50:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00018406299999999999 2024-09-11T06:50:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.598255+0000", 2024-09-11T06:50:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00062644999999999997 2024-09-11T06:50:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:570 2.1 2:88b29bb7:::rbd_data.1314f0a2bfa2.000000000000000f:head [sparse-read 1839104~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.601061+0000", 2024-09-11T06:50:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.860315183, 2024-09-11T06:50:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001866014, 2024-09-11T06:50:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 570 2024-09-11T06:50:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.601061+0000", 2024-09-11T06:50:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.601061+0000", 2024-09-11T06:50:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.601062+0000", 2024-09-11T06:50:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.319e-06 2024-09-11T06:50:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.601068+0000", 2024-09-11T06:50:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1430000000000003e-06 2024-09-11T06:50:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.601093+0000", 2024-09-11T06:50:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4726e-05 2024-09-11T06:50:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.601104+0000", 2024-09-11T06:50:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.081e-05 2024-09-11T06:50:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.601703+0000", 2024-09-11T06:50:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00059961400000000005 2024-09-11T06:50:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.601880+0000", 2024-09-11T06:50:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000176904 2024-09-11T06:50:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.602927+0000", 2024-09-11T06:50:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010464980000000001 2024-09-11T06:50:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:572 2.1 2:88b29bb7:::rbd_data.1314f0a2bfa2.000000000000000f:head [sparse-read 2101248~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.603828+0000", 2024-09-11T06:50:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.8575482890000004, 2024-09-11T06:50:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013180749999999999, 2024-09-11T06:50:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 572 2024-09-11T06:50:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.603828+0000", 2024-09-11T06:50:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.603828+0000", 2024-09-11T06:50:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.520 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.520 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.520 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:26.520 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.603829+0000", 2024-09-11T06:50:26.520 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5400000000000001e-06 2024-09-11T06:50:26.520 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.520 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.520 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:26.520 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.603837+0000", 2024-09-11T06:50:26.520 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6399999999999997e-06 2024-09-11T06:50:26.520 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.520 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.520 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:26.520 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.603856+0000", 2024-09-11T06:50:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8719000000000002e-05 2024-09-11T06:50:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.603868+0000", 2024-09-11T06:50:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2476e-05 2024-09-11T06:50:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.604004+0000", 2024-09-11T06:50:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013555899999999999 2024-09-11T06:50:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.604165+0000", 2024-09-11T06:50:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016147700000000001 2024-09-11T06:50:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.605146+0000", 2024-09-11T06:50:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00098066400000000001 2024-09-11T06:50:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:574 2.1 2:88b29bb7:::rbd_data.1314f0a2bfa2.000000000000000f:head [sparse-read 2363392~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.609749+0000", 2024-09-11T06:50:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.8516274160000004, 2024-09-11T06:50:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0024026590000000001, 2024-09-11T06:50:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 574 2024-09-11T06:50:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.609749+0000", 2024-09-11T06:50:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.609749+0000", 2024-09-11T06:50:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.609750+0000", 2024-09-11T06:50:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2640000000000001e-06 2024-09-11T06:50:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.609758+0000", 2024-09-11T06:50:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9929999999999995e-06 2024-09-11T06:50:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.609795+0000", 2024-09-11T06:50:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7410999999999998e-05 2024-09-11T06:50:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.609812+0000", 2024-09-11T06:50:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6396999999999999e-05 2024-09-11T06:50:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.610974+0000", 2024-09-11T06:50:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011627670000000001 2024-09-11T06:50:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.611152+0000", 2024-09-11T06:50:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000178013 2024-09-11T06:50:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.612151+0000", 2024-09-11T06:50:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000998814 2024-09-11T06:50:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:576 2.1 2:88b29bb7:::rbd_data.1314f0a2bfa2.000000000000000f:head [read 2617344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.612142+0000", 2024-09-11T06:50:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.8492341699999999, 2024-09-11T06:50:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001875537, 2024-09-11T06:50:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 576 2024-09-11T06:50:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.612142+0000", 2024-09-11T06:50:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.612142+0000", 2024-09-11T06:50:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.612143+0000", 2024-09-11T06:50:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4699999999999999e-06 2024-09-11T06:50:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.612152+0000", 2024-09-11T06:50:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2989999999999997e-06 2024-09-11T06:50:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.612171+0000", 2024-09-11T06:50:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9313e-05 2024-09-11T06:50:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.612187+0000", 2024-09-11T06:50:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5892e-05 2024-09-11T06:50:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.613414+0000", 2024-09-11T06:50:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012269589999999999 2024-09-11T06:50:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.613567+0000", 2024-09-11T06:50:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000153578 2024-09-11T06:50:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.614017+0000", 2024-09-11T06:50:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00045002600000000002 2024-09-11T06:50:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:577 2.1 2:88b29bb7:::rbd_data.1314f0a2bfa2.000000000000000f:head [sparse-read 2625536~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.613284+0000", 2024-09-11T06:50:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.8480917459999997, 2024-09-11T06:50:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001865591, 2024-09-11T06:50:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 577 2024-09-11T06:50:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.613284+0000", 2024-09-11T06:50:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.613284+0000", 2024-09-11T06:50:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.613287+0000", 2024-09-11T06:50:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7539999999999998e-06 2024-09-11T06:50:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.613299+0000", 2024-09-11T06:50:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1532000000000001e-05 2024-09-11T06:50:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.613323+0000", 2024-09-11T06:50:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.48e-05 2024-09-11T06:50:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.613341+0000", 2024-09-11T06:50:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7618999999999999e-05 2024-09-11T06:50:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.614127+0000", 2024-09-11T06:50:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00078619099999999999 2024-09-11T06:50:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.614269+0000", 2024-09-11T06:50:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014188499999999999 2024-09-11T06:50:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.615150+0000", 2024-09-11T06:50:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00088080999999999999 2024-09-11T06:50:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:578 2.1 2:88b29bb7:::rbd_data.1314f0a2bfa2.000000000000000f:head [sparse-read 2756608~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.614705+0000", 2024-09-11T06:50:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.8466710910000002, 2024-09-11T06:50:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001744542, 2024-09-11T06:50:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 578 2024-09-11T06:50:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.614705+0000", 2024-09-11T06:50:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.614705+0000", 2024-09-11T06:50:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.614707+0000", 2024-09-11T06:50:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1560000000000002e-06 2024-09-11T06:50:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.614718+0000", 2024-09-11T06:50:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0696000000000001e-05 2024-09-11T06:50:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.614746+0000", 2024-09-11T06:50:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8183000000000001e-05 2024-09-11T06:50:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.614783+0000", 2024-09-11T06:50:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6949000000000001e-05 2024-09-11T06:50:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.615266+0000", 2024-09-11T06:50:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00048334499999999999 2024-09-11T06:50:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.615423+0000", 2024-09-11T06:50:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000156587 2024-09-11T06:50:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.616449+0000", 2024-09-11T06:50:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010266260000000001 2024-09-11T06:50:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:583 2.1 2:88b29bb7:::rbd_data.1314f0a2bfa2.000000000000000f:head [sparse-read 3411968~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.624454+0000", 2024-09-11T06:50:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.8369220019999997, 2024-09-11T06:50:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0018395919999999999, 2024-09-11T06:50:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 583 2024-09-11T06:50:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.624454+0000", 2024-09-11T06:50:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.624454+0000", 2024-09-11T06:50:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.624455+0000", 2024-09-11T06:50:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2899999999999999e-06 2024-09-11T06:50:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.624462+0000", 2024-09-11T06:50:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7159999999999997e-06 2024-09-11T06:50:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.624499+0000", 2024-09-11T06:50:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6752000000000003e-05 2024-09-11T06:50:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.624515+0000", 2024-09-11T06:50:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6500000000000001e-05 2024-09-11T06:50:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.625594+0000", 2024-09-11T06:50:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010788729999999999 2024-09-11T06:50:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.625777+0000", 2024-09-11T06:50:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000182583 2024-09-11T06:50:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.626294+0000", 2024-09-11T06:50:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00051687800000000002 2024-09-11T06:50:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:586 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-11T06:50:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:22.227534+0000", 2024-09-11T06:50:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.23384172, 2024-09-11T06:50:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014088219999999999, 2024-09-11T06:50:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 586 2024-09-11T06:50:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.227534+0000", 2024-09-11T06:50:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.227534+0000", 2024-09-11T06:50:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.227535+0000", 2024-09-11T06:50:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8100000000000001e-07 2024-09-11T06:50:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.227540+0000", 2024-09-11T06:50:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4730000000000002e-06 2024-09-11T06:50:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.227552+0000", 2024-09-11T06:50:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2187e-05 2024-09-11T06:50:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.227558+0000", 2024-09-11T06:50:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7860000000000002e-06 2024-09-11T06:50:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.227639+0000", 2024-09-11T06:50:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1668000000000005e-05 2024-09-11T06:50:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.227710+0000", 2024-09-11T06:50:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1056000000000005e-05 2024-09-11T06:50:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.227866+0000", 2024-09-11T06:50:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015520900000000001 2024-09-11T06:50:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.228627+0000", 2024-09-11T06:50:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00076154700000000001 2024-09-11T06:50:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.228865+0000", 2024-09-11T06:50:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000237524 2024-09-11T06:50:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.228897+0000", 2024-09-11T06:50:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2267999999999997e-05 2024-09-11T06:50:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.228943+0000", 2024-09-11T06:50:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6223000000000001e-05 2024-09-11T06:50:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5055.0:74 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch reconnect cookie 140069822272256 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-11T06:50:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:25.640770+0000", 2024-09-11T06:50:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.820606379, 2024-09-11T06:50:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001305959, 2024-09-11T06:50:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5055", 2024-09-11T06:50:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2224097229", 2024-09-11T06:50:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74 2024-09-11T06:50:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:25.640770+0000", 2024-09-11T06:50:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:25.640770+0000", 2024-09-11T06:50:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:25.640771+0000", 2024-09-11T06:50:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1090000000000001e-06 2024-09-11T06:50:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:25.640775+0000", 2024-09-11T06:50:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8929999999999998e-06 2024-09-11T06:50:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:25.640785+0000", 2024-09-11T06:50:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0494e-05 2024-09-11T06:50:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:25.640793+0000", 2024-09-11T06:50:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6680000000000001e-06 2024-09-11T06:50:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:25.640874+0000", 2024-09-11T06:50:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1545999999999995e-05 2024-09-11T06:50:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:25.640954+0000", 2024-09-11T06:50:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0061999999999993e-05 2024-09-11T06:50:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:25.642076+0000", 2024-09-11T06:50:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011211870000000001 2024-09-11T06:50:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:26.555 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:26.557 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:50:26.557 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:26.551+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:26.658 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:26.655+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:26.690 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:26.690 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:26.690 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:50:26.690 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:26.704 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:50:26.759 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:26.760+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:26.820 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:26.821 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:26.821 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:50:26.821 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:50:26.821 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:26.834 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:50:26.876 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:26.876+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:26.961 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:26.959+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:27.001 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:27.001 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:27.001 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:27.001 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:50:27.001 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.002 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:27.002 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:50:27.002 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 80.829724696, 2024-09-11T06:50:27.002 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011420669, 2024-09-11T06:50:27.002 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.002 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.002 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:27.002 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:27.002 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:27.002 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:27.002 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 37 2024-09-11T06:50:27.002 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.002 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.003 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.003 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.003 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:50:27.003 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.003 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.003 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.003 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.003 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:50:27.003 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.003 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.003 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.003 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.004 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170509+0000", 2024-09-11T06:50:27.004 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.13e-06 2024-09-11T06:50:27.004 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.004 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.004 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.004 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170521+0000", 2024-09-11T06:50:27.004 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1776999999999999e-05 2024-09-11T06:50:27.004 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.004 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.004 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.004 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170538+0000", 2024-09-11T06:50:27.005 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7413000000000001e-05 2024-09-11T06:50:27.005 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.005 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.005 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.005 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170570+0000", 2024-09-11T06:50:27.005 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2313e-05 2024-09-11T06:50:27.005 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.005 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.005 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.005 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179768+0000", 2024-09-11T06:50:27.005 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0091978010000000002 2024-09-11T06:50:27.006 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.006 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.006 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.006 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179951+0000", 2024-09-11T06:50:27.006 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018227900000000001 2024-09-11T06:50:27.006 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.006 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.006 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:27.006 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.180175+0000", 2024-09-11T06:50:27.006 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022432800000000001 2024-09-11T06:50:27.007 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.007 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.007 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:27.007 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181112+0000", 2024-09-11T06:50:27.007 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00093712500000000002 2024-09-11T06:50:27.007 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.008 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.008 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:27.008 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181789+0000", 2024-09-11T06:50:27.008 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00067705100000000002 2024-09-11T06:50:27.008 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.008 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.008 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:27.008 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181871+0000", 2024-09-11T06:50:27.009 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1816000000000004e-05 2024-09-11T06:50:27.009 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.009 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.009 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.009 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181929+0000", 2024-09-11T06:50:27.009 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7636000000000002e-05 2024-09-11T06:50:27.009 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.009 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.009 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.009 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.009 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.009 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:27.009 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:27.009 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 80.728465735, 2024-09-11T06:50:27.010 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283336, 2024-09-11T06:50:27.010 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.010 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.010 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:27.010 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:27.010 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:27.010 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:27.010 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 87 2024-09-11T06:50:27.010 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.010 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.010 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.011 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.011 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:27.011 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.011 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.011 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.011 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.011 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:27.011 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.011 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.011 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.011 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.011 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271768+0000", 2024-09-11T06:50:27.011 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7599999999999996e-07 2024-09-11T06:50:27.012 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.012 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.012 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.012 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271860+0000", 2024-09-11T06:50:27.012 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2639999999999994e-05 2024-09-11T06:50:27.012 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.012 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.012 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.012 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271873+0000", 2024-09-11T06:50:27.012 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3139000000000001e-05 2024-09-11T06:50:27.012 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.012 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.012 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.013 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271882+0000", 2024-09-11T06:50:27.013 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2369999999999992e-06 2024-09-11T06:50:27.013 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.013 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.013 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.013 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277150+0000", 2024-09-11T06:50:27.013 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052684419999999999 2024-09-11T06:50:27.013 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.013 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.013 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.013 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277273+0000", 2024-09-11T06:50:27.013 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012278899999999999 2024-09-11T06:50:27.013 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.013 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.014 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:27.014 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277466+0000", 2024-09-11T06:50:27.014 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193578 2024-09-11T06:50:27.014 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.014 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.014 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:27.014 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.280808+0000", 2024-09-11T06:50:27.014 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003341971 2024-09-11T06:50:27.014 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.014 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.014 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:27.014 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284419+0000", 2024-09-11T06:50:27.014 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003610982 2024-09-11T06:50:27.014 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.015 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.015 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:27.015 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284531+0000", 2024-09-11T06:50:27.015 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011121299999999999 2024-09-11T06:50:27.015 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.015 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.015 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.015 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284600+0000", 2024-09-11T06:50:27.015 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9592999999999996e-05 2024-09-11T06:50:27.015 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.015 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.015 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.015 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.015 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.015 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:27.016 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:27.016 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 80.728335482999995, 2024-09-11T06:50:27.016 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283314, 2024-09-11T06:50:27.016 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.016 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.016 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:27.016 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:27.016 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:27.016 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:27.016 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 88 2024-09-11T06:50:27.016 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.017 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.017 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.017 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.017 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:27.017 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.017 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.017 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.017 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.017 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:27.017 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.017 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.017 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.017 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.018 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271898+0000", 2024-09-11T06:50:27.018 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2999999999999999e-07 2024-09-11T06:50:27.018 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.018 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.018 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.018 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271904+0000", 2024-09-11T06:50:27.018 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.767e-06 2024-09-11T06:50:27.018 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.018 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.018 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.018 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271913+0000", 2024-09-11T06:50:27.018 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7200000000000001e-06 2024-09-11T06:50:27.018 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.018 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.019 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.019 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271920+0000", 2024-09-11T06:50:27.019 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7100000000000001e-06 2024-09-11T06:50:27.019 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.019 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.019 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.019 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278684+0000", 2024-09-11T06:50:27.019 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0067642650000000002 2024-09-11T06:50:27.019 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.019 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.019 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.019 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278838+0000", 2024-09-11T06:50:27.019 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015371499999999999 2024-09-11T06:50:27.019 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.020 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.020 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:27.020 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.279040+0000", 2024-09-11T06:50:27.020 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020207499999999999 2024-09-11T06:50:27.020 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.020 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.020 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:27.020 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.281448+0000", 2024-09-11T06:50:27.020 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002407686 2024-09-11T06:50:27.020 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.020 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.020 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:27.021 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284615+0000", 2024-09-11T06:50:27.021 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031675499999999999 2024-09-11T06:50:27.021 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.021 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.021 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:27.021 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284682+0000", 2024-09-11T06:50:27.021 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7032000000000005e-05 2024-09-11T06:50:27.021 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.021 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.021 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.021 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284730+0000", 2024-09-11T06:50:27.021 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7790000000000002e-05 2024-09-11T06:50:27.021 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.022 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.022 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.022 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.022 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.022 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:27.022 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:50:27.022 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 80.560434260999997, 2024-09-11T06:50:27.022 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011114877, 2024-09-11T06:50:27.022 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.022 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.022 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:27.022 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:27.022 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:27.023 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:27.023 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 107 2024-09-11T06:50:27.023 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.023 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.023 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.023 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.023 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:50:27.023 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.023 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.023 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.023 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.023 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:50:27.023 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.023 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.024 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.024 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.024 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439799+0000", 2024-09-11T06:50:27.024 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.16e-07 2024-09-11T06:50:27.024 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.024 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.024 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.024 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439806+0000", 2024-09-11T06:50:27.024 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3640000000000004e-06 2024-09-11T06:50:27.024 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.024 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.024 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.024 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439816+0000", 2024-09-11T06:50:27.024 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0485999999999999e-05 2024-09-11T06:50:27.024 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.025 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.025 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.025 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439824+0000", 2024-09-11T06:50:27.025 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.6699999999999994e-06 2024-09-11T06:50:27.025 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.025 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.025 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.025 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443328+0000", 2024-09-11T06:50:27.025 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035044329999999999 2024-09-11T06:50:27.025 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.025 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.025 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.025 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443461+0000", 2024-09-11T06:50:27.025 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013272199999999999 2024-09-11T06:50:27.026 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.026 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.026 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:27.026 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443678+0000", 2024-09-11T06:50:27.026 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021674500000000001 2024-09-11T06:50:27.026 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.026 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.026 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:27.026 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449635+0000", 2024-09-11T06:50:27.026 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0059572119999999999 2024-09-11T06:50:27.026 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.026 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.026 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:27.026 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450750+0000", 2024-09-11T06:50:27.027 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001115654 2024-09-11T06:50:27.027 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.027 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.027 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:27.027 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450832+0000", 2024-09-11T06:50:27.027 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1188999999999998e-05 2024-09-11T06:50:27.027 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.027 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.027 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.027 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450913+0000", 2024-09-11T06:50:27.027 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1485999999999999e-05 2024-09-11T06:50:27.027 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.027 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.027 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.027 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.028 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.028 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:27.028 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:50:27.028 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 80.559843286000003, 2024-09-11T06:50:27.028 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011236556, 2024-09-11T06:50:27.028 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.028 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.028 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:27.028 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:27.028 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:27.028 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:27.028 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 109 2024-09-11T06:50:27.028 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.028 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.028 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.029 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.029 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:50:27.029 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.029 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.029 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.029 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.029 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:50:27.029 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.029 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.029 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.029 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.029 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440391+0000", 2024-09-11T06:50:27.029 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8360000000000001e-06 2024-09-11T06:50:27.029 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.030 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.030 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.030 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440560+0000", 2024-09-11T06:50:27.030 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000169085 2024-09-11T06:50:27.030 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.030 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.030 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.030 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440585+0000", 2024-09-11T06:50:27.030 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4530000000000001e-05 2024-09-11T06:50:27.030 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.030 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.030 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.030 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440606+0000", 2024-09-11T06:50:27.030 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1277e-05 2024-09-11T06:50:27.031 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.031 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.031 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.031 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446479+0000", 2024-09-11T06:50:27.031 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0058734989999999999 2024-09-11T06:50:27.031 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.031 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.031 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.031 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446649+0000", 2024-09-11T06:50:27.031 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016948199999999999 2024-09-11T06:50:27.031 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.031 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.031 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:27.032 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446854+0000", 2024-09-11T06:50:27.032 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020491500000000001 2024-09-11T06:50:27.032 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.032 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.032 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:27.032 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451380+0000", 2024-09-11T06:50:27.032 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0045264110000000001 2024-09-11T06:50:27.032 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.032 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.032 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:27.032 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451479+0000", 2024-09-11T06:50:27.032 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.8975999999999996e-05 2024-09-11T06:50:27.032 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.032 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.033 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:27.033 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451562+0000", 2024-09-11T06:50:27.033 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3213000000000004e-05 2024-09-11T06:50:27.033 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.033 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.033 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.033 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451626+0000", 2024-09-11T06:50:27.033 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3331999999999996e-05 2024-09-11T06:50:27.033 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.033 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.033 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.033 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.033 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.033 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:27.034 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:50:27.034 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 80.559594157999996, 2024-09-11T06:50:27.034 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011252866, 2024-09-11T06:50:27.034 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.034 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.034 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:27.034 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:27.034 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:27.034 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:27.034 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 110 2024-09-11T06:50:27.034 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.035 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.035 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.035 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.035 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:50:27.035 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.035 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.035 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.035 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.035 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:50:27.035 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.035 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.035 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.035 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.035 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440640+0000", 2024-09-11T06:50:27.036 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5430000000000001e-06 2024-09-11T06:50:27.036 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.036 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.036 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.036 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440651+0000", 2024-09-11T06:50:27.036 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0672999999999999e-05 2024-09-11T06:50:27.036 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.036 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.036 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.036 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440670+0000", 2024-09-11T06:50:27.036 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9726999999999999e-05 2024-09-11T06:50:27.036 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.036 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.036 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.037 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440691+0000", 2024-09-11T06:50:27.037 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0526e-05 2024-09-11T06:50:27.037 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.037 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.037 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.037 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448191+0000", 2024-09-11T06:50:27.037 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0075005990000000002 2024-09-11T06:50:27.037 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.037 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.037 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.037 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448390+0000", 2024-09-11T06:50:27.037 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000198815 2024-09-11T06:50:27.037 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.037 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.038 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:27.038 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448735+0000", 2024-09-11T06:50:27.038 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034424999999999999 2024-09-11T06:50:27.038 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.038 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.038 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:27.038 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451742+0000", 2024-09-11T06:50:27.038 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030077440000000001 2024-09-11T06:50:27.038 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.038 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.038 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:27.038 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451761+0000", 2024-09-11T06:50:27.038 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9196999999999999e-05 2024-09-11T06:50:27.039 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.039 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.039 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:27.039 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451834+0000", 2024-09-11T06:50:27.039 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2071999999999997e-05 2024-09-11T06:50:27.039 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.039 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.039 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.039 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451891+0000", 2024-09-11T06:50:27.039 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7720000000000003e-05 2024-09-11T06:50:27.039 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.039 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.039 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.039 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.040 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.040 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:27.040 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:27.040 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 80.559486282999998, 2024-09-11T06:50:27.040 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014944941, 2024-09-11T06:50:27.040 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.040 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.040 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:27.040 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:27.040 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:27.040 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:27.040 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 111 2024-09-11T06:50:27.040 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.040 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.041 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.041 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.041 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:27.041 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.041 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.041 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.041 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.041 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:27.041 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.041 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.041 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.041 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.041 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440748+0000", 2024-09-11T06:50:27.041 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:50:27.042 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.042 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.042 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.042 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440760+0000", 2024-09-11T06:50:27.042 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1897e-05 2024-09-11T06:50:27.042 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.042 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.042 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.042 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440782+0000", 2024-09-11T06:50:27.042 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2552999999999999e-05 2024-09-11T06:50:27.042 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.042 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.042 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.042 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440793+0000", 2024-09-11T06:50:27.043 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0643e-05 2024-09-11T06:50:27.043 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.043 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.043 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.043 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449755+0000", 2024-09-11T06:50:27.043 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089622620000000004 2024-09-11T06:50:27.043 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.043 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.043 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.043 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449918+0000", 2024-09-11T06:50:27.043 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163086 2024-09-11T06:50:27.043 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.043 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.043 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:27.044 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450150+0000", 2024-09-11T06:50:27.044 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023204500000000001 2024-09-11T06:50:27.044 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.044 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.044 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:27.044 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454533+0000", 2024-09-11T06:50:27.044 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0043831349999999998 2024-09-11T06:50:27.044 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.044 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.044 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:27.044 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455511+0000", 2024-09-11T06:50:27.044 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0009776050000000001 2024-09-11T06:50:27.044 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.044 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.045 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:27.045 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455597+0000", 2024-09-11T06:50:27.045 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6446000000000005e-05 2024-09-11T06:50:27.045 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.045 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.045 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.045 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455691+0000", 2024-09-11T06:50:27.045 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3832000000000006e-05 2024-09-11T06:50:27.045 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.045 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.045 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.045 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.045 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.046 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:27.046 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:27.046 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 80.559416519999999, 2024-09-11T06:50:27.046 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018885411000000001, 2024-09-11T06:50:27.046 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.046 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.046 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:27.046 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:27.046 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:27.046 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:27.046 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 112 2024-09-11T06:50:27.046 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.046 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.046 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.047 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.047 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:27.047 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.047 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.047 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.047 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.047 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:27.047 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.047 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.047 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.047 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.047 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440817+0000", 2024-09-11T06:50:27.047 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:50:27.047 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.048 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.048 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.048 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441003+0000", 2024-09-11T06:50:27.048 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018661200000000001 2024-09-11T06:50:27.048 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.048 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.048 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.048 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441027+0000", 2024-09-11T06:50:27.048 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3920000000000001e-05 2024-09-11T06:50:27.048 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.048 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.048 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.048 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441062+0000", 2024-09-11T06:50:27.048 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4663000000000003e-05 2024-09-11T06:50:27.049 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.049 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.049 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.049 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451981+0000", 2024-09-11T06:50:27.049 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010918926000000001 2024-09-11T06:50:27.049 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.049 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.049 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.049 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452111+0000", 2024-09-11T06:50:27.049 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129529 2024-09-11T06:50:27.049 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.049 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.049 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:27.049 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452311+0000", 2024-09-11T06:50:27.050 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020013799999999999 2024-09-11T06:50:27.050 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.050 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.050 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:27.050 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455809+0000", 2024-09-11T06:50:27.050 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003498443 2024-09-11T06:50:27.050 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.050 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.050 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:27.050 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459566+0000", 2024-09-11T06:50:27.050 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003757271 2024-09-11T06:50:27.050 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.050 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.050 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:27.051 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459640+0000", 2024-09-11T06:50:27.051 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3949000000000006e-05 2024-09-11T06:50:27.051 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.051 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.051 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.051 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459701+0000", 2024-09-11T06:50:27.051 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1117000000000005e-05 2024-09-11T06:50:27.051 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.051 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.051 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.051 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.051 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.051 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:27.051 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:27.052 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 80.559137668999995, 2024-09-11T06:50:27.052 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018760915, 2024-09-11T06:50:27.052 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.052 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.052 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:27.052 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:27.052 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:27.052 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:27.052 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 113 2024-09-11T06:50:27.052 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.052 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.052 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.052 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.052 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:27.053 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.053 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.053 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.053 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.053 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:27.053 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.053 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.053 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.053 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.053 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441096+0000", 2024-09-11T06:50:27.053 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4130000000000001e-06 2024-09-11T06:50:27.053 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.053 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.054 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.054 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441110+0000", 2024-09-11T06:50:27.054 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3900000000000001e-05 2024-09-11T06:50:27.054 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.054 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.054 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.054 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441131+0000", 2024-09-11T06:50:27.054 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0636999999999999e-05 2024-09-11T06:50:27.054 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.054 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.054 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.054 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441147+0000", 2024-09-11T06:50:27.054 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6512999999999999e-05 2024-09-11T06:50:27.054 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.054 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.055 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.055 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453274+0000", 2024-09-11T06:50:27.055 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012126764999999999 2024-09-11T06:50:27.055 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.055 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.055 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.055 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453498+0000", 2024-09-11T06:50:27.055 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000223568 2024-09-11T06:50:27.055 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.055 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.055 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:27.056 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453872+0000", 2024-09-11T06:50:27.056 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037467099999999998 2024-09-11T06:50:27.056 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.056 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.056 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:27.056 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458709+0000", 2024-09-11T06:50:27.056 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0048365680000000003 2024-09-11T06:50:27.056 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.056 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.056 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:27.056 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459716+0000", 2024-09-11T06:50:27.056 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010075240000000001 2024-09-11T06:50:27.056 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.056 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.057 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:27.057 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459779+0000", 2024-09-11T06:50:27.057 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2336999999999996e-05 2024-09-11T06:50:27.057 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.057 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.057 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.057 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459856+0000", 2024-09-11T06:50:27.057 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7019000000000002e-05 2024-09-11T06:50:27.057 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.058 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.058 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.058 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.058 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.058 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:27.058 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:27.058 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 80.559060930000001, 2024-09-11T06:50:27.058 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018963114, 2024-09-11T06:50:27.058 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.058 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.058 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:27.059 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:27.059 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:27.059 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:27.059 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 114 2024-09-11T06:50:27.059 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.059 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.059 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.059 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.059 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:27.059 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.059 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.059 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.059 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.060 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:27.060 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.060 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.060 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.060 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.060 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:27.060 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9299999999999996e-07 2024-09-11T06:50:27.060 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.060 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.060 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.060 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441180+0000", 2024-09-11T06:50:27.060 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7200000000000006e-06 2024-09-11T06:50:27.060 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.061 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.061 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.061 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441265+0000", 2024-09-11T06:50:27.061 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4753000000000007e-05 2024-09-11T06:50:27.061 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.061 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.061 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.061 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441288+0000", 2024-09-11T06:50:27.061 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2860000000000001e-05 2024-09-11T06:50:27.061 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.061 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.061 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.062 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454585+0000", 2024-09-11T06:50:27.062 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013297286 2024-09-11T06:50:27.062 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.062 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.062 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.062 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454714+0000", 2024-09-11T06:50:27.062 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000128465 2024-09-11T06:50:27.062 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.062 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.062 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:27.062 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454909+0000", 2024-09-11T06:50:27.062 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019522499999999999 2024-09-11T06:50:27.063 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.063 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.063 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:27.063 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459868+0000", 2024-09-11T06:50:27.063 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0049591239999999997 2024-09-11T06:50:27.063 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.063 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.063 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:27.063 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459988+0000", 2024-09-11T06:50:27.063 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000120532 2024-09-11T06:50:27.063 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.063 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.063 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:27.064 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460071+0000", 2024-09-11T06:50:27.064 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2256000000000007e-05 2024-09-11T06:50:27.064 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.064 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.064 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.064 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460135+0000", 2024-09-11T06:50:27.064 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3999999999999997e-05 2024-09-11T06:50:27.064 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.064 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.064 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.064 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.064 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.065 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:27.065 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:27.065 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 80.558903301000001, 2024-09-11T06:50:27.065 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.019036849000000002, 2024-09-11T06:50:27.065 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.065 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.065 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:27.065 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:27.065 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:27.065 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:27.065 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 115 2024-09-11T06:50:27.065 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.065 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.065 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.066 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.066 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:27.066 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.066 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.066 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.066 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.066 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:27.066 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.066 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.066 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.066 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.066 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441330+0000", 2024-09-11T06:50:27.066 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1930000000000001e-06 2024-09-11T06:50:27.066 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.067 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.067 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.067 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441544+0000", 2024-09-11T06:50:27.067 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021405799999999999 2024-09-11T06:50:27.067 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.067 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.067 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.067 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441572+0000", 2024-09-11T06:50:27.067 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7317e-05 2024-09-11T06:50:27.067 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.067 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.067 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.068 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441593+0000", 2024-09-11T06:50:27.068 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1702999999999999e-05 2024-09-11T06:50:27.068 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.068 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.068 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.068 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455891+0000", 2024-09-11T06:50:27.068 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01429708 2024-09-11T06:50:27.068 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.068 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.068 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.068 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456046+0000", 2024-09-11T06:50:27.068 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000155682 2024-09-11T06:50:27.068 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.068 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.068 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:27.069 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456242+0000", 2024-09-11T06:50:27.069 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000195692 2024-09-11T06:50:27.069 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.069 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.069 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:27.069 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459876+0000", 2024-09-11T06:50:27.069 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036336480000000002 2024-09-11T06:50:27.069 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.069 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.069 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:27.069 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460245+0000", 2024-09-11T06:50:27.069 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036901700000000002 2024-09-11T06:50:27.069 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.069 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.069 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:27.070 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460316+0000", 2024-09-11T06:50:27.070 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1846000000000003e-05 2024-09-11T06:50:27.070 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.070 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.070 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.070 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460366+0000", 2024-09-11T06:50:27.070 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9613000000000001e-05 2024-09-11T06:50:27.070 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.070 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.070 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.070 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.070 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.070 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:27.070 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:27.070 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 80.558581220999997, 2024-09-11T06:50:27.070 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01904432, 2024-09-11T06:50:27.071 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.071 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.071 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:27.071 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:27.071 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:27.071 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:27.071 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 116 2024-09-11T06:50:27.071 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.071 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.071 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.071 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.071 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:27.071 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.072 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.072 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.072 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.072 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:27.072 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.072 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.072 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.072 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.072 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441653+0000", 2024-09-11T06:50:27.072 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:50:27.072 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.072 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.072 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.073 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441665+0000", 2024-09-11T06:50:27.073 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1653e-05 2024-09-11T06:50:27.073 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.073 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.073 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.073 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441684+0000", 2024-09-11T06:50:27.073 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9490000000000001e-05 2024-09-11T06:50:27.073 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.073 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.073 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.073 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441699+0000", 2024-09-11T06:50:27.073 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5044e-05 2024-09-11T06:50:27.073 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.073 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.074 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.074 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457297+0000", 2024-09-11T06:50:27.074 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015598241000000001 2024-09-11T06:50:27.074 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.074 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.074 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.074 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457577+0000", 2024-09-11T06:50:27.074 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027930799999999998 2024-09-11T06:50:27.074 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.074 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.074 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:27.074 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457957+0000", 2024-09-11T06:50:27.074 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00038088 2024-09-11T06:50:27.075 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.075 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.075 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:27.075 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460432+0000", 2024-09-11T06:50:27.075 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024743349999999998 2024-09-11T06:50:27.075 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.075 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.075 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:27.075 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460584+0000", 2024-09-11T06:50:27.075 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015232900000000001 2024-09-11T06:50:27.075 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.075 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.075 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:27.075 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460646+0000", 2024-09-11T06:50:27.075 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1883000000000004e-05 2024-09-11T06:50:27.076 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.076 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.076 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.076 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460696+0000", 2024-09-11T06:50:27.076 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9589999999999998e-05 2024-09-11T06:50:27.076 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.076 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.076 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.076 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.076 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.076 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:27.076 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:27.076 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 80.558509154999996, 2024-09-11T06:50:27.076 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.021834473, 2024-09-11T06:50:27.076 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.076 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.077 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:27.077 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:27.077 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:27.077 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:27.077 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 117 2024-09-11T06:50:27.077 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.077 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.077 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.077 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.077 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:27.077 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.077 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.078 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.078 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.078 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:27.078 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.078 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.078 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.078 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.078 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441725+0000", 2024-09-11T06:50:27.078 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5069999999999999e-06 2024-09-11T06:50:27.078 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.078 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.078 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.078 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441738+0000", 2024-09-11T06:50:27.078 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2813000000000001e-05 2024-09-11T06:50:27.078 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.079 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.079 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.079 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441759+0000", 2024-09-11T06:50:27.079 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1246999999999998e-05 2024-09-11T06:50:27.079 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.079 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.079 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.079 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441779+0000", 2024-09-11T06:50:27.079 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0392999999999999e-05 2024-09-11T06:50:27.079 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.079 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.079 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.079 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458804+0000", 2024-09-11T06:50:27.079 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.017024182999999998 2024-09-11T06:50:27.080 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.080 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.080 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.080 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458977+0000", 2024-09-11T06:50:27.080 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000173926 2024-09-11T06:50:27.080 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.080 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.080 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:27.080 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459170+0000", 2024-09-11T06:50:27.080 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000192184 2024-09-11T06:50:27.080 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.080 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.080 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:27.080 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.462300+0000", 2024-09-11T06:50:27.081 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031304969999999999 2024-09-11T06:50:27.081 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.081 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.081 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:27.081 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463408+0000", 2024-09-11T06:50:27.081 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011077839999999999 2024-09-11T06:50:27.081 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.081 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.081 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:27.081 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463497+0000", 2024-09-11T06:50:27.081 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8828999999999999e-05 2024-09-11T06:50:27.081 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.081 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.082 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.082 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463558+0000", 2024-09-11T06:50:27.082 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1110000000000003e-05 2024-09-11T06:50:27.082 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.082 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.082 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.082 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.082 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.082 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:27.082 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:50:27.082 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 80.182001467000006, 2024-09-11T06:50:27.082 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012216898, 2024-09-11T06:50:27.082 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.082 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.082 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:27.082 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:27.083 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:27.083 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:27.083 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 137 2024-09-11T06:50:27.083 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.083 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.083 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.083 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.083 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:50:27.083 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.083 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.083 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.083 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.083 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:50:27.083 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.084 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.084 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.084 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.084 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818232+0000", 2024-09-11T06:50:27.084 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2839999999999999e-06 2024-09-11T06:50:27.084 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.084 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.084 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.084 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818326+0000", 2024-09-11T06:50:27.084 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3531999999999998e-05 2024-09-11T06:50:27.084 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.084 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.084 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.085 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818337+0000", 2024-09-11T06:50:27.085 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0655999999999999e-05 2024-09-11T06:50:27.085 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.085 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.085 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.085 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818342+0000", 2024-09-11T06:50:27.085 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4600000000000002e-06 2024-09-11T06:50:27.085 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.085 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.085 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.085 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823890+0000", 2024-09-11T06:50:27.085 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0055480700000000004 2024-09-11T06:50:27.085 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.085 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.085 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.086 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823953+0000", 2024-09-11T06:50:27.086 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2848999999999994e-05 2024-09-11T06:50:27.086 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.086 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.086 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:27.086 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824074+0000", 2024-09-11T06:50:27.086 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121309 2024-09-11T06:50:27.086 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.086 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.086 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:27.086 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828043+0000", 2024-09-11T06:50:27.086 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0039685240000000002 2024-09-11T06:50:27.086 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.086 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.087 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:27.087 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830293+0000", 2024-09-11T06:50:27.087 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0022503179999999998 2024-09-11T06:50:27.087 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.087 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.087 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:27.087 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830366+0000", 2024-09-11T06:50:27.087 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2539999999999993e-05 2024-09-11T06:50:27.087 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.087 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.087 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.087 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830448+0000", 2024-09-11T06:50:27.087 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2355999999999996e-05 2024-09-11T06:50:27.087 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.087 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.088 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.088 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.088 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.088 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:27.088 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:27.088 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 80.181867652999998, 2024-09-11T06:50:27.088 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013136297, 2024-09-11T06:50:27.088 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.088 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.088 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:27.088 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:27.088 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:27.088 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:27.088 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 138 2024-09-11T06:50:27.088 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.089 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.089 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.089 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.089 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:27.089 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.089 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.089 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.089 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.089 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:27.089 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.089 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.090 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.090 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.090 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:27.090 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.89e-07 2024-09-11T06:50:27.090 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.090 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.090 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.090 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818369+0000", 2024-09-11T06:50:27.090 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4170000000000001e-06 2024-09-11T06:50:27.090 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.090 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.090 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.090 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818376+0000", 2024-09-11T06:50:27.090 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4270000000000002e-06 2024-09-11T06:50:27.091 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.091 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.091 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.091 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818383+0000", 2024-09-11T06:50:27.091 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3060000000000003e-06 2024-09-11T06:50:27.091 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.091 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.091 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.091 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824783+0000", 2024-09-11T06:50:27.091 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064003020000000001 2024-09-11T06:50:27.091 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.091 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.091 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.091 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825013+0000", 2024-09-11T06:50:27.091 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023039499999999999 2024-09-11T06:50:27.092 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.092 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.092 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:27.092 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825484+0000", 2024-09-11T06:50:27.092 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000470562 2024-09-11T06:50:27.092 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.092 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.092 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:27.092 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828633+0000", 2024-09-11T06:50:27.092 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031496689999999999 2024-09-11T06:50:27.092 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.092 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.092 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:27.092 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831413+0000", 2024-09-11T06:50:27.092 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027793219999999999 2024-09-11T06:50:27.093 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.093 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.093 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:27.093 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831450+0000", 2024-09-11T06:50:27.093 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7370999999999998e-05 2024-09-11T06:50:27.093 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.093 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.093 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.093 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831501+0000", 2024-09-11T06:50:27.093 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1036999999999999e-05 2024-09-11T06:50:27.093 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.094 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.094 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.094 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.094 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.094 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:27.094 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:27.094 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 80.181843173000004, 2024-09-11T06:50:27.094 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013223011, 2024-09-11T06:50:27.095 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.095 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.095 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:27.095 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:27.095 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:27.095 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:27.095 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 139 2024-09-11T06:50:27.095 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.095 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.096 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.096 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.096 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:27.096 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.096 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.096 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.096 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.096 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:27.096 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.096 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.096 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.096 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.096 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818390+0000", 2024-09-11T06:50:27.096 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5600000000000001e-07 2024-09-11T06:50:27.097 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.097 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.097 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.097 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818394+0000", 2024-09-11T06:50:27.097 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5929999999999999e-06 2024-09-11T06:50:27.097 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.097 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.097 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.097 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818399+0000", 2024-09-11T06:50:27.097 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7740000000000004e-06 2024-09-11T06:50:27.097 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.097 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.097 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.097 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818412+0000", 2024-09-11T06:50:27.098 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2625e-05 2024-09-11T06:50:27.098 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.098 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.098 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.098 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826205+0000", 2024-09-11T06:50:27.098 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0077930730000000002 2024-09-11T06:50:27.098 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.098 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.098 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.098 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826281+0000", 2024-09-11T06:50:27.098 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5876000000000003e-05 2024-09-11T06:50:27.098 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.098 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.098 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:27.098 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826384+0000", 2024-09-11T06:50:27.098 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103838 2024-09-11T06:50:27.099 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.099 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.099 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:27.099 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830642+0000", 2024-09-11T06:50:27.099 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004257083 2024-09-11T06:50:27.099 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.099 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.099 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:27.099 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831509+0000", 2024-09-11T06:50:27.099 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086730900000000003 2024-09-11T06:50:27.099 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.099 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.099 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:27.099 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831547+0000", 2024-09-11T06:50:27.099 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7793000000000002e-05 2024-09-11T06:50:27.100 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.100 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.100 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.100 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831612+0000", 2024-09-11T06:50:27.100 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5691000000000005e-05 2024-09-11T06:50:27.100 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.100 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.100 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.100 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.100 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.100 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:27.100 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:27.101 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 80.181814478999996, 2024-09-11T06:50:27.101 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016098024999999998, 2024-09-11T06:50:27.101 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.101 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.101 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:27.101 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:27.101 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:27.101 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:27.101 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 140 2024-09-11T06:50:27.101 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.101 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.101 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.101 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.101 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:27.101 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.102 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.102 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.102 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.102 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:27.102 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.102 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.102 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.102 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.102 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:27.102 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5199999999999998e-07 2024-09-11T06:50:27.102 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.102 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.102 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.102 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818479+0000", 2024-09-11T06:50:27.103 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0368000000000002e-05 2024-09-11T06:50:27.103 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.103 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.103 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.103 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818487+0000", 2024-09-11T06:50:27.103 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2339999999999994e-06 2024-09-11T06:50:27.103 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.103 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.103 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.103 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818491+0000", 2024-09-11T06:50:27.103 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4329999999999997e-06 2024-09-11T06:50:27.103 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.103 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.103 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.104 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828080+0000", 2024-09-11T06:50:27.104 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0095890449999999992 2024-09-11T06:50:27.104 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.104 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.104 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.104 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828130+0000", 2024-09-11T06:50:27.104 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9239000000000001e-05 2024-09-11T06:50:27.104 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.104 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.104 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:27.104 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828208+0000", 2024-09-11T06:50:27.105 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.852e-05 2024-09-11T06:50:27.105 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.105 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.105 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:27.105 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831625+0000", 2024-09-11T06:50:27.105 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034169410000000002 2024-09-11T06:50:27.105 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.105 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.105 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:27.105 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834411+0000", 2024-09-11T06:50:27.105 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002786111 2024-09-11T06:50:27.105 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.106 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.106 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:27.106 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834457+0000", 2024-09-11T06:50:27.106 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6041000000000002e-05 2024-09-11T06:50:27.106 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.106 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.106 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.106 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834516+0000", 2024-09-11T06:50:27.106 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8740999999999999e-05 2024-09-11T06:50:27.106 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.106 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.106 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.106 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.107 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.107 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:27.107 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:27.107 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 80.181733412, 2024-09-11T06:50:27.107 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016588772000000002, 2024-09-11T06:50:27.107 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.107 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.107 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:27.107 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:27.107 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:27.107 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:27.107 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 141 2024-09-11T06:50:27.107 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.107 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.108 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.108 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.108 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:27.108 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.108 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.108 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.108 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.108 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:27.108 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.108 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.108 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.108 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.108 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818500+0000", 2024-09-11T06:50:27.109 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0800000000000005e-07 2024-09-11T06:50:27.109 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.109 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.109 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.109 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818505+0000", 2024-09-11T06:50:27.109 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2429999999999996e-06 2024-09-11T06:50:27.109 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.109 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.109 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.109 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818510+0000", 2024-09-11T06:50:27.109 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9339999999999997e-06 2024-09-11T06:50:27.109 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.110 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.110 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.110 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818514+0000", 2024-09-11T06:50:27.110 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7270000000000001e-06 2024-09-11T06:50:27.110 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.110 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.110 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.110 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828711+0000", 2024-09-11T06:50:27.110 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010197031 2024-09-11T06:50:27.110 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.110 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.110 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.111 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828853+0000", 2024-09-11T06:50:27.111 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014201200000000001 2024-09-11T06:50:27.111 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.111 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.111 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:27.111 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.829115+0000", 2024-09-11T06:50:27.111 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026257500000000003 2024-09-11T06:50:27.111 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.111 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.111 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:27.111 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831629+0000", 2024-09-11T06:50:27.111 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025138169999999998 2024-09-11T06:50:27.111 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.112 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.112 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:27.112 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834991+0000", 2024-09-11T06:50:27.112 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033624459999999998 2024-09-11T06:50:27.112 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.112 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.112 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:27.112 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835032+0000", 2024-09-11T06:50:27.112 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1001000000000003e-05 2024-09-11T06:50:27.112 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.112 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.112 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.112 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835088+0000", 2024-09-11T06:50:27.113 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5478e-05 2024-09-11T06:50:27.113 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.113 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.113 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.113 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.113 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.113 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:50:27.113 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:50:27.113 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 79.558098509000004, 2024-09-11T06:50:27.113 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011161912, 2024-09-11T06:50:27.113 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.113 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.113 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:27.113 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:27.114 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:27.114 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:27.114 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 158 2024-09-11T06:50:27.114 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.114 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.114 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.114 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.114 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:50:27.114 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.114 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.114 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.114 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.114 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:50:27.114 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.115 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.115 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.115 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.115 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442135+0000", 2024-09-11T06:50:27.115 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3e-07 2024-09-11T06:50:27.115 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.115 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.115 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.115 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442259+0000", 2024-09-11T06:50:27.115 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000123903 2024-09-11T06:50:27.115 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.115 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.115 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.115 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442289+0000", 2024-09-11T06:50:27.116 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0833000000000001e-05 2024-09-11T06:50:27.116 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.116 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.116 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.116 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442301+0000", 2024-09-11T06:50:27.116 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1433e-05 2024-09-11T06:50:27.116 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.116 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.116 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.116 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449796+0000", 2024-09-11T06:50:27.116 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0074954959999999999 2024-09-11T06:50:27.116 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.116 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.116 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.117 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449917+0000", 2024-09-11T06:50:27.117 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121033 2024-09-11T06:50:27.117 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.117 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.117 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:27.117 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.450166+0000", 2024-09-11T06:50:27.117 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00024832400000000002 2024-09-11T06:50:27.117 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.117 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.117 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:27.117 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.452630+0000", 2024-09-11T06:50:27.118 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002464188 2024-09-11T06:50:27.118 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.118 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.118 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:27.118 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453149+0000", 2024-09-11T06:50:27.118 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00051860999999999997 2024-09-11T06:50:27.118 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.118 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.118 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:27.118 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453234+0000", 2024-09-11T06:50:27.118 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.5865000000000005e-05 2024-09-11T06:50:27.118 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.118 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.119 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.119 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453296+0000", 2024-09-11T06:50:27.119 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1496999999999998e-05 2024-09-11T06:50:27.119 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.119 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.119 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.119 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.119 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.119 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:50:27.119 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:50:27.119 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 79.557912407000003, 2024-09-11T06:50:27.119 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011561900999999999, 2024-09-11T06:50:27.119 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.119 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.120 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:27.120 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:27.120 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:27.120 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:27.120 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 159 2024-09-11T06:50:27.120 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.120 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.120 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.120 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.120 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:50:27.120 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.121 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.121 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.121 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.121 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:50:27.121 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.121 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.121 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.121 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.121 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442321+0000", 2024-09-11T06:50:27.121 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8400000000000003e-07 2024-09-11T06:50:27.121 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.121 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.121 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.122 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442328+0000", 2024-09-11T06:50:27.122 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5130000000000002e-06 2024-09-11T06:50:27.122 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.122 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.122 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.122 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442341+0000", 2024-09-11T06:50:27.122 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.363e-05 2024-09-11T06:50:27.122 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.122 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.122 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.122 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442349+0000", 2024-09-11T06:50:27.122 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4760000000000001e-06 2024-09-11T06:50:27.122 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.122 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.123 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.123 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451257+0000", 2024-09-11T06:50:27.123 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089088569999999992 2024-09-11T06:50:27.123 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.123 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.123 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.123 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451389+0000", 2024-09-11T06:50:27.123 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013110899999999999 2024-09-11T06:50:27.123 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.123 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.123 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:27.123 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451608+0000", 2024-09-11T06:50:27.123 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000219588 2024-09-11T06:50:27.123 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.124 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.124 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:27.124 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453517+0000", 2024-09-11T06:50:27.124 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0019088779999999999 2024-09-11T06:50:27.124 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.124 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.124 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:27.124 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453727+0000", 2024-09-11T06:50:27.124 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020955100000000001 2024-09-11T06:50:27.124 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.124 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.124 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:27.124 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453815+0000", 2024-09-11T06:50:27.124 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8858999999999997e-05 2024-09-11T06:50:27.125 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.125 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.125 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.125 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453882+0000", 2024-09-11T06:50:27.125 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.6556000000000004e-05 2024-09-11T06:50:27.125 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.125 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.125 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.125 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.125 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.125 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:27.127 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:50:27.127 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:27.123+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:27.227 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:27.223+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:27.230 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5, 1] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T06:50:27.230 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T06:50:27.230 INFO:tasks.thrashosds.thrasher:Removing osd 5, in_osds are: [0, 2, 4, 5, 1] 2024-09-11T06:50:27.230 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 5 2024-09-11T06:50:27.299 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:27.299 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:27.299 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:50:27.299 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:27.314 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:50:27.328 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:27.323+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:27.428 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:27.427+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:27.438 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:27.438 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:27.438 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:50:27.438 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:50:27.438 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:27.452 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:50:27.518 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:50:27.539 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:27.539+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:27.552 INFO:tasks.ceph.mon.a.smithi071.stderr:2024-09-11T06:50:27.547+0000 7f1296554640 -1 mon.a@0(leader).osd e65 definitely_dead 0 2024-09-11T06:50:27.624 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:27.624 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:27.624 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:27.625 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:50:27.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.625 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4558.0:1 3.3 3:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-11T06:50:27.625 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:50:27.625 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 10.532333563, 2024-09-11T06:50:27.625 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0085520600000000002, 2024-09-11T06:50:27.625 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.625 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.625 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:27.625 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:27.625 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4558", 2024-09-11T06:50:27.625 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1478388898", 2024-09-11T06:50:27.625 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:50:27.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.626 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.626 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.626 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.626 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:50:27.626 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.626 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.626 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.626 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:50:27.626 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.626 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.626 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.626 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090533+0000", 2024-09-11T06:50:27.627 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2629999999999999e-06 2024-09-11T06:50:27.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.627 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.627 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090546+0000", 2024-09-11T06:50:27.627 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2918999999999999e-05 2024-09-11T06:50:27.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.627 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.627 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090572+0000", 2024-09-11T06:50:27.627 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6412000000000001e-05 2024-09-11T06:50:27.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.627 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.627 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090594+0000", 2024-09-11T06:50:27.628 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2476e-05 2024-09-11T06:50:27.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.628 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.628 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.628 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090938+0000", 2024-09-11T06:50:27.628 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034325099999999998 2024-09-11T06:50:27.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.628 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.628 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.628 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.091149+0000", 2024-09-11T06:50:27.628 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021151300000000001 2024-09-11T06:50:27.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.628 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.628 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:27.628 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.091377+0000", 2024-09-11T06:50:27.629 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022785399999999999 2024-09-11T06:50:27.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.629 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.629 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:27.629 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.092912+0000", 2024-09-11T06:50:27.629 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0015346089999999999 2024-09-11T06:50:27.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.629 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.629 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:27.629 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098919+0000", 2024-09-11T06:50:27.629 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0060076799999999996 2024-09-11T06:50:27.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.629 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.629 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:27.630 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.099009+0000", 2024-09-11T06:50:27.630 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0056000000000006e-05 2024-09-11T06:50:27.630 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.630 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.630 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.630 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.099081+0000", 2024-09-11T06:50:27.630 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2026999999999993e-05 2024-09-11T06:50:27.630 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.630 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.630 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.630 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.630 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.630 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop_reply(client.4558.0:1 3.3 e56/55 ondisk, result = 0)", 2024-09-11T06:50:27.630 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:17.098705+0000", 2024-09-11T06:50:27.630 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 10.524158409, 2024-09-11T06:50:27.630 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00039016499999999999, 2024-09-11T06:50:27.631 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.631 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.631 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:27.631 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.631 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.631 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098705+0000", 2024-09-11T06:50:27.631 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.631 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.631 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098705+0000", 2024-09-11T06:50:27.631 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.632 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.632 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098706+0000", 2024-09-11T06:50:27.632 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.626e-06 2024-09-11T06:50:27.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.632 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.632 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098717+0000", 2024-09-11T06:50:27.632 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0699e-05 2024-09-11T06:50:27.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.632 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.632 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098743+0000", 2024-09-11T06:50:27.632 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6449e-05 2024-09-11T06:50:27.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.633 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.633 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098759+0000", 2024-09-11T06:50:27.633 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5815999999999999e-05 2024-09-11T06:50:27.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.633 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.633 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098894+0000", 2024-09-11T06:50:27.633 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013442099999999999 2024-09-11T06:50:27.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.633 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.634 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098910+0000", 2024-09-11T06:50:27.634 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6696000000000001e-05 2024-09-11T06:50:27.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.634 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.634 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.634 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.099095+0000", 2024-09-11T06:50:27.634 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000184458 2024-09-11T06:50:27.634 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.634 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.634 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.634 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.634 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:50:27.634 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:50:27.634 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.4908444830000001, 2024-09-11T06:50:27.635 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0071924390000000001, 2024-09-11T06:50:27.635 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.635 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.635 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:27.635 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.635 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.635 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.635 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:50:27.635 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.635 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.635 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.635 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:50:27.636 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.636 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.636 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.636 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132020+0000", 2024-09-11T06:50:27.636 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.846e-06 2024-09-11T06:50:27.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.636 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.636 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.636 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135688+0000", 2024-09-11T06:50:27.636 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036677350000000001 2024-09-11T06:50:27.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.636 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.637 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.637 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135747+0000", 2024-09-11T06:50:27.637 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.9314999999999997e-05 2024-09-11T06:50:27.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.637 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.637 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135797+0000", 2024-09-11T06:50:27.637 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9984000000000001e-05 2024-09-11T06:50:27.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.637 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.637 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136143+0000", 2024-09-11T06:50:27.637 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034594299999999998 2024-09-11T06:50:27.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.638 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.638 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136179+0000", 2024-09-11T06:50:27.638 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5636000000000002e-05 2024-09-11T06:50:27.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.638 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.638 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.139211+0000", 2024-09-11T06:50:27.638 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030319800000000001 2024-09-11T06:50:27.638 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.638 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.638 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.639 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:50:27.639 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:50:27.639 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.4887812289999998, 2024-09-11T06:50:27.639 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064954139999999997, 2024-09-11T06:50:27.639 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.639 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.639 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:27.639 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.639 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.639 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:50:27.639 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.639 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.640 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:50:27.640 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.640 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.640 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.640 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134084+0000", 2024-09-11T06:50:27.640 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.894e-06 2024-09-11T06:50:27.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.640 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.640 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.640 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136513+0000", 2024-09-11T06:50:27.640 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024290969999999999 2024-09-11T06:50:27.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.641 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.641 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136554+0000", 2024-09-11T06:50:27.641 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.0932e-05 2024-09-11T06:50:27.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.641 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.641 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136584+0000", 2024-09-11T06:50:27.641 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0056000000000002e-05 2024-09-11T06:50:27.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.641 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.641 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136805+0000", 2024-09-11T06:50:27.641 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022086100000000001 2024-09-11T06:50:27.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.642 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.642 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136864+0000", 2024-09-11T06:50:27.642 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.9907000000000001e-05 2024-09-11T06:50:27.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.642 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.642 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.140577+0000", 2024-09-11T06:50:27.642 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0037126669999999998 2024-09-11T06:50:27.642 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.642 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.642 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.643 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:50:27.643 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:50:27.643 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.4804904289999996, 2024-09-11T06:50:27.643 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035763180000000002, 2024-09-11T06:50:27.643 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.643 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.643 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:27.643 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.643 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.643 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:50:27.643 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.643 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.644 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:50:27.644 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.644 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.644 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142374+0000", 2024-09-11T06:50:27.644 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5460000000000001e-06 2024-09-11T06:50:27.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.644 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.644 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143902+0000", 2024-09-11T06:50:27.644 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00152808 2024-09-11T06:50:27.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.644 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.645 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143946+0000", 2024-09-11T06:50:27.645 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3874e-05 2024-09-11T06:50:27.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.645 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143978+0000", 2024-09-11T06:50:27.645 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2425999999999998e-05 2024-09-11T06:50:27.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.645 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.144177+0000", 2024-09-11T06:50:27.645 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019837500000000001 2024-09-11T06:50:27.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.646 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.646 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.144198+0000", 2024-09-11T06:50:27.646 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1639000000000001e-05 2024-09-11T06:50:27.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.646 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.646 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.145949+0000", 2024-09-11T06:50:27.646 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0017503779999999999 2024-09-11T06:50:27.646 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.646 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.646 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.646 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:50:27.646 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:50:27.647 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.4761747270000001, 2024-09-11T06:50:27.647 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0057693220000000003, 2024-09-11T06:50:27.647 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.647 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.647 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:27.647 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.647 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.647 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:50:27.647 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.647 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.647 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:50:27.648 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.648 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.648 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146690+0000", 2024-09-11T06:50:27.648 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.607e-06 2024-09-11T06:50:27.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.648 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.648 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148693+0000", 2024-09-11T06:50:27.648 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0020029739999999998 2024-09-11T06:50:27.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.648 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.649 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148735+0000", 2024-09-11T06:50:27.649 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1958e-05 2024-09-11T06:50:27.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.649 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.649 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148764+0000", 2024-09-11T06:50:27.649 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8999000000000001e-05 2024-09-11T06:50:27.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.649 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.649 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149069+0000", 2024-09-11T06:50:27.649 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000305032 2024-09-11T06:50:27.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.650 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.650 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149204+0000", 2024-09-11T06:50:27.650 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013495 2024-09-11T06:50:27.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.650 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.650 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.152458+0000", 2024-09-11T06:50:27.650 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0032538020000000001 2024-09-11T06:50:27.650 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.650 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.650 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.650 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPushReply(2.7 61/60 [PushReplyOp(2:eab0161e:::rbd_data.1314f0a2bfa2.0000000000000001:head)])", 2024-09-11T06:50:27.651 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.147351+0000", 2024-09-11T06:50:27.651 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.4755122099999998, 2024-09-11T06:50:27.651 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0026083040000000001, 2024-09-11T06:50:27.651 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.651 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.651 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:27.651 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.651 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.651 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147351+0000", 2024-09-11T06:50:27.651 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.651 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.651 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147351+0000", 2024-09-11T06:50:27.652 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.652 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.652 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147352+0000", 2024-09-11T06:50:27.652 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.593e-06 2024-09-11T06:50:27.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.652 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.652 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147363+0000", 2024-09-11T06:50:27.652 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.111e-05 2024-09-11T06:50:27.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.652 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.652 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147385+0000", 2024-09-11T06:50:27.653 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1532e-05 2024-09-11T06:50:27.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.653 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.653 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147399+0000", 2024-09-11T06:50:27.653 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.448e-05 2024-09-11T06:50:27.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.653 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.653 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147572+0000", 2024-09-11T06:50:27.653 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000172423 2024-09-11T06:50:27.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.654 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.654 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149959+0000", 2024-09-11T06:50:27.654 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002387166 2024-09-11T06:50:27.654 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.654 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.654 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.654 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:50:27.654 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:50:27.654 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.4752761860000003, 2024-09-11T06:50:27.654 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0042425329999999997, 2024-09-11T06:50:27.654 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.654 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.654 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:27.655 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.655 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.655 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:50:27.655 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.655 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.655 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:50:27.655 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.655 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.655 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147588+0000", 2024-09-11T06:50:27.655 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.277e-06 2024-09-11T06:50:27.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.656 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.656 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149051+0000", 2024-09-11T06:50:27.656 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001462669 2024-09-11T06:50:27.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.656 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.656 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149090+0000", 2024-09-11T06:50:27.656 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.8967999999999997e-05 2024-09-11T06:50:27.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.656 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.656 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149120+0000", 2024-09-11T06:50:27.656 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0766000000000003e-05 2024-09-11T06:50:27.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.657 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.657 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149395+0000", 2024-09-11T06:50:27.657 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027484900000000002 2024-09-11T06:50:27.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.657 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.657 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149439+0000", 2024-09-11T06:50:27.657 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3940999999999998e-05 2024-09-11T06:50:27.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.657 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.658 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.151829+0000", 2024-09-11T06:50:27.658 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002390063 2024-09-11T06:50:27.658 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.658 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.658 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.658 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPushReply(2.7 61/60 [PushReplyOp(2:eab0161e:::rbd_data.1314f0a2bfa2.0000000000000001:head)])", 2024-09-11T06:50:27.658 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.152951+0000", 2024-09-11T06:50:27.658 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.469911765, 2024-09-11T06:50:27.658 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002099357, 2024-09-11T06:50:27.658 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.658 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.659 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:27.659 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.659 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.659 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.152951+0000", 2024-09-11T06:50:27.659 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.659 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.659 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.152951+0000", 2024-09-11T06:50:27.659 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.660 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.660 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.152953+0000", 2024-09-11T06:50:27.660 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.854e-06 2024-09-11T06:50:27.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.660 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.660 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.152968+0000", 2024-09-11T06:50:27.660 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5096000000000001e-05 2024-09-11T06:50:27.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.660 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.660 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.152995+0000", 2024-09-11T06:50:27.660 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6432e-05 2024-09-11T06:50:27.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.661 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.661 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.661 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153015+0000", 2024-09-11T06:50:27.661 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0679999999999999e-05 2024-09-11T06:50:27.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.661 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.661 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.661 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153197+0000", 2024-09-11T06:50:27.661 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018195599999999999 2024-09-11T06:50:27.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.661 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.661 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.662 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155051+0000", 2024-09-11T06:50:27.662 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0018533390000000001 2024-09-11T06:50:27.662 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.662 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.662 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.662 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.662 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:50:27.662 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:50:27.662 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.4691355550000003, 2024-09-11T06:50:27.662 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0046568219999999997, 2024-09-11T06:50:27.662 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.662 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.663 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:27.663 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.663 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.663 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:50:27.663 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.663 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.663 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:50:27.663 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.663 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.664 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153729+0000", 2024-09-11T06:50:27.664 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5629999999999999e-06 2024-09-11T06:50:27.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.664 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.664 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.664 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155179+0000", 2024-09-11T06:50:27.664 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001450469 2024-09-11T06:50:27.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.664 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.664 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.664 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155221+0000", 2024-09-11T06:50:27.664 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1448e-05 2024-09-11T06:50:27.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.665 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.665 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.665 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155249+0000", 2024-09-11T06:50:27.665 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8433e-05 2024-09-11T06:50:27.665 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.665 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.665 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.665 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155420+0000", 2024-09-11T06:50:27.665 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017070599999999999 2024-09-11T06:50:27.665 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.665 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.665 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.665 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155445+0000", 2024-09-11T06:50:27.666 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4813000000000001e-05 2024-09-11T06:50:27.666 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.666 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.666 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.666 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.158384+0000", 2024-09-11T06:50:27.666 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0029393900000000001 2024-09-11T06:50:27.666 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.666 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.666 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.666 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.666 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.666 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:50:27.666 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:50:27.666 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.4671088059999997, 2024-09-11T06:50:27.667 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0046826020000000001, 2024-09-11T06:50:27.667 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.667 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.667 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:27.667 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.667 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.667 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.667 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:50:27.667 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.667 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.667 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.667 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:50:27.668 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.668 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.668 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155756+0000", 2024-09-11T06:50:27.668 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.384e-06 2024-09-11T06:50:27.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.668 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.668 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157205+0000", 2024-09-11T06:50:27.668 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014496260000000001 2024-09-11T06:50:27.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.668 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.669 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157252+0000", 2024-09-11T06:50:27.669 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6368000000000001e-05 2024-09-11T06:50:27.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.669 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.669 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.669 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157286+0000", 2024-09-11T06:50:27.669 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4944999999999998e-05 2024-09-11T06:50:27.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.669 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.669 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.669 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157540+0000", 2024-09-11T06:50:27.669 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00025307999999999998 2024-09-11T06:50:27.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.670 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.670 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157576+0000", 2024-09-11T06:50:27.670 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.6878999999999997e-05 2024-09-11T06:50:27.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.670 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.670 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.160437+0000", 2024-09-11T06:50:27.670 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00286032 2024-09-11T06:50:27.670 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.670 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.670 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.671 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.671 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPushReply(2.7 61/60 [PushReplyOp(2:eab0161e:::rbd_data.1314f0a2bfa2.0000000000000001:head)])", 2024-09-11T06:50:27.671 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.158551+0000", 2024-09-11T06:50:27.671 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.464311736, 2024-09-11T06:50:27.671 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0023625930000000001, 2024-09-11T06:50:27.671 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.671 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.671 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:27.671 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.671 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.671 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.671 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.158551+0000", 2024-09-11T06:50:27.671 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.672 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.672 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.672 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.158551+0000", 2024-09-11T06:50:27.672 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.672 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.672 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.672 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.158553+0000", 2024-09-11T06:50:27.672 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9690000000000001e-06 2024-09-11T06:50:27.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.672 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.672 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.672 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.158566+0000", 2024-09-11T06:50:27.673 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.294e-05 2024-09-11T06:50:27.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.673 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.673 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.158603+0000", 2024-09-11T06:50:27.673 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7202000000000001e-05 2024-09-11T06:50:27.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.673 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.673 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.158622+0000", 2024-09-11T06:50:27.673 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8335999999999999e-05 2024-09-11T06:50:27.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.673 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.674 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.158767+0000", 2024-09-11T06:50:27.674 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000145544 2024-09-11T06:50:27.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.674 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.674 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.674 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.160914+0000", 2024-09-11T06:50:27.674 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021466020000000001 2024-09-11T06:50:27.674 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.674 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.674 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.674 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.674 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:774f7988:ns::rbd_directory:head, version: 54'3, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 3, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:774f7988:ns::rbd_directory:head@54'3, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:50:27.674 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.160495+0000", 2024-09-11T06:50:27.674 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.462368176, 2024-09-11T06:50:27.675 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012141890000000001, 2024-09-11T06:50:27.675 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.675 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.675 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:27.675 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.675 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.675 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.675 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.160495+0000", 2024-09-11T06:50:27.675 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.675 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.675 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.675 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.160495+0000", 2024-09-11T06:50:27.675 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.676 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.676 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.160496+0000", 2024-09-11T06:50:27.676 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.483e-06 2024-09-11T06:50:27.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.676 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.676 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.160533+0000", 2024-09-11T06:50:27.676 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.6641999999999999e-05 2024-09-11T06:50:27.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.676 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.676 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.160590+0000", 2024-09-11T06:50:27.677 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.6774000000000002e-05 2024-09-11T06:50:27.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.677 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.677 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.160663+0000", 2024-09-11T06:50:27.677 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3233999999999997e-05 2024-09-11T06:50:27.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.677 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.677 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.160878+0000", 2024-09-11T06:50:27.677 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021538500000000001 2024-09-11T06:50:27.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.678 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.678 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.160918+0000", 2024-09-11T06:50:27.678 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.9715000000000002e-05 2024-09-11T06:50:27.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.678 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.678 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.678 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.161709+0000", 2024-09-11T06:50:27.678 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00079095599999999999 2024-09-11T06:50:27.678 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.678 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.678 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.678 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.678 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5055.0:93 2.7 2:eab0161e:::rbd_data.1314f0a2bfa2.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-11T06:50:27.678 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:25.654406+0000", 2024-09-11T06:50:27.678 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 1.9684567290000001, 2024-09-11T06:50:27.679 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00077837000000000002, 2024-09-11T06:50:27.679 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:27.679 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:27.679 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:27.679 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:27.679 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5055", 2024-09-11T06:50:27.679 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2224097229", 2024-09-11T06:50:27.679 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 93 2024-09-11T06:50:27.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.679 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:27.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.679 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:27.679 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:25.654406+0000", 2024-09-11T06:50:27.679 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.680 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:27.680 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:25.654406+0000", 2024-09-11T06:50:27.680 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:27.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.680 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:27.680 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:25.654409+0000", 2024-09-11T06:50:27.680 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4260000000000002e-06 2024-09-11T06:50:27.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.680 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:27.680 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:25.654423+0000", 2024-09-11T06:50:27.681 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4433000000000001e-05 2024-09-11T06:50:27.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.681 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:27.681 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:25.654454+0000", 2024-09-11T06:50:27.681 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.1375999999999999e-05 2024-09-11T06:50:27.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.681 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:27.681 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:25.654470+0000", 2024-09-11T06:50:27.681 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5818999999999999e-05 2024-09-11T06:50:27.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.682 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:27.682 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:25.654621+0000", 2024-09-11T06:50:27.682 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015119100000000001 2024-09-11T06:50:27.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.682 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:27.682 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:25.654859+0000", 2024-09-11T06:50:27.682 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000237307 2024-09-11T06:50:27.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:27.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:27.682 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:27.682 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:25.655185+0000", 2024-09-11T06:50:27.682 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00032581800000000002 2024-09-11T06:50:27.682 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.682 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.682 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.683 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:27.683 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:27.683 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:27.684 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:50:27.684 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:50:27.685 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:50:27.685 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:50:27.685 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:50:27.685 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:27.683+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:27.692 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:50:27.692 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:50:27.713 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:50:27.713 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:50:27.716 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd -p rbd --format xml du testrbdnbd21607 2024-09-11T06:50:27.717 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -m //stats/images/image/provisioned_size -v . 2024-09-11T06:50:27.734 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:50:27.734 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:50:27.755 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:50:27.755 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:50:27.780 INFO:tasks.workunit.client.0.smithi176.stderr:+ provisioned=67108864 2024-09-11T06:50:27.780 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd -p rbd --format xml du testrbdnbd21607 2024-09-11T06:50:27.781 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -m //stats/images/image/used_size -v . 2024-09-11T06:50:27.817 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:27.815+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:27.838 INFO:tasks.workunit.client.0.smithi176.stderr:+ used=67108864 2024-09-11T06:50:27.838 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 67108864 -eq 67108864 ']' 2024-09-11T06:50:27.838 INFO:tasks.workunit.client.0.smithi176.stderr:+ unmap_device /dev/nbd0 22121 2024-09-11T06:50:27.838 INFO:tasks.workunit.client.0.smithi176.stderr:+ local args=/dev/nbd0 2024-09-11T06:50:27.838 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pid=22121 2024-09-11T06:50:27.838 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd0 2024-09-11T06:50:27.838 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:27.838 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:27.839 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:27.839 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:50:27.840 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:50:27.840 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:27.840 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd0 2024-09-11T06:50:27.855 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:27.855 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:27.869 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:50:27.885 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:27.883+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:27.916 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:50:27.917 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^22121\b' 2024-09-11T06:50:27.917 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^22121\b' 2024-09-11T06:50:27.963 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:27.964 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -C rbd-nbd 2024-09-11T06:50:27.964 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^ *22121\b' 2024-09-11T06:50:27.964 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^ *22121\b' 2024-09-11T06:50:27.970 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:27.970 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 0.5 2024-09-11T06:50:27.987 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:27.983+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:27.992 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:27.992 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:28.006 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:50:28.088 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:28.087+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:28.128 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:28.129 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:28.143 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:50:28.160 INFO:teuthology.orchestra.run.smithi071.stderr:marked out osd.5. 2024-09-11T06:50:28.174 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T06:50:28.189 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:28.187+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:28.272 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:28.272 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:28.287 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:50:28.289 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:28.287+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:28.390 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:28.387+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:28.404 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:28.404 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:28.418 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:50:28.472 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo rbd device --device-type nbd map rbd/testrbdnbd21607 2024-09-11T06:50:28.472 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:50:28.472 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:50:28.473 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:50:28.474 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which rbd 2024-09-11T06:50:28.475 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/rbd 2024-09-11T06:50:28.475 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:50:28.475 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map rbd/testrbdnbd21607 2024-09-11T06:50:28.491 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:28.487+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:28.538 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:28.538 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:28.553 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:50:28.597 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:28.595+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:28.692 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:28.687+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:28.719 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:28.720 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:28.734 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:50:28.793 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:28.791+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:28.894 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:28.891+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:28.901 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:28.901 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:28.916 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:50:29.005 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:29.003+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:29.081 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:29.081 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:29.095 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:29.091+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:29.096 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:50:29.162 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is unset 2024-09-11T06:50:29.196 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:29.191+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:29.205 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV=/dev/nbd0 2024-09-11T06:50:29.205 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd 2024-09-11T06:50:29.205 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:50:29.205 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:50:29.206 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:50:29.206 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd0'\'']/id' 2024-09-11T06:50:29.277 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=22277 2024-09-11T06:50:29.277 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 22277 2024-09-11T06:50:29.277 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 22277 -C rbd-nbd 2024-09-11T06:50:29.281 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:29.281 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:29.283 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:50:29.283 INFO:tasks.workunit.client.0.smithi176.stdout: 22277 ? 00:00:00 rbd-nbd 2024-09-11T06:50:29.284 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd -p rbd --format xml du testrbdnbd21607 2024-09-11T06:50:29.284 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -m //stats/images/image/provisioned_size -v . 2024-09-11T06:50:29.295 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:50:29.297 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:29.295+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:29.337 INFO:tasks.workunit.client.0.smithi176.stderr:+ provisioned=67108864 2024-09-11T06:50:29.337 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd -p rbd --format xml du testrbdnbd21607 2024-09-11T06:50:29.337 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -m //stats/images/image/used_size -v . 2024-09-11T06:50:29.387 INFO:tasks.workunit.client.0.smithi176.stderr:+ used=67108864 2024-09-11T06:50:29.387 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 67108864 -eq 67108864 ']' 2024-09-11T06:50:29.387 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo blkdiscard /dev/nbd0 2024-09-11T06:50:29.387 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:29.387 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:29.388 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:29.388 INFO:tasks.workunit.client.0.smithi176.stderr:++ which blkdiscard 2024-09-11T06:50:29.389 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/sbin/blkdiscard 2024-09-11T06:50:29.389 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:29.389 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/sbin/blkdiscard /dev/nbd0 2024-09-11T06:50:29.398 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:29.395+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:29.429 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:29.429 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:29.448 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:50:29.453 INFO:tasks.workunit.client.0.smithi176.stderr:+ sync 2024-09-11T06:50:29.499 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:29.495+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:29.563 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:29.563 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:29.577 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:50:29.600 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:29.595+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:29.607 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd -p rbd --format xml du testrbdnbd21607 2024-09-11T06:50:29.608 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -m //stats/images/image/provisioned_size -v . 2024-09-11T06:50:29.681 INFO:tasks.workunit.client.0.smithi176.stderr:+ provisioned=67108864 2024-09-11T06:50:29.682 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd -p rbd --format xml du testrbdnbd21607 2024-09-11T06:50:29.682 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -m //stats/images/image/used_size -v . 2024-09-11T06:50:29.700 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:29.699+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:29.710 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:29.710 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:29.725 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:50:29.748 INFO:tasks.workunit.client.0.smithi176.stderr:+ used=0 2024-09-11T06:50:29.748 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 0 -lt 67108864 ']' 2024-09-11T06:50:29.748 INFO:tasks.workunit.client.0.smithi176.stderr:+ unmap_device /dev/nbd0 22277 2024-09-11T06:50:29.748 INFO:tasks.workunit.client.0.smithi176.stderr:+ local args=/dev/nbd0 2024-09-11T06:50:29.748 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pid=22277 2024-09-11T06:50:29.748 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd0 2024-09-11T06:50:29.749 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:29.749 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:29.749 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:29.749 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:50:29.751 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:50:29.751 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:29.751 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd0 2024-09-11T06:50:29.774 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:50:29.812 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:29.811+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:29.834 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:50:29.834 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^22277\b' 2024-09-11T06:50:29.834 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^22277\b' 2024-09-11T06:50:29.881 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:29.882 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -C rbd-nbd 2024-09-11T06:50:29.882 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^ *22277\b' 2024-09-11T06:50:29.882 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^ *22277\b' 2024-09-11T06:50:29.888 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:29.888 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 0.5 2024-09-11T06:50:29.891 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:29.891 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:29.902 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:29.899+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:29.905 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:50:29.929 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:50:29.930 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:50:29.946 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:50:29.947 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:50:29.962 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:50:29.963 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:50:29.980 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:50:29.980 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:50:30.001 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:50:30.001 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:50:30.002 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:29.999+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:30.020 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:50:30.020 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:50:30.077 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:30.077 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:30.078 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:30.078 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:50:30.078 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.078 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:30.078 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:50:30.078 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 83.903917380999999, 2024-09-11T06:50:30.078 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011420669, 2024-09-11T06:50:30.078 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.078 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.078 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:30.078 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:30.078 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:30.078 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:30.078 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 37 2024-09-11T06:50:30.079 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.079 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.079 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.079 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.079 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:50:30.079 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.079 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.079 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.079 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.079 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:50:30.079 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.079 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.079 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.079 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.079 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170509+0000", 2024-09-11T06:50:30.080 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.13e-06 2024-09-11T06:50:30.080 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.080 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.080 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.080 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170521+0000", 2024-09-11T06:50:30.080 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1776999999999999e-05 2024-09-11T06:50:30.080 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.080 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.080 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.080 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170538+0000", 2024-09-11T06:50:30.080 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7413000000000001e-05 2024-09-11T06:50:30.080 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.080 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.081 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.081 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170570+0000", 2024-09-11T06:50:30.081 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2313e-05 2024-09-11T06:50:30.081 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.081 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.081 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.081 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179768+0000", 2024-09-11T06:50:30.081 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0091978010000000002 2024-09-11T06:50:30.081 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.081 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.081 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.081 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179951+0000", 2024-09-11T06:50:30.081 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018227900000000001 2024-09-11T06:50:30.082 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.082 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.082 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:30.082 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.180175+0000", 2024-09-11T06:50:30.082 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022432800000000001 2024-09-11T06:50:30.082 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.082 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.082 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:30.082 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181112+0000", 2024-09-11T06:50:30.082 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00093712500000000002 2024-09-11T06:50:30.082 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.082 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.082 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:30.082 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181789+0000", 2024-09-11T06:50:30.083 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00067705100000000002 2024-09-11T06:50:30.083 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.083 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.083 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:30.083 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181871+0000", 2024-09-11T06:50:30.083 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1816000000000004e-05 2024-09-11T06:50:30.083 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.083 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.083 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.083 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181929+0000", 2024-09-11T06:50:30.083 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7636000000000002e-05 2024-09-11T06:50:30.083 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.083 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.083 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.084 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.084 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.084 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:30.084 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:30.084 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 83.80265842, 2024-09-11T06:50:30.084 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283336, 2024-09-11T06:50:30.084 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.084 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.084 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:30.084 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:30.084 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:30.084 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:30.084 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 87 2024-09-11T06:50:30.084 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.085 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.085 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.085 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.085 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:30.085 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.085 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.085 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.085 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.085 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:30.085 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.085 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.085 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.085 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.086 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271768+0000", 2024-09-11T06:50:30.086 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7599999999999996e-07 2024-09-11T06:50:30.086 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.086 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.086 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.086 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271860+0000", 2024-09-11T06:50:30.086 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2639999999999994e-05 2024-09-11T06:50:30.086 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.086 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.086 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.086 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271873+0000", 2024-09-11T06:50:30.086 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3139000000000001e-05 2024-09-11T06:50:30.086 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.086 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.087 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.087 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271882+0000", 2024-09-11T06:50:30.087 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2369999999999992e-06 2024-09-11T06:50:30.087 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.087 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.087 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.087 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277150+0000", 2024-09-11T06:50:30.087 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052684419999999999 2024-09-11T06:50:30.087 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.087 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.087 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.087 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277273+0000", 2024-09-11T06:50:30.087 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012278899999999999 2024-09-11T06:50:30.087 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.087 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.087 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:30.088 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277466+0000", 2024-09-11T06:50:30.088 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193578 2024-09-11T06:50:30.088 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.088 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.088 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:30.088 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.280808+0000", 2024-09-11T06:50:30.088 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003341971 2024-09-11T06:50:30.088 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.088 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.088 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:30.088 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284419+0000", 2024-09-11T06:50:30.088 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003610982 2024-09-11T06:50:30.088 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.088 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.088 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:30.089 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284531+0000", 2024-09-11T06:50:30.089 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011121299999999999 2024-09-11T06:50:30.089 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.089 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.089 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.089 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284600+0000", 2024-09-11T06:50:30.089 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9592999999999996e-05 2024-09-11T06:50:30.089 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.089 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.089 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.089 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.089 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.089 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:30.089 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:30.090 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 83.802528167999995, 2024-09-11T06:50:30.090 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283314, 2024-09-11T06:50:30.090 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.090 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.090 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:30.090 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:30.090 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:30.090 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:30.090 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 88 2024-09-11T06:50:30.090 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.090 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.090 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.090 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.090 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:30.090 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.091 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.091 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.091 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.091 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:30.091 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.091 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.091 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.091 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.091 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271898+0000", 2024-09-11T06:50:30.091 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2999999999999999e-07 2024-09-11T06:50:30.091 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.091 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.091 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.091 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271904+0000", 2024-09-11T06:50:30.091 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.767e-06 2024-09-11T06:50:30.092 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.092 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.092 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.092 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271913+0000", 2024-09-11T06:50:30.092 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7200000000000001e-06 2024-09-11T06:50:30.092 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.092 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.092 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.092 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271920+0000", 2024-09-11T06:50:30.092 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7100000000000001e-06 2024-09-11T06:50:30.092 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.092 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.092 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.092 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278684+0000", 2024-09-11T06:50:30.092 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0067642650000000002 2024-09-11T06:50:30.093 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.093 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.093 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.093 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278838+0000", 2024-09-11T06:50:30.093 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015371499999999999 2024-09-11T06:50:30.093 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.093 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.093 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:30.093 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.279040+0000", 2024-09-11T06:50:30.093 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020207499999999999 2024-09-11T06:50:30.093 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.093 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.093 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:30.094 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.281448+0000", 2024-09-11T06:50:30.094 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002407686 2024-09-11T06:50:30.094 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.094 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.094 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:30.094 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284615+0000", 2024-09-11T06:50:30.094 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031675499999999999 2024-09-11T06:50:30.094 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.094 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.094 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:30.094 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284682+0000", 2024-09-11T06:50:30.094 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7032000000000005e-05 2024-09-11T06:50:30.094 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.094 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.094 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.095 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284730+0000", 2024-09-11T06:50:30.095 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7790000000000002e-05 2024-09-11T06:50:30.095 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.095 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.095 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.095 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.095 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.095 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:30.095 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:50:30.095 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 83.634626945999997, 2024-09-11T06:50:30.095 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011114877, 2024-09-11T06:50:30.095 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.095 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.095 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:30.096 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:30.096 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:30.096 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:30.096 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 107 2024-09-11T06:50:30.096 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.096 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.096 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.096 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.096 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:50:30.096 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.096 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.096 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.096 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.096 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439798+0000", 2024-09-11T06:50:30.097 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.097 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.097 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.097 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.097 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439799+0000", 2024-09-11T06:50:30.097 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.16e-07 2024-09-11T06:50:30.097 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.097 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.097 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.097 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439806+0000", 2024-09-11T06:50:30.097 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3640000000000004e-06 2024-09-11T06:50:30.097 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.097 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.098 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.098 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439816+0000", 2024-09-11T06:50:30.098 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0485999999999999e-05 2024-09-11T06:50:30.098 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.098 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.098 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.098 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.439824+0000", 2024-09-11T06:50:30.098 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.6699999999999994e-06 2024-09-11T06:50:30.098 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.098 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.098 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.098 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443328+0000", 2024-09-11T06:50:30.098 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035044329999999999 2024-09-11T06:50:30.098 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.099 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.099 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.099 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443461+0000", 2024-09-11T06:50:30.099 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013272199999999999 2024-09-11T06:50:30.099 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.099 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.099 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:30.099 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.443678+0000", 2024-09-11T06:50:30.099 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021674500000000001 2024-09-11T06:50:30.099 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.099 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.099 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:30.099 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449635+0000", 2024-09-11T06:50:30.099 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0059572119999999999 2024-09-11T06:50:30.100 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.100 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.100 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:30.100 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450750+0000", 2024-09-11T06:50:30.100 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001115654 2024-09-11T06:50:30.100 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.100 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.100 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:30.100 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450832+0000", 2024-09-11T06:50:30.100 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1188999999999998e-05 2024-09-11T06:50:30.100 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.100 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.100 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.100 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450913+0000", 2024-09-11T06:50:30.100 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1485999999999999e-05 2024-09-11T06:50:30.101 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.101 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.101 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.101 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.101 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.101 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:30.101 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:50:30.101 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 83.634035971000003, 2024-09-11T06:50:30.101 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011236556, 2024-09-11T06:50:30.101 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.101 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.101 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:30.101 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:30.101 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:30.102 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:30.102 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 109 2024-09-11T06:50:30.102 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.102 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.102 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.102 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.102 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:50:30.102 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.102 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.102 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.102 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.102 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:50:30.102 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.102 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.102 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.102 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.103 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440391+0000", 2024-09-11T06:50:30.103 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8360000000000001e-06 2024-09-11T06:50:30.103 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.103 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.103 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.103 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440560+0000", 2024-09-11T06:50:30.103 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000169085 2024-09-11T06:50:30.103 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.103 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.103 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.103 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440585+0000", 2024-09-11T06:50:30.103 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4530000000000001e-05 2024-09-11T06:50:30.103 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.103 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.103 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.103 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440606+0000", 2024-09-11T06:50:30.104 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1277e-05 2024-09-11T06:50:30.104 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.104 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.104 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.104 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446479+0000", 2024-09-11T06:50:30.104 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0058734989999999999 2024-09-11T06:50:30.104 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.104 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.104 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.104 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446649+0000", 2024-09-11T06:50:30.104 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016948199999999999 2024-09-11T06:50:30.104 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.104 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.104 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:30.105 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446854+0000", 2024-09-11T06:50:30.105 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020491500000000001 2024-09-11T06:50:30.105 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.105 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.105 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:30.105 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451380+0000", 2024-09-11T06:50:30.105 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0045264110000000001 2024-09-11T06:50:30.105 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.105 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.105 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:30.105 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451479+0000", 2024-09-11T06:50:30.105 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.8975999999999996e-05 2024-09-11T06:50:30.105 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.105 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.106 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:30.106 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451562+0000", 2024-09-11T06:50:30.106 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3213000000000004e-05 2024-09-11T06:50:30.106 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.106 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.106 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.106 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451626+0000", 2024-09-11T06:50:30.106 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3331999999999996e-05 2024-09-11T06:50:30.106 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.107 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.107 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.107 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.107 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.107 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:30.107 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:50:30.107 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 83.633786842999996, 2024-09-11T06:50:30.107 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011252866, 2024-09-11T06:50:30.107 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.107 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.107 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:30.107 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:30.107 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:30.107 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:30.108 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 110 2024-09-11T06:50:30.108 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.108 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.108 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.108 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.108 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:50:30.108 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.108 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.108 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.108 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.108 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:50:30.108 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.108 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.109 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.109 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.109 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440640+0000", 2024-09-11T06:50:30.109 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5430000000000001e-06 2024-09-11T06:50:30.109 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.109 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.109 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.109 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440651+0000", 2024-09-11T06:50:30.109 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0672999999999999e-05 2024-09-11T06:50:30.109 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.109 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.109 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.109 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440670+0000", 2024-09-11T06:50:30.109 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9726999999999999e-05 2024-09-11T06:50:30.109 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.110 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.110 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.110 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440691+0000", 2024-09-11T06:50:30.110 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0526e-05 2024-09-11T06:50:30.110 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.110 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.110 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.110 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448191+0000", 2024-09-11T06:50:30.110 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0075005990000000002 2024-09-11T06:50:30.110 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.110 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.110 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.110 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448390+0000", 2024-09-11T06:50:30.111 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000198815 2024-09-11T06:50:30.111 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.111 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.111 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:30.111 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448735+0000", 2024-09-11T06:50:30.111 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034424999999999999 2024-09-11T06:50:30.111 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.111 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.111 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:30.111 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451742+0000", 2024-09-11T06:50:30.111 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030077440000000001 2024-09-11T06:50:30.111 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.111 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.111 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:30.111 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451761+0000", 2024-09-11T06:50:30.112 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9196999999999999e-05 2024-09-11T06:50:30.112 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.112 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.112 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:30.112 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451834+0000", 2024-09-11T06:50:30.112 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2071999999999997e-05 2024-09-11T06:50:30.112 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.112 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.112 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.112 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451891+0000", 2024-09-11T06:50:30.112 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7720000000000003e-05 2024-09-11T06:50:30.112 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.112 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.113 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.113 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.113 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.113 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:30.113 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:30.113 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 83.633678967999998, 2024-09-11T06:50:30.113 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014944941, 2024-09-11T06:50:30.113 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.113 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.113 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:30.113 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:30.113 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:30.113 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:30.113 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 111 2024-09-11T06:50:30.113 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.114 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.114 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.114 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.114 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:30.114 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.114 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.114 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.114 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.114 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:30.114 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.114 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.114 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.114 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.114 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440748+0000", 2024-09-11T06:50:30.115 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:50:30.115 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.115 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.115 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.115 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440760+0000", 2024-09-11T06:50:30.115 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1897e-05 2024-09-11T06:50:30.115 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.115 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.115 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.115 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440782+0000", 2024-09-11T06:50:30.115 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2552999999999999e-05 2024-09-11T06:50:30.115 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.115 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.115 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.115 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440793+0000", 2024-09-11T06:50:30.115 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0643e-05 2024-09-11T06:50:30.116 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.116 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.116 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.116 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449755+0000", 2024-09-11T06:50:30.116 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089622620000000004 2024-09-11T06:50:30.116 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.116 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.116 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.116 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449918+0000", 2024-09-11T06:50:30.116 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163086 2024-09-11T06:50:30.116 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.116 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.116 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:30.116 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450150+0000", 2024-09-11T06:50:30.117 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023204500000000001 2024-09-11T06:50:30.117 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.117 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.117 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:30.117 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454533+0000", 2024-09-11T06:50:30.117 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0043831349999999998 2024-09-11T06:50:30.117 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.117 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.117 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:30.117 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455511+0000", 2024-09-11T06:50:30.117 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0009776050000000001 2024-09-11T06:50:30.117 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.117 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.117 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:30.117 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455597+0000", 2024-09-11T06:50:30.118 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6446000000000005e-05 2024-09-11T06:50:30.118 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.118 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.118 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.118 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455691+0000", 2024-09-11T06:50:30.118 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3832000000000006e-05 2024-09-11T06:50:30.118 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.118 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.118 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.118 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.118 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.118 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:30.118 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:30.118 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 83.633609204999999, 2024-09-11T06:50:30.119 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018885411000000001, 2024-09-11T06:50:30.119 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.119 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.119 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:30.119 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:30.119 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:30.119 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:30.119 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 112 2024-09-11T06:50:30.119 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.119 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.119 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.119 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.119 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:30.119 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.120 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.120 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.120 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.120 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:30.120 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.120 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.120 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.120 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.120 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440817+0000", 2024-09-11T06:50:30.120 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:50:30.120 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.120 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.120 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.120 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441003+0000", 2024-09-11T06:50:30.120 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018661200000000001 2024-09-11T06:50:30.121 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.121 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.121 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.121 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441027+0000", 2024-09-11T06:50:30.121 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3920000000000001e-05 2024-09-11T06:50:30.121 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.121 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.121 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.121 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441062+0000", 2024-09-11T06:50:30.121 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4663000000000003e-05 2024-09-11T06:50:30.121 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.121 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.121 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.121 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451981+0000", 2024-09-11T06:50:30.121 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010918926000000001 2024-09-11T06:50:30.122 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.122 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.122 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.122 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452111+0000", 2024-09-11T06:50:30.122 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129529 2024-09-11T06:50:30.122 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.122 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.122 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:30.122 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452311+0000", 2024-09-11T06:50:30.122 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020013799999999999 2024-09-11T06:50:30.122 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.122 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.122 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:30.122 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455809+0000", 2024-09-11T06:50:30.123 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003498443 2024-09-11T06:50:30.123 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.123 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.123 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:30.123 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459566+0000", 2024-09-11T06:50:30.123 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003757271 2024-09-11T06:50:30.123 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.123 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.123 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:30.123 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459640+0000", 2024-09-11T06:50:30.123 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3949000000000006e-05 2024-09-11T06:50:30.123 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.123 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.124 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.124 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459701+0000", 2024-09-11T06:50:30.124 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1117000000000005e-05 2024-09-11T06:50:30.124 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.124 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.124 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.124 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.124 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.124 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:30.124 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:30.124 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 83.633330353999995, 2024-09-11T06:50:30.124 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018760915, 2024-09-11T06:50:30.124 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.125 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.125 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:30.125 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:30.125 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:30.125 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:30.125 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 113 2024-09-11T06:50:30.125 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.125 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.125 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.125 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.125 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:30.125 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.125 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.126 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.126 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.126 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:30.126 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.126 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.126 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.126 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.126 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441096+0000", 2024-09-11T06:50:30.126 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4130000000000001e-06 2024-09-11T06:50:30.126 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.126 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.126 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.126 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441110+0000", 2024-09-11T06:50:30.126 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3900000000000001e-05 2024-09-11T06:50:30.127 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.127 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.127 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.127 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441131+0000", 2024-09-11T06:50:30.127 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0636999999999999e-05 2024-09-11T06:50:30.127 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.127 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.127 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.127 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441147+0000", 2024-09-11T06:50:30.127 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6512999999999999e-05 2024-09-11T06:50:30.127 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.127 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.127 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.127 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453274+0000", 2024-09-11T06:50:30.127 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012126764999999999 2024-09-11T06:50:30.128 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.128 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.128 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.128 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453498+0000", 2024-09-11T06:50:30.128 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000223568 2024-09-11T06:50:30.128 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.128 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.128 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:30.128 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453872+0000", 2024-09-11T06:50:30.128 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037467099999999998 2024-09-11T06:50:30.128 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.128 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.128 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:30.128 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458709+0000", 2024-09-11T06:50:30.128 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0048365680000000003 2024-09-11T06:50:30.129 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.129 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.129 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:30.129 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459716+0000", 2024-09-11T06:50:30.129 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010075240000000001 2024-09-11T06:50:30.129 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.129 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.129 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:30.129 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459779+0000", 2024-09-11T06:50:30.129 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2336999999999996e-05 2024-09-11T06:50:30.129 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.129 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.129 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.129 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459856+0000", 2024-09-11T06:50:30.129 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7019000000000002e-05 2024-09-11T06:50:30.130 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.130 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.130 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.130 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.130 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.130 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:30.130 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:30.130 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 83.633253615000001, 2024-09-11T06:50:30.130 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018963114, 2024-09-11T06:50:30.130 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.130 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.130 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:30.130 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:30.130 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:30.130 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:30.130 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 114 2024-09-11T06:50:30.131 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.131 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.131 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.131 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.131 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:30.131 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.131 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.131 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.131 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.131 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:30.131 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.131 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.131 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.131 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.131 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:30.131 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9299999999999996e-07 2024-09-11T06:50:30.132 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.132 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.132 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.132 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441180+0000", 2024-09-11T06:50:30.132 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7200000000000006e-06 2024-09-11T06:50:30.132 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.132 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.132 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.132 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441265+0000", 2024-09-11T06:50:30.132 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4753000000000007e-05 2024-09-11T06:50:30.132 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.132 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.132 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.132 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441288+0000", 2024-09-11T06:50:30.133 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2860000000000001e-05 2024-09-11T06:50:30.133 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.133 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.133 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.133 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454585+0000", 2024-09-11T06:50:30.133 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013297286 2024-09-11T06:50:30.133 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.133 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.133 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.133 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454714+0000", 2024-09-11T06:50:30.133 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000128465 2024-09-11T06:50:30.133 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.133 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.133 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:30.133 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454909+0000", 2024-09-11T06:50:30.133 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019522499999999999 2024-09-11T06:50:30.134 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.134 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.134 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:30.134 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459868+0000", 2024-09-11T06:50:30.134 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0049591239999999997 2024-09-11T06:50:30.134 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.134 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.134 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:30.134 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459988+0000", 2024-09-11T06:50:30.134 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000120532 2024-09-11T06:50:30.134 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.134 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.134 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:30.134 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460071+0000", 2024-09-11T06:50:30.134 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2256000000000007e-05 2024-09-11T06:50:30.135 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.135 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.135 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.135 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460135+0000", 2024-09-11T06:50:30.135 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3999999999999997e-05 2024-09-11T06:50:30.135 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.135 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.135 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.135 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.135 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.135 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:30.135 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:30.135 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 83.633095986000001, 2024-09-11T06:50:30.135 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.019036849000000002, 2024-09-11T06:50:30.135 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.136 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.136 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:30.136 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:30.136 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:30.136 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:30.136 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 115 2024-09-11T06:50:30.136 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.136 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.136 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.136 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.136 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:30.136 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.136 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.136 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.136 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.137 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:30.137 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.137 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.137 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.137 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.137 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441330+0000", 2024-09-11T06:50:30.137 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1930000000000001e-06 2024-09-11T06:50:30.137 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.137 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.137 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.137 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441544+0000", 2024-09-11T06:50:30.137 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021405799999999999 2024-09-11T06:50:30.137 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.137 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.137 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.138 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441572+0000", 2024-09-11T06:50:30.138 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7317e-05 2024-09-11T06:50:30.138 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.138 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.138 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.138 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441593+0000", 2024-09-11T06:50:30.138 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1702999999999999e-05 2024-09-11T06:50:30.138 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.138 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.138 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.138 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455891+0000", 2024-09-11T06:50:30.138 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01429708 2024-09-11T06:50:30.138 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.138 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.138 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.139 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456046+0000", 2024-09-11T06:50:30.139 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000155682 2024-09-11T06:50:30.139 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.139 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.139 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:30.139 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456242+0000", 2024-09-11T06:50:30.139 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000195692 2024-09-11T06:50:30.139 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.139 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.139 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:30.139 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459876+0000", 2024-09-11T06:50:30.139 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036336480000000002 2024-09-11T06:50:30.139 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.139 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.140 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:30.140 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460245+0000", 2024-09-11T06:50:30.140 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036901700000000002 2024-09-11T06:50:30.140 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.140 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.140 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:30.140 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460316+0000", 2024-09-11T06:50:30.140 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1846000000000003e-05 2024-09-11T06:50:30.140 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.140 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.140 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.140 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460366+0000", 2024-09-11T06:50:30.140 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9613000000000001e-05 2024-09-11T06:50:30.140 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.140 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.141 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.141 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.141 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.141 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:30.141 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:30.141 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 83.632773905999997, 2024-09-11T06:50:30.141 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01904432, 2024-09-11T06:50:30.141 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.141 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.141 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:30.141 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:30.141 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:30.141 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:30.142 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 116 2024-09-11T06:50:30.142 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.142 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.142 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.142 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.142 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:30.142 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.142 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.142 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.142 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.142 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:30.142 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.142 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.142 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.142 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.143 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441653+0000", 2024-09-11T06:50:30.143 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:50:30.143 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.143 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.143 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.143 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441665+0000", 2024-09-11T06:50:30.143 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1653e-05 2024-09-11T06:50:30.143 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.143 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.143 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.143 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441684+0000", 2024-09-11T06:50:30.143 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9490000000000001e-05 2024-09-11T06:50:30.143 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.143 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.143 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.144 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441699+0000", 2024-09-11T06:50:30.144 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5044e-05 2024-09-11T06:50:30.144 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.144 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.144 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.144 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457297+0000", 2024-09-11T06:50:30.144 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015598241000000001 2024-09-11T06:50:30.144 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.144 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.144 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.144 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457577+0000", 2024-09-11T06:50:30.144 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027930799999999998 2024-09-11T06:50:30.144 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.144 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.144 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:30.145 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457957+0000", 2024-09-11T06:50:30.145 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00038088 2024-09-11T06:50:30.145 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.145 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.145 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:30.145 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460432+0000", 2024-09-11T06:50:30.145 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024743349999999998 2024-09-11T06:50:30.145 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.145 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.145 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:30.145 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460584+0000", 2024-09-11T06:50:30.145 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015232900000000001 2024-09-11T06:50:30.145 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.145 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.146 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:30.146 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460646+0000", 2024-09-11T06:50:30.146 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1883000000000004e-05 2024-09-11T06:50:30.146 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.146 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.146 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.146 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460696+0000", 2024-09-11T06:50:30.146 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9589999999999998e-05 2024-09-11T06:50:30.146 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.146 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.146 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.146 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.146 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.146 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:30.146 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:30.147 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 83.632701839999996, 2024-09-11T06:50:30.147 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.021834473, 2024-09-11T06:50:30.147 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.147 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.147 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:30.147 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:30.147 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:30.147 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:30.147 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 117 2024-09-11T06:50:30.147 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.147 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.147 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.147 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.147 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:30.147 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.147 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.148 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.148 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.148 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:30.148 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.148 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.148 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.148 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.148 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441725+0000", 2024-09-11T06:50:30.148 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5069999999999999e-06 2024-09-11T06:50:30.148 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.148 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.148 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.148 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441738+0000", 2024-09-11T06:50:30.148 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2813000000000001e-05 2024-09-11T06:50:30.149 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.149 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.149 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.149 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441759+0000", 2024-09-11T06:50:30.149 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1246999999999998e-05 2024-09-11T06:50:30.149 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.149 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.149 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.149 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441779+0000", 2024-09-11T06:50:30.149 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0392999999999999e-05 2024-09-11T06:50:30.149 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.149 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.149 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.149 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458804+0000", 2024-09-11T06:50:30.150 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.017024182999999998 2024-09-11T06:50:30.150 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.150 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.150 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.150 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458977+0000", 2024-09-11T06:50:30.150 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000173926 2024-09-11T06:50:30.150 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.150 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.150 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:30.150 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459170+0000", 2024-09-11T06:50:30.150 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000192184 2024-09-11T06:50:30.150 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.150 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.150 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:30.150 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.462300+0000", 2024-09-11T06:50:30.151 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031304969999999999 2024-09-11T06:50:30.151 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.151 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.151 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:30.151 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463408+0000", 2024-09-11T06:50:30.151 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011077839999999999 2024-09-11T06:50:30.151 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.151 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.151 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:30.151 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463497+0000", 2024-09-11T06:50:30.151 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8828999999999999e-05 2024-09-11T06:50:30.151 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.151 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.151 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.152 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463558+0000", 2024-09-11T06:50:30.152 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1110000000000003e-05 2024-09-11T06:50:30.152 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.152 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.152 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.152 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.152 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.152 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:30.152 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:50:30.152 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 83.256194152000006, 2024-09-11T06:50:30.152 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012216898, 2024-09-11T06:50:30.152 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.152 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.152 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:30.153 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:30.153 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:30.153 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:30.153 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 137 2024-09-11T06:50:30.153 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.153 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.153 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.153 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.153 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:50:30.153 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.153 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.153 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.153 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.153 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:50:30.153 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.154 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.154 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.154 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.154 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818232+0000", 2024-09-11T06:50:30.154 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2839999999999999e-06 2024-09-11T06:50:30.154 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.154 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.154 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.154 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818326+0000", 2024-09-11T06:50:30.154 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3531999999999998e-05 2024-09-11T06:50:30.154 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.154 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.154 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.154 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818337+0000", 2024-09-11T06:50:30.154 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0655999999999999e-05 2024-09-11T06:50:30.155 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.155 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.155 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.155 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818342+0000", 2024-09-11T06:50:30.155 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4600000000000002e-06 2024-09-11T06:50:30.155 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.155 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.155 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.155 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823890+0000", 2024-09-11T06:50:30.155 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0055480700000000004 2024-09-11T06:50:30.155 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.155 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.155 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.155 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823953+0000", 2024-09-11T06:50:30.155 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2848999999999994e-05 2024-09-11T06:50:30.155 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.156 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.156 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:30.156 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824074+0000", 2024-09-11T06:50:30.156 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121309 2024-09-11T06:50:30.156 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.156 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.156 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:30.156 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828043+0000", 2024-09-11T06:50:30.156 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0039685240000000002 2024-09-11T06:50:30.156 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.156 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.156 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:30.156 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830293+0000", 2024-09-11T06:50:30.157 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0022503179999999998 2024-09-11T06:50:30.157 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.157 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.157 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:30.157 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830366+0000", 2024-09-11T06:50:30.157 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2539999999999993e-05 2024-09-11T06:50:30.157 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.157 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.157 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.157 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830448+0000", 2024-09-11T06:50:30.157 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2355999999999996e-05 2024-09-11T06:50:30.157 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.157 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.158 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.158 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.158 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.158 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:30.158 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:30.158 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 83.256060337999998, 2024-09-11T06:50:30.158 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013136297, 2024-09-11T06:50:30.158 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.158 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.158 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:30.158 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:30.158 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:30.158 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:30.158 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 138 2024-09-11T06:50:30.158 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.159 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.159 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.159 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.159 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:30.159 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.159 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.159 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.159 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.159 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:30.159 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.159 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.159 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.159 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.159 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:30.160 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.89e-07 2024-09-11T06:50:30.160 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.160 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.160 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.160 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818369+0000", 2024-09-11T06:50:30.160 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4170000000000001e-06 2024-09-11T06:50:30.160 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.160 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.160 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.160 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818376+0000", 2024-09-11T06:50:30.160 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4270000000000002e-06 2024-09-11T06:50:30.160 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.160 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.160 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.161 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818383+0000", 2024-09-11T06:50:30.161 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3060000000000003e-06 2024-09-11T06:50:30.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.161 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.161 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.161 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824783+0000", 2024-09-11T06:50:30.161 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064003020000000001 2024-09-11T06:50:30.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.161 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.161 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.161 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825013+0000", 2024-09-11T06:50:30.161 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023039499999999999 2024-09-11T06:50:30.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.161 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.161 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:30.162 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825484+0000", 2024-09-11T06:50:30.162 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000470562 2024-09-11T06:50:30.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.162 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.162 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:30.162 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828633+0000", 2024-09-11T06:50:30.162 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031496689999999999 2024-09-11T06:50:30.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.162 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.162 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:30.162 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831413+0000", 2024-09-11T06:50:30.162 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027793219999999999 2024-09-11T06:50:30.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.162 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.162 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:30.163 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831450+0000", 2024-09-11T06:50:30.163 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7370999999999998e-05 2024-09-11T06:50:30.163 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.163 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.163 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.163 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831501+0000", 2024-09-11T06:50:30.163 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1036999999999999e-05 2024-09-11T06:50:30.163 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.163 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.163 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.163 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.163 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.163 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:30.163 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:30.164 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 83.256035858000004, 2024-09-11T06:50:30.164 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013223011, 2024-09-11T06:50:30.164 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.164 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.164 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:30.164 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:30.164 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:30.164 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:30.164 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 139 2024-09-11T06:50:30.164 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.164 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.164 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.164 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.165 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:30.165 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.165 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.165 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.165 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:30.165 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.165 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.165 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.165 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818390+0000", 2024-09-11T06:50:30.165 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5600000000000001e-07 2024-09-11T06:50:30.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.166 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.166 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.166 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818394+0000", 2024-09-11T06:50:30.166 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5929999999999999e-06 2024-09-11T06:50:30.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.166 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.166 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.166 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818399+0000", 2024-09-11T06:50:30.166 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7740000000000004e-06 2024-09-11T06:50:30.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.166 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.166 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.166 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818412+0000", 2024-09-11T06:50:30.166 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2625e-05 2024-09-11T06:50:30.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.167 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.167 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.167 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826205+0000", 2024-09-11T06:50:30.167 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0077930730000000002 2024-09-11T06:50:30.167 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.167 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.167 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.167 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826281+0000", 2024-09-11T06:50:30.167 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5876000000000003e-05 2024-09-11T06:50:30.167 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.167 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.167 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:30.167 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826384+0000", 2024-09-11T06:50:30.167 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103838 2024-09-11T06:50:30.168 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.168 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.168 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:30.168 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830642+0000", 2024-09-11T06:50:30.168 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004257083 2024-09-11T06:50:30.168 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.168 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.168 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:30.168 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831509+0000", 2024-09-11T06:50:30.168 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086730900000000003 2024-09-11T06:50:30.168 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.168 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.168 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:30.168 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831547+0000", 2024-09-11T06:50:30.168 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7793000000000002e-05 2024-09-11T06:50:30.169 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.169 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.169 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.169 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831612+0000", 2024-09-11T06:50:30.169 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5691000000000005e-05 2024-09-11T06:50:30.169 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.169 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.169 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.169 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.169 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.169 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:30.169 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:30.169 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 83.256007163999996, 2024-09-11T06:50:30.169 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016098024999999998, 2024-09-11T06:50:30.170 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.170 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.170 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:30.170 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:30.170 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:30.170 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:30.170 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 140 2024-09-11T06:50:30.170 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.170 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.170 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.170 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.170 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:30.170 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.170 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.170 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.171 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.171 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:30.171 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.171 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.171 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.171 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.171 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:30.171 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5199999999999998e-07 2024-09-11T06:50:30.171 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.171 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.171 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.171 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818479+0000", 2024-09-11T06:50:30.172 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0368000000000002e-05 2024-09-11T06:50:30.172 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.172 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.172 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.172 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818487+0000", 2024-09-11T06:50:30.172 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2339999999999994e-06 2024-09-11T06:50:30.172 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.172 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.172 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.172 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818491+0000", 2024-09-11T06:50:30.172 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4329999999999997e-06 2024-09-11T06:50:30.172 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.172 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.172 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.173 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828080+0000", 2024-09-11T06:50:30.173 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0095890449999999992 2024-09-11T06:50:30.173 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.173 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.173 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.173 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828130+0000", 2024-09-11T06:50:30.173 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9239000000000001e-05 2024-09-11T06:50:30.173 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.173 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.173 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:30.173 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828208+0000", 2024-09-11T06:50:30.173 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.852e-05 2024-09-11T06:50:30.173 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.174 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.174 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:30.174 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831625+0000", 2024-09-11T06:50:30.174 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034169410000000002 2024-09-11T06:50:30.174 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.174 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.174 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:30.174 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834411+0000", 2024-09-11T06:50:30.174 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002786111 2024-09-11T06:50:30.174 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.174 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.174 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:30.174 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834457+0000", 2024-09-11T06:50:30.175 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6041000000000002e-05 2024-09-11T06:50:30.175 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.175 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.175 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.175 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834516+0000", 2024-09-11T06:50:30.175 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8740999999999999e-05 2024-09-11T06:50:30.175 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.175 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.175 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.175 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.175 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.175 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:30.175 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:30.175 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 83.255926097, 2024-09-11T06:50:30.176 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016588772000000002, 2024-09-11T06:50:30.176 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.176 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.176 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:30.176 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:30.176 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:30.176 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:30.176 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 141 2024-09-11T06:50:30.176 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.176 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.176 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.176 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.176 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:30.177 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.177 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.177 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.177 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.177 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:30.177 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.177 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.177 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.177 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.177 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818500+0000", 2024-09-11T06:50:30.177 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0800000000000005e-07 2024-09-11T06:50:30.177 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.177 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.178 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.178 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818505+0000", 2024-09-11T06:50:30.178 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2429999999999996e-06 2024-09-11T06:50:30.178 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.178 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.178 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.178 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818510+0000", 2024-09-11T06:50:30.178 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9339999999999997e-06 2024-09-11T06:50:30.178 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.178 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.178 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.178 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818514+0000", 2024-09-11T06:50:30.178 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7270000000000001e-06 2024-09-11T06:50:30.178 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.179 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.179 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.179 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828711+0000", 2024-09-11T06:50:30.179 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010197031 2024-09-11T06:50:30.179 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.179 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.179 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.179 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828853+0000", 2024-09-11T06:50:30.179 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014201200000000001 2024-09-11T06:50:30.179 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.179 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.179 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:30.179 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.829115+0000", 2024-09-11T06:50:30.180 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026257500000000003 2024-09-11T06:50:30.180 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.180 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.180 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:30.180 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831629+0000", 2024-09-11T06:50:30.180 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025138169999999998 2024-09-11T06:50:30.180 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.180 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.180 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:30.180 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834991+0000", 2024-09-11T06:50:30.180 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033624459999999998 2024-09-11T06:50:30.180 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.180 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.181 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:30.181 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835032+0000", 2024-09-11T06:50:30.181 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1001000000000003e-05 2024-09-11T06:50:30.181 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.181 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.181 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.181 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835088+0000", 2024-09-11T06:50:30.181 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5478e-05 2024-09-11T06:50:30.181 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.181 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.181 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.181 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.181 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.181 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:50:30.182 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:50:30.182 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 82.632291194000004, 2024-09-11T06:50:30.182 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011161912, 2024-09-11T06:50:30.182 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.182 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.182 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:30.182 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:30.182 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:30.182 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:30.182 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 158 2024-09-11T06:50:30.182 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.182 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.182 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.182 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.183 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:50:30.183 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.183 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.183 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.183 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.183 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442134+0000", 2024-09-11T06:50:30.183 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.183 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.183 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.183 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.183 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442135+0000", 2024-09-11T06:50:30.183 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3e-07 2024-09-11T06:50:30.183 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.183 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.184 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.184 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442259+0000", 2024-09-11T06:50:30.184 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000123903 2024-09-11T06:50:30.184 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.184 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.184 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.184 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442289+0000", 2024-09-11T06:50:30.184 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0833000000000001e-05 2024-09-11T06:50:30.184 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.184 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.184 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.184 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442301+0000", 2024-09-11T06:50:30.184 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1433e-05 2024-09-11T06:50:30.184 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.185 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.185 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.185 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449796+0000", 2024-09-11T06:50:30.185 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0074954959999999999 2024-09-11T06:50:30.185 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.185 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.185 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.185 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.449917+0000", 2024-09-11T06:50:30.185 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121033 2024-09-11T06:50:30.185 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.185 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.185 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:30.186 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.450166+0000", 2024-09-11T06:50:30.186 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00024832400000000002 2024-09-11T06:50:30.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.186 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.186 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:30.186 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.452630+0000", 2024-09-11T06:50:30.186 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002464188 2024-09-11T06:50:30.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.186 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.186 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:30.186 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453149+0000", 2024-09-11T06:50:30.186 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00051860999999999997 2024-09-11T06:50:30.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.186 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.187 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:30.187 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453234+0000", 2024-09-11T06:50:30.187 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.5865000000000005e-05 2024-09-11T06:50:30.187 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.187 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.187 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.187 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453296+0000", 2024-09-11T06:50:30.187 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1496999999999998e-05 2024-09-11T06:50:30.187 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.187 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.187 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.187 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.187 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.187 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:50:30.187 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:50:30.188 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 82.632105092000003, 2024-09-11T06:50:30.188 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011561900999999999, 2024-09-11T06:50:30.188 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.188 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.188 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:30.188 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:30.188 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:30.188 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:30.188 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 159 2024-09-11T06:50:30.188 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.188 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.188 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.188 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.188 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:50:30.188 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.189 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.189 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.189 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:50:30.189 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.189 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.189 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.189 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442321+0000", 2024-09-11T06:50:30.189 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8400000000000003e-07 2024-09-11T06:50:30.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.189 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.189 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.189 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442328+0000", 2024-09-11T06:50:30.190 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5130000000000002e-06 2024-09-11T06:50:30.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.190 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.190 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.190 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442341+0000", 2024-09-11T06:50:30.190 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.363e-05 2024-09-11T06:50:30.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.190 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.190 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.190 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442349+0000", 2024-09-11T06:50:30.190 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4760000000000001e-06 2024-09-11T06:50:30.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.190 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.190 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.190 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451257+0000", 2024-09-11T06:50:30.191 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089088569999999992 2024-09-11T06:50:30.191 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.191 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.191 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.191 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451389+0000", 2024-09-11T06:50:30.191 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013110899999999999 2024-09-11T06:50:30.191 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.191 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.191 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:30.191 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451608+0000", 2024-09-11T06:50:30.191 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000219588 2024-09-11T06:50:30.191 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.191 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.191 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:30.191 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453517+0000", 2024-09-11T06:50:30.191 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0019088779999999999 2024-09-11T06:50:30.192 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.192 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.192 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:30.192 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453727+0000", 2024-09-11T06:50:30.192 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020955100000000001 2024-09-11T06:50:30.192 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.192 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.192 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:30.192 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453815+0000", 2024-09-11T06:50:30.192 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8858999999999997e-05 2024-09-11T06:50:30.192 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.192 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.192 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.192 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453882+0000", 2024-09-11T06:50:30.192 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.6556000000000004e-05 2024-09-11T06:50:30.193 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.193 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.193 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.193 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.193 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.193 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:30.194 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:50:30.195 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:30.191+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:30.296 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:30.291+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:30.321 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:30.322 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:30.322 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:50:30.322 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:30.336 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:50:30.390 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo rbd device -t nbd -o try-netlink map rbd/testrbdnbd21607 2024-09-11T06:50:30.391 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:50:30.391 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:50:30.393 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:50:30.393 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which rbd 2024-09-11T06:50:30.396 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/rbd 2024-09-11T06:50:30.396 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:50:30.396 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/rbd device -t nbd -o try-netlink map rbd/testrbdnbd21607 2024-09-11T06:50:30.397 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:30.395+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:30.457 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:30.457 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:30.457 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:50:30.457 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:50:30.457 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:30.472 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:50:30.516 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:30.515+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:30.561 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV=/dev/nbd0 2024-09-11T06:50:30.561 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd 2024-09-11T06:50:30.561 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:50:30.561 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:50:30.562 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:50:30.562 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd0'\'']/id' 2024-09-11T06:50:30.598 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:30.595+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:30.630 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=22433 2024-09-11T06:50:30.630 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 22433 2024-09-11T06:50:30.630 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 22433 -C rbd-nbd 2024-09-11T06:50:30.637 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:50:30.637 INFO:tasks.workunit.client.0.smithi176.stdout: 22433 ? 00:00:00 rbd-nbd 2024-09-11T06:50:30.638 INFO:tasks.workunit.client.0.smithi176.stderr:++ basename /dev/nbd0 2024-09-11T06:50:30.638 INFO:tasks.workunit.client.0.smithi176.stderr:+ devname=nbd0 2024-09-11T06:50:30.639 INFO:tasks.workunit.client.0.smithi176.stderr:++ awk -v dev=nbd0 '$4 == dev {print $3}' /proc/partitions 2024-09-11T06:50:30.640 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:30.640 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:30.640 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:30.640 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:50:30.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.641 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4558.0:1 3.3 3:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-11T06:50:30.641 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:50:30.641 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 13.548915493999999, 2024-09-11T06:50:30.641 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0085520600000000002, 2024-09-11T06:50:30.641 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.641 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.641 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:30.641 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:30.641 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4558", 2024-09-11T06:50:30.641 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1478388898", 2024-09-11T06:50:30.641 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:50:30.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.642 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.642 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.642 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:50:30.642 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.642 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.642 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:50:30.642 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.642 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.643 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090533+0000", 2024-09-11T06:50:30.643 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2629999999999999e-06 2024-09-11T06:50:30.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.643 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.643 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090546+0000", 2024-09-11T06:50:30.643 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2918999999999999e-05 2024-09-11T06:50:30.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.643 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.643 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090572+0000", 2024-09-11T06:50:30.643 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6412000000000001e-05 2024-09-11T06:50:30.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.644 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.644 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090594+0000", 2024-09-11T06:50:30.644 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2476e-05 2024-09-11T06:50:30.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.644 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.644 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090938+0000", 2024-09-11T06:50:30.644 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034325099999999998 2024-09-11T06:50:30.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.644 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.644 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.091149+0000", 2024-09-11T06:50:30.644 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021151300000000001 2024-09-11T06:50:30.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:30.645 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.091377+0000", 2024-09-11T06:50:30.645 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022785399999999999 2024-09-11T06:50:30.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:30.645 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.092912+0000", 2024-09-11T06:50:30.645 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0015346089999999999 2024-09-11T06:50:30.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:30.645 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098919+0000", 2024-09-11T06:50:30.646 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0060076799999999996 2024-09-11T06:50:30.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.646 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:30.646 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.099009+0000", 2024-09-11T06:50:30.646 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0056000000000006e-05 2024-09-11T06:50:30.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.646 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.647 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.099081+0000", 2024-09-11T06:50:30.647 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2026999999999993e-05 2024-09-11T06:50:30.647 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.647 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.647 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.647 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop_reply(client.4558.0:1 3.3 e56/55 ondisk, result = 0)", 2024-09-11T06:50:30.647 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:17.098705+0000", 2024-09-11T06:50:30.647 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 13.540740339999999, 2024-09-11T06:50:30.647 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00039016499999999999, 2024-09-11T06:50:30.648 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.648 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.648 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:30.648 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.648 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.648 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098705+0000", 2024-09-11T06:50:30.648 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.648 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.648 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098705+0000", 2024-09-11T06:50:30.648 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.649 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.649 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098706+0000", 2024-09-11T06:50:30.649 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.626e-06 2024-09-11T06:50:30.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.649 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.649 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098717+0000", 2024-09-11T06:50:30.649 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0699e-05 2024-09-11T06:50:30.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.650 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.650 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098743+0000", 2024-09-11T06:50:30.650 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6449e-05 2024-09-11T06:50:30.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.650 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.650 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098759+0000", 2024-09-11T06:50:30.650 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5815999999999999e-05 2024-09-11T06:50:30.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.650 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.650 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098894+0000", 2024-09-11T06:50:30.650 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013442099999999999 2024-09-11T06:50:30.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.651 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.651 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098910+0000", 2024-09-11T06:50:30.651 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6696000000000001e-05 2024-09-11T06:50:30.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.651 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.651 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.099095+0000", 2024-09-11T06:50:30.651 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000184458 2024-09-11T06:50:30.651 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.652 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.652 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.652 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:50:30.652 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:50:30.652 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 7.5074264140000002, 2024-09-11T06:50:30.652 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0071924390000000001, 2024-09-11T06:50:30.652 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.652 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.653 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:30.653 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.653 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.653 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:50:30.653 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.653 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.653 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:50:30.653 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.653 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.654 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132020+0000", 2024-09-11T06:50:30.654 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.846e-06 2024-09-11T06:50:30.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.654 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.654 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135688+0000", 2024-09-11T06:50:30.654 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036677350000000001 2024-09-11T06:50:30.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.654 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.654 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135747+0000", 2024-09-11T06:50:30.654 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.9314999999999997e-05 2024-09-11T06:50:30.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.655 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.655 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135797+0000", 2024-09-11T06:50:30.655 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9984000000000001e-05 2024-09-11T06:50:30.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.655 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.655 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136143+0000", 2024-09-11T06:50:30.655 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034594299999999998 2024-09-11T06:50:30.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.655 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.655 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136179+0000", 2024-09-11T06:50:30.655 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5636000000000002e-05 2024-09-11T06:50:30.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.656 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.656 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.139211+0000", 2024-09-11T06:50:30.656 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030319800000000001 2024-09-11T06:50:30.656 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.656 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.656 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.656 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:50:30.656 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:50:30.656 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 7.5053631599999999, 2024-09-11T06:50:30.656 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064954139999999997, 2024-09-11T06:50:30.657 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.657 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.657 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:30.657 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.657 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.657 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:50:30.657 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.658 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.658 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:50:30.658 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.658 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.658 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134084+0000", 2024-09-11T06:50:30.658 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.894e-06 2024-09-11T06:50:30.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.659 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.659 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136513+0000", 2024-09-11T06:50:30.659 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024290969999999999 2024-09-11T06:50:30.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.659 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.659 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136554+0000", 2024-09-11T06:50:30.659 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.0932e-05 2024-09-11T06:50:30.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.659 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.659 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136584+0000", 2024-09-11T06:50:30.659 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0056000000000002e-05 2024-09-11T06:50:30.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.660 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.660 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136805+0000", 2024-09-11T06:50:30.660 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022086100000000001 2024-09-11T06:50:30.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.660 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.660 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136864+0000", 2024-09-11T06:50:30.660 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.9907000000000001e-05 2024-09-11T06:50:30.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.660 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.660 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.140577+0000", 2024-09-11T06:50:30.660 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0037126669999999998 2024-09-11T06:50:30.660 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.660 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.661 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.661 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.661 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:50:30.661 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:50:30.661 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 7.4970723599999998, 2024-09-11T06:50:30.661 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035763180000000002, 2024-09-11T06:50:30.661 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.661 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.661 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:30.661 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.661 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.661 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.662 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:50:30.662 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.662 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.662 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.662 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:50:30.662 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.662 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.662 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.662 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142374+0000", 2024-09-11T06:50:30.662 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5460000000000001e-06 2024-09-11T06:50:30.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.662 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.663 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.663 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143902+0000", 2024-09-11T06:50:30.663 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00152808 2024-09-11T06:50:30.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.663 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.663 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143946+0000", 2024-09-11T06:50:30.663 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3874e-05 2024-09-11T06:50:30.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.663 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.664 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143978+0000", 2024-09-11T06:50:30.664 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2425999999999998e-05 2024-09-11T06:50:30.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.664 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.664 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.664 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.144177+0000", 2024-09-11T06:50:30.664 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019837500000000001 2024-09-11T06:50:30.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.664 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.664 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.664 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.144198+0000", 2024-09-11T06:50:30.664 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1639000000000001e-05 2024-09-11T06:50:30.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.664 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.665 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.665 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.145949+0000", 2024-09-11T06:50:30.665 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0017503779999999999 2024-09-11T06:50:30.665 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.665 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.665 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.665 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.665 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.665 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:50:30.665 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:50:30.665 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 7.4927566580000002, 2024-09-11T06:50:30.665 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0057693220000000003, 2024-09-11T06:50:30.666 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.666 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.666 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:30.666 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.666 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.666 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.666 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:50:30.666 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.666 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.666 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.666 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.666 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:50:30.666 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.667 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.667 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.667 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146690+0000", 2024-09-11T06:50:30.667 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.607e-06 2024-09-11T06:50:30.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.667 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.667 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.667 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148693+0000", 2024-09-11T06:50:30.667 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0020029739999999998 2024-09-11T06:50:30.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.668 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.668 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148735+0000", 2024-09-11T06:50:30.668 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1958e-05 2024-09-11T06:50:30.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.668 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.668 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148764+0000", 2024-09-11T06:50:30.668 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8999000000000001e-05 2024-09-11T06:50:30.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.669 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.669 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149069+0000", 2024-09-11T06:50:30.669 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000305032 2024-09-11T06:50:30.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.669 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.669 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.669 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149204+0000", 2024-09-11T06:50:30.669 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013495 2024-09-11T06:50:30.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.669 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.669 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.670 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.152458+0000", 2024-09-11T06:50:30.670 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0032538020000000001 2024-09-11T06:50:30.670 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.670 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.670 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.670 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPushReply(2.7 61/60 [PushReplyOp(2:eab0161e:::rbd_data.1314f0a2bfa2.0000000000000001:head)])", 2024-09-11T06:50:30.670 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.147351+0000", 2024-09-11T06:50:30.670 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 7.4920941409999999, 2024-09-11T06:50:30.670 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0026083040000000001, 2024-09-11T06:50:30.670 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.670 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.670 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:30.671 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.671 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.671 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.671 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147351+0000", 2024-09-11T06:50:30.671 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.671 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.671 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.671 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147351+0000", 2024-09-11T06:50:30.671 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.671 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.671 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.672 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147352+0000", 2024-09-11T06:50:30.672 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.593e-06 2024-09-11T06:50:30.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.672 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.672 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.672 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147363+0000", 2024-09-11T06:50:30.672 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.111e-05 2024-09-11T06:50:30.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.672 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.672 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.672 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147385+0000", 2024-09-11T06:50:30.672 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1532e-05 2024-09-11T06:50:30.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.672 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.673 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.673 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147399+0000", 2024-09-11T06:50:30.673 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.448e-05 2024-09-11T06:50:30.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.673 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.673 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147572+0000", 2024-09-11T06:50:30.673 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000172423 2024-09-11T06:50:30.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.674 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.674 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149959+0000", 2024-09-11T06:50:30.674 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002387166 2024-09-11T06:50:30.674 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.674 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.674 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.674 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.674 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:50:30.674 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:50:30.674 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 7.4918581169999996, 2024-09-11T06:50:30.674 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0042425329999999997, 2024-09-11T06:50:30.675 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.675 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.675 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:30.675 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.675 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.675 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.675 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:50:30.675 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.675 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.675 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.675 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:50:30.675 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.676 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.676 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147588+0000", 2024-09-11T06:50:30.676 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.277e-06 2024-09-11T06:50:30.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.676 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.676 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149051+0000", 2024-09-11T06:50:30.676 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001462669 2024-09-11T06:50:30.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.676 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.677 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149090+0000", 2024-09-11T06:50:30.677 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.8967999999999997e-05 2024-09-11T06:50:30.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.677 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.677 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149120+0000", 2024-09-11T06:50:30.677 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0766000000000003e-05 2024-09-11T06:50:30.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.677 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.677 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149395+0000", 2024-09-11T06:50:30.677 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027484900000000002 2024-09-11T06:50:30.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.678 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.678 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.678 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149439+0000", 2024-09-11T06:50:30.678 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3940999999999998e-05 2024-09-11T06:50:30.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.678 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.678 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.678 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.151829+0000", 2024-09-11T06:50:30.678 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002390063 2024-09-11T06:50:30.679 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.679 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.679 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.679 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPushReply(2.7 61/60 [PushReplyOp(2:eab0161e:::rbd_data.1314f0a2bfa2.0000000000000001:head)])", 2024-09-11T06:50:30.679 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.152951+0000", 2024-09-11T06:50:30.679 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 7.4864936960000001, 2024-09-11T06:50:30.679 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002099357, 2024-09-11T06:50:30.679 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.679 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.679 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:30.679 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.680 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.680 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.152951+0000", 2024-09-11T06:50:30.680 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.680 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.680 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.152951+0000", 2024-09-11T06:50:30.680 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.680 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.681 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.152953+0000", 2024-09-11T06:50:30.681 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.854e-06 2024-09-11T06:50:30.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.681 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.681 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.152968+0000", 2024-09-11T06:50:30.681 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5096000000000001e-05 2024-09-11T06:50:30.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.681 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.681 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.152995+0000", 2024-09-11T06:50:30.682 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6432e-05 2024-09-11T06:50:30.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.682 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.682 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153015+0000", 2024-09-11T06:50:30.682 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0679999999999999e-05 2024-09-11T06:50:30.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.682 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.682 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153197+0000", 2024-09-11T06:50:30.682 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018195599999999999 2024-09-11T06:50:30.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.682 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.683 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155051+0000", 2024-09-11T06:50:30.683 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0018533390000000001 2024-09-11T06:50:30.683 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.683 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.683 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.683 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.683 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:50:30.683 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:50:30.683 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 7.4857174860000004, 2024-09-11T06:50:30.683 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0046568219999999997, 2024-09-11T06:50:30.684 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.684 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.684 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:30.684 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.684 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.684 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:50:30.684 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.685 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.685 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:50:30.685 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.685 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.685 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153729+0000", 2024-09-11T06:50:30.685 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5629999999999999e-06 2024-09-11T06:50:30.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.685 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.685 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155179+0000", 2024-09-11T06:50:30.686 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001450469 2024-09-11T06:50:30.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.686 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.686 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155221+0000", 2024-09-11T06:50:30.686 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1448e-05 2024-09-11T06:50:30.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.686 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.686 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155249+0000", 2024-09-11T06:50:30.686 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8433e-05 2024-09-11T06:50:30.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.686 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.686 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155420+0000", 2024-09-11T06:50:30.687 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017070599999999999 2024-09-11T06:50:30.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.687 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.687 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155445+0000", 2024-09-11T06:50:30.687 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4813000000000001e-05 2024-09-11T06:50:30.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.687 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.687 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.158384+0000", 2024-09-11T06:50:30.687 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0029393900000000001 2024-09-11T06:50:30.687 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.687 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.687 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.688 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:50:30.688 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:50:30.688 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 7.4836907369999999, 2024-09-11T06:50:30.688 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0046826020000000001, 2024-09-11T06:50:30.688 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.688 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.688 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:30.688 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.688 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.689 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:50:30.689 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.689 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.689 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:50:30.689 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.689 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.689 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155756+0000", 2024-09-11T06:50:30.689 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.384e-06 2024-09-11T06:50:30.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.690 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.690 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157205+0000", 2024-09-11T06:50:30.690 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014496260000000001 2024-09-11T06:50:30.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.690 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.690 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157252+0000", 2024-09-11T06:50:30.690 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6368000000000001e-05 2024-09-11T06:50:30.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.690 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.690 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157286+0000", 2024-09-11T06:50:30.690 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4944999999999998e-05 2024-09-11T06:50:30.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.691 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.691 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157540+0000", 2024-09-11T06:50:30.691 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00025307999999999998 2024-09-11T06:50:30.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.691 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.691 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157576+0000", 2024-09-11T06:50:30.691 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.6878999999999997e-05 2024-09-11T06:50:30.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.691 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.691 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.160437+0000", 2024-09-11T06:50:30.691 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00286032 2024-09-11T06:50:30.691 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.691 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.691 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.692 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPushReply(2.7 61/60 [PushReplyOp(2:eab0161e:::rbd_data.1314f0a2bfa2.0000000000000001:head)])", 2024-09-11T06:50:30.692 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.158551+0000", 2024-09-11T06:50:30.692 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 7.4808936670000001, 2024-09-11T06:50:30.692 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0023625930000000001, 2024-09-11T06:50:30.692 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.692 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.692 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:30.692 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.692 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.692 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.158551+0000", 2024-09-11T06:50:30.692 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.693 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.693 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.158551+0000", 2024-09-11T06:50:30.693 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.693 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.693 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.158553+0000", 2024-09-11T06:50:30.693 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9690000000000001e-06 2024-09-11T06:50:30.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.693 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.694 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.158566+0000", 2024-09-11T06:50:30.694 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.294e-05 2024-09-11T06:50:30.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.694 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.694 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.158603+0000", 2024-09-11T06:50:30.694 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7202000000000001e-05 2024-09-11T06:50:30.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.695 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.695 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.158622+0000", 2024-09-11T06:50:30.695 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8335999999999999e-05 2024-09-11T06:50:30.695 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.695 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.695 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.695 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.158767+0000", 2024-09-11T06:50:30.695 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000145544 2024-09-11T06:50:30.695 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.695 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.695 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.695 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.160914+0000", 2024-09-11T06:50:30.695 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021466020000000001 2024-09-11T06:50:30.696 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.696 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.696 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.696 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.696 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:774f7988:ns::rbd_directory:head, version: 54'3, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 3, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:774f7988:ns::rbd_directory:head@54'3, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:50:30.696 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.160495+0000", 2024-09-11T06:50:30.696 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 7.4789501070000002, 2024-09-11T06:50:30.696 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012141890000000001, 2024-09-11T06:50:30.696 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.696 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.696 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:30.697 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.697 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.697 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.697 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.160495+0000", 2024-09-11T06:50:30.697 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.697 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.697 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.697 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.160495+0000", 2024-09-11T06:50:30.697 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.697 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.697 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.698 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.160496+0000", 2024-09-11T06:50:30.698 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.483e-06 2024-09-11T06:50:30.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.698 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.698 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.698 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.160533+0000", 2024-09-11T06:50:30.698 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.6641999999999999e-05 2024-09-11T06:50:30.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.698 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.698 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.698 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.160590+0000", 2024-09-11T06:50:30.699 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.6774000000000002e-05 2024-09-11T06:50:30.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.699 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.699 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.160663+0000", 2024-09-11T06:50:30.699 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3233999999999997e-05 2024-09-11T06:50:30.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.699 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.699 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.160878+0000", 2024-09-11T06:50:30.700 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021538500000000001 2024-09-11T06:50:30.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.700 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.700 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.700 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.160918+0000", 2024-09-11T06:50:30.700 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.9715000000000002e-05 2024-09-11T06:50:30.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.700 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.700 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.700 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.161709+0000", 2024-09-11T06:50:30.700 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00079095599999999999 2024-09-11T06:50:30.700 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.701 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.701 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.701 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.701 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.701 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5055.0:93 2.7 2:eab0161e:::rbd_data.1314f0a2bfa2.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-11T06:50:30.701 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:25.654406+0000", 2024-09-11T06:50:30.701 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.9850386599999998, 2024-09-11T06:50:30.701 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00077837000000000002, 2024-09-11T06:50:30.702 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:30.702 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:30.702 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:30.702 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:30.702 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5055", 2024-09-11T06:50:30.702 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2224097229", 2024-09-11T06:50:30.702 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 93 2024-09-11T06:50:30.702 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.702 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:30.702 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.702 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:30.702 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:25.654406+0000", 2024-09-11T06:50:30.702 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.702 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.703 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.703 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:30.703 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:25.654406+0000", 2024-09-11T06:50:30.703 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:30.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.703 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.703 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:30.703 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:25.654409+0000", 2024-09-11T06:50:30.703 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4260000000000002e-06 2024-09-11T06:50:30.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.703 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.703 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:30.703 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:25.654423+0000", 2024-09-11T06:50:30.703 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4433000000000001e-05 2024-09-11T06:50:30.704 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.704 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.704 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:30.704 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:25.654454+0000", 2024-09-11T06:50:30.704 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.1375999999999999e-05 2024-09-11T06:50:30.704 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.704 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.704 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:30.705 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:25.654470+0000", 2024-09-11T06:50:30.705 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5818999999999999e-05 2024-09-11T06:50:30.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.705 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.705 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:30.705 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:25.654621+0000", 2024-09-11T06:50:30.705 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015119100000000001 2024-09-11T06:50:30.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.705 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.705 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:30.705 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:25.654859+0000", 2024-09-11T06:50:30.705 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000237307 2024-09-11T06:50:30.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:30.706 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:30.706 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:30.706 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:25.655185+0000", 2024-09-11T06:50:30.706 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00032581800000000002 2024-09-11T06:50:30.706 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.706 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.706 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.706 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:30.706 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:30.706 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:30.707 INFO:tasks.workunit.client.0.smithi176.stderr:+ blocks=65536 2024-09-11T06:50:30.708 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 65536 2024-09-11T06:50:30.708 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd resize rbd/testrbdnbd21607 --size 128M 2024-09-11T06:50:30.708 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:50:30.709 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:30.707+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:30.723 INFO:tasks.workunit.client.0.smithi176.stderr: Resizing image: 100% complete...done. 2024-09-11T06:50:30.741 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd info rbd/testrbdnbd21607 2024-09-11T06:50:30.802 INFO:tasks.workunit.client.0.smithi176.stdout:rbd image 'testrbdnbd21607': 2024-09-11T06:50:30.802 INFO:tasks.workunit.client.0.smithi176.stdout: size 128 MiB in 32 objects 2024-09-11T06:50:30.802 INFO:tasks.workunit.client.0.smithi176.stdout: order 22 (4 MiB objects) 2024-09-11T06:50:30.803 INFO:tasks.workunit.client.0.smithi176.stdout: snapshot_count: 0 2024-09-11T06:50:30.803 INFO:tasks.workunit.client.0.smithi176.stdout: id: 1314f0a2bfa2 2024-09-11T06:50:30.803 INFO:tasks.workunit.client.0.smithi176.stdout: block_name_prefix: rbd_data.1314f0a2bfa2 2024-09-11T06:50:30.803 INFO:tasks.workunit.client.0.smithi176.stdout: format: 2 2024-09-11T06:50:30.803 INFO:tasks.workunit.client.0.smithi176.stdout: features: layering, exclusive-lock, object-map, fast-diff, deep-flatten 2024-09-11T06:50:30.803 INFO:tasks.workunit.client.0.smithi176.stdout: op_features: 2024-09-11T06:50:30.803 INFO:tasks.workunit.client.0.smithi176.stdout: flags: 2024-09-11T06:50:30.803 INFO:tasks.workunit.client.0.smithi176.stdout: create_timestamp: Wed Sep 11 06:50:14 2024 2024-09-11T06:50:30.803 INFO:tasks.workunit.client.0.smithi176.stdout: access_timestamp: Wed Sep 11 06:50:14 2024 2024-09-11T06:50:30.803 INFO:tasks.workunit.client.0.smithi176.stdout: modify_timestamp: Wed Sep 11 06:50:14 2024 2024-09-11T06:50:30.807 INFO:tasks.workunit.client.0.smithi176.stderr:++ awk -v dev=nbd0 '$4 == dev {print $3}' /proc/partitions 2024-09-11T06:50:30.808 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:30.803+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:30.809 INFO:tasks.workunit.client.0.smithi176.stderr:+ blocks2=131072 2024-09-11T06:50:30.809 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 131072 2024-09-11T06:50:30.809 INFO:tasks.workunit.client.0.smithi176.stderr:+ test 131072 -eq 131072 2024-09-11T06:50:30.809 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd resize rbd/testrbdnbd21607 --allow-shrink --size 64M 2024-09-11T06:50:30.879 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:30.879 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:30.879 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:50:30.879 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:30.884 INFO:tasks.workunit.client.0.smithi176.stderr: Resizing image: 50% complete... Resizing image: 53% complete... Resizing image: 56% complete... Resizing image: 59% complete... Resizing image: 62% complete... Resizing image: 65% complete... Resizing image: 68% complete... Resizing image: 71% complete... Resizing image: 75% complete... Resizing image: 78% complete... Resizing image: 81% complete... Resizing image: 84% complete... Resizing image: 87% complete... Resizing image: 90% complete... Resizing image: 93% complete... Resizing image: 96% complete... Resizing image: 100% complete...done. 2024-09-11T06:50:30.893 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:50:30.902 INFO:tasks.workunit.client.0.smithi176.stderr:++ awk -v dev=nbd0 '$4 == dev {print $3}' /proc/partitions 2024-09-11T06:50:30.905 INFO:tasks.workunit.client.0.smithi176.stderr:+ blocks2=65536 2024-09-11T06:50:30.906 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 65536 2024-09-11T06:50:30.906 INFO:tasks.workunit.client.0.smithi176.stderr:+ test 65536 -eq 65536 2024-09-11T06:50:30.906 INFO:tasks.workunit.client.0.smithi176.stderr:+ unmap_device /dev/nbd0 22433 2024-09-11T06:50:30.906 INFO:tasks.workunit.client.0.smithi176.stderr:+ local args=/dev/nbd0 2024-09-11T06:50:30.906 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pid=22433 2024-09-11T06:50:30.906 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd0 2024-09-11T06:50:30.906 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:30.906 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:30.906 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:30.907 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:50:30.908 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:50:30.908 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:30.908 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd0 2024-09-11T06:50:30.909 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:30.907+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:30.994 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:50:30.994 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^22433\b' 2024-09-11T06:50:30.994 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^22433\b' 2024-09-11T06:50:31.009 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:31.007+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:31.015 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:31.015 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:31.015 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:50:31.015 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:50:31.015 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:31.029 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:50:31.043 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:31.044 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -C rbd-nbd 2024-09-11T06:50:31.044 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^ *22433\b' 2024-09-11T06:50:31.044 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^ *22433\b' 2024-09-11T06:50:31.052 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:31.052 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 0.5 2024-09-11T06:50:31.110 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:31.107+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:31.147 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:31.148 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:31.148 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:31.148 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:50:31.148 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.148 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:50:31.148 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:50:31.148 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 79.916867726000007, 2024-09-11T06:50:31.148 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010553425999999999, 2024-09-11T06:50:31.149 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:31.149 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:31.149 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:31.149 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:31.149 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:31.149 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:31.149 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:50:31.149 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.149 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:31.149 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.149 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:31.149 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:50:31.149 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.149 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.150 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.150 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:31.150 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:50:31.150 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.150 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.150 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.150 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:31.150 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230274+0000", 2024-09-11T06:50:31.150 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.533e-06 2024-09-11T06:50:31.150 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.150 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.150 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:31.150 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230285+0000", 2024-09-11T06:50:31.150 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0540000000000001e-05 2024-09-11T06:50:31.151 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.151 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.151 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:31.151 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230309+0000", 2024-09-11T06:50:31.151 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4223000000000002e-05 2024-09-11T06:50:31.151 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.151 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.151 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.151 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230327+0000", 2024-09-11T06:50:31.151 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:50:31.151 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.151 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.151 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:31.152 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230478+0000", 2024-09-11T06:50:31.152 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000151459 2024-09-11T06:50:31.152 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.152 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.152 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:31.152 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230490+0000", 2024-09-11T06:50:31.152 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.147e-05 2024-09-11T06:50:31.152 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.152 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.152 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:31.152 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231803+0000", 2024-09-11T06:50:31.152 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001313158 2024-09-11T06:50:31.152 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.153 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.153 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:50:31.153 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231817+0000", 2024-09-11T06:50:31.153 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4197e-05 2024-09-11T06:50:31.153 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.153 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.153 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:31.153 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.235945+0000", 2024-09-11T06:50:31.153 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0041282480000000002 2024-09-11T06:50:31.153 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.154 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.154 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:31.154 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236312+0000", 2024-09-11T06:50:31.154 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036652700000000001 2024-09-11T06:50:31.154 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.154 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.154 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:50:31.154 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236689+0000", 2024-09-11T06:50:31.154 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000377117 2024-09-11T06:50:31.154 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.154 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.154 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:31.154 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238460+0000", 2024-09-11T06:50:31.154 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001770918 2024-09-11T06:50:31.155 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.155 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.155 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:31.155 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240627+0000", 2024-09-11T06:50:31.155 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021670779999999998 2024-09-11T06:50:31.155 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.155 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.155 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:31.155 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240751+0000", 2024-09-11T06:50:31.155 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012384900000000001 2024-09-11T06:50:31.155 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.155 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.155 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:31.156 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240821+0000", 2024-09-11T06:50:31.156 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0516e-05 2024-09-11T06:50:31.156 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.156 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:31.156 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.156 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.156 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.156 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-11T06:50:31.156 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:50:31.156 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 78.914696883000005, 2024-09-11T06:50:31.156 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069904770000000001, 2024-09-11T06:50:31.156 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:31.156 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:31.156 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:31.157 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:31.157 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:31.157 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:31.157 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 5 2024-09-11T06:50:31.157 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.157 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:31.157 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.157 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:31.157 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:50:31.157 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.157 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.157 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.157 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:31.158 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:50:31.158 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.158 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.158 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.158 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:31.158 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232442+0000", 2024-09-11T06:50:31.158 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.044e-06 2024-09-11T06:50:31.158 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.158 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.158 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:31.158 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232460+0000", 2024-09-11T06:50:31.158 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7863000000000002e-05 2024-09-11T06:50:31.159 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.159 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.159 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:31.159 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232492+0000", 2024-09-11T06:50:31.159 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2812999999999999e-05 2024-09-11T06:50:31.159 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.159 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.159 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.159 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232510+0000", 2024-09-11T06:50:31.160 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7552999999999999e-05 2024-09-11T06:50:31.160 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.160 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.160 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:31.160 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232640+0000", 2024-09-11T06:50:31.160 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013029900000000001 2024-09-11T06:50:31.160 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.160 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.160 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:31.160 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232650+0000", 2024-09-11T06:50:31.160 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0023999999999999e-05 2024-09-11T06:50:31.160 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.160 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.161 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:31.161 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235641+0000", 2024-09-11T06:50:31.161 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00299089 2024-09-11T06:50:31.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.161 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.161 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:31.161 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235980+0000", 2024-09-11T06:50:31.161 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000338907 2024-09-11T06:50:31.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.161 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.161 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:31.161 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.236351+0000", 2024-09-11T06:50:31.161 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037087699999999998 2024-09-11T06:50:31.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.162 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.162 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:31.162 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238074+0000", 2024-09-11T06:50:31.162 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001722859 2024-09-11T06:50:31.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.162 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.162 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:31.162 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239160+0000", 2024-09-11T06:50:31.162 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001086074 2024-09-11T06:50:31.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.162 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.162 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:31.162 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239299+0000", 2024-09-11T06:50:31.163 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013933500000000001 2024-09-11T06:50:31.163 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.163 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.163 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:31.163 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239429+0000", 2024-09-11T06:50:31.163 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129939 2024-09-11T06:50:31.163 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.163 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:31.163 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.163 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.163 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.163 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-11T06:50:31.163 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:50:31.163 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 77.900608930999994, 2024-09-11T06:50:31.163 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069253250000000004, 2024-09-11T06:50:31.164 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:31.164 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:31.164 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:31.164 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:31.164 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:31.164 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:31.164 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 7 2024-09-11T06:50:31.164 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.164 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:31.164 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.164 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:31.164 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:50:31.165 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.165 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.165 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:31.165 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:50:31.165 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.165 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.165 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:31.165 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246530+0000", 2024-09-11T06:50:31.165 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9469999999999999e-06 2024-09-11T06:50:31.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.165 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.165 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:31.165 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246545+0000", 2024-09-11T06:50:31.166 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5973000000000001e-05 2024-09-11T06:50:31.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.166 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.166 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:31.166 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246578+0000", 2024-09-11T06:50:31.166 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2759999999999998e-05 2024-09-11T06:50:31.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.166 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.166 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.166 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246595+0000", 2024-09-11T06:50:31.166 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6939000000000002e-05 2024-09-11T06:50:31.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.166 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.166 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:31.166 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246770+0000", 2024-09-11T06:50:31.167 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000174723 2024-09-11T06:50:31.167 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.167 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.167 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:31.167 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246789+0000", 2024-09-11T06:50:31.167 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9159000000000001e-05 2024-09-11T06:50:31.167 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.167 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.167 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:31.167 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250337+0000", 2024-09-11T06:50:31.167 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035480759999999998 2024-09-11T06:50:31.167 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.167 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.167 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:31.167 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250549+0000", 2024-09-11T06:50:31.167 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021192199999999999 2024-09-11T06:50:31.168 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.168 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.168 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:31.168 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250750+0000", 2024-09-11T06:50:31.168 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020104199999999999 2024-09-11T06:50:31.168 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.168 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.168 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:31.168 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.252455+0000", 2024-09-11T06:50:31.168 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001705249 2024-09-11T06:50:31.168 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.168 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.168 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:31.168 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253305+0000", 2024-09-11T06:50:31.168 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00084941200000000002 2024-09-11T06:50:31.169 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.169 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.169 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:31.169 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253388+0000", 2024-09-11T06:50:31.169 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3642999999999999e-05 2024-09-11T06:50:31.169 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.169 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.169 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:31.169 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253452+0000", 2024-09-11T06:50:31.169 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3479999999999995e-05 2024-09-11T06:50:31.169 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.169 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:31.169 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.169 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.170 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.170 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(client.4558.0:5 3.2 e57/55 3:5cea7035:::rbd_info:head v 57'1, mlcod=0'0)", 2024-09-11T06:50:31.170 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:50:31.170 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 13.052504569, 2024-09-11T06:50:31.170 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011016419999999999, 2024-09-11T06:50:31.170 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:31.170 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:31.170 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:31.170 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:31.170 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.170 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:31.170 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:50:31.170 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.170 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.171 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.171 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:31.171 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:50:31.171 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.171 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.171 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.171 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:31.171 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:50:31.171 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4000000000000002e-07 2024-09-11T06:50:31.171 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.171 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.171 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:31.171 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094640+0000", 2024-09-11T06:50:31.171 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6230000000000003e-06 2024-09-11T06:50:31.171 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.172 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.172 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:31.172 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094669+0000", 2024-09-11T06:50:31.172 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8462e-05 2024-09-11T06:50:31.172 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.172 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.172 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.172 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094684+0000", 2024-09-11T06:50:31.172 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5536000000000002e-05 2024-09-11T06:50:31.172 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.172 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.172 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:31.172 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094798+0000", 2024-09-11T06:50:31.172 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011439599999999999 2024-09-11T06:50:31.173 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.173 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.173 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:31.173 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094819+0000", 2024-09-11T06:50:31.173 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0296e-05 2024-09-11T06:50:31.173 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.173 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.173 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:31.173 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.095687+0000", 2024-09-11T06:50:31.173 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086851799999999998 2024-09-11T06:50:31.173 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.173 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.173 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:31.173 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.095733+0000", 2024-09-11T06:50:31.174 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5271000000000001e-05 2024-09-11T06:50:31.174 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.174 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:31.174 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.174 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.174 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.174 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:160 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 4096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:31.174 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:50:31.174 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 10.113389562, 2024-09-11T06:50:31.174 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001122215, 2024-09-11T06:50:31.174 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:31.174 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:31.174 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:31.174 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:31.175 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:31.175 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:31.175 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 160 2024-09-11T06:50:31.175 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.175 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:31.175 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.175 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:31.175 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:50:31.175 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.175 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.175 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.175 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:31.175 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:50:31.176 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.176 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.176 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.176 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:31.176 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:50:31.176 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0999999999999999e-07 2024-09-11T06:50:31.176 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.176 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.176 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:31.176 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033749+0000", 2024-09-11T06:50:31.176 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8930000000000001e-06 2024-09-11T06:50:31.176 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.176 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.176 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:31.177 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033758+0000", 2024-09-11T06:50:31.177 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.242e-06 2024-09-11T06:50:31.177 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.177 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.177 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.177 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033764+0000", 2024-09-11T06:50:31.177 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0510000000000002e-06 2024-09-11T06:50:31.177 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.177 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.177 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:31.177 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.034207+0000", 2024-09-11T06:50:31.177 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00044320600000000003 2024-09-11T06:50:31.177 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.177 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.178 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:31.178 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.034280+0000", 2024-09-11T06:50:31.178 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3094000000000002e-05 2024-09-11T06:50:31.178 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.178 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.178 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:31.178 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.034868+0000", 2024-09-11T06:50:31.178 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00058821899999999996 2024-09-11T06:50:31.178 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.178 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:31.178 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.178 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.178 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.178 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:162 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:31.178 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.039519+0000", 2024-09-11T06:50:31.179 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 10.107616444, 2024-09-11T06:50:31.179 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001298485, 2024-09-11T06:50:31.179 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:31.179 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:31.179 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:31.179 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:31.179 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:31.179 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:31.179 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 162 2024-09-11T06:50:31.179 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.179 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:31.179 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.179 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:31.179 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039519+0000", 2024-09-11T06:50:31.179 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.180 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.180 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.180 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:31.180 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039519+0000", 2024-09-11T06:50:31.180 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.180 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.180 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.180 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:31.180 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039520+0000", 2024-09-11T06:50:31.180 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9299999999999998e-07 2024-09-11T06:50:31.180 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.181 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.181 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:31.181 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039523+0000", 2024-09-11T06:50:31.181 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.247e-06 2024-09-11T06:50:31.181 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.181 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.181 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:31.181 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039529+0000", 2024-09-11T06:50:31.181 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8300000000000001e-06 2024-09-11T06:50:31.181 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.181 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.181 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.181 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039533+0000", 2024-09-11T06:50:31.182 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.2379999999999999e-06 2024-09-11T06:50:31.182 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.182 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.182 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:31.182 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.040236+0000", 2024-09-11T06:50:31.182 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00070326500000000001 2024-09-11T06:50:31.182 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.182 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.182 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:31.182 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.040299+0000", 2024-09-11T06:50:31.182 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3077999999999995e-05 2024-09-11T06:50:31.182 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.182 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.182 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:31.183 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.040818+0000", 2024-09-11T06:50:31.183 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00051833399999999996 2024-09-11T06:50:31.183 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.183 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:31.183 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.183 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.183 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.183 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:164 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 528384~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:31.183 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.041536+0000", 2024-09-11T06:50:31.183 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 10.105599721999999, 2024-09-11T06:50:31.183 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00119962, 2024-09-11T06:50:31.183 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:31.183 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:31.183 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:31.183 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:31.183 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:31.184 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:31.184 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 164 2024-09-11T06:50:31.184 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.184 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:31.184 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.184 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:31.184 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041536+0000", 2024-09-11T06:50:31.184 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.184 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.184 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.184 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:31.184 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041536+0000", 2024-09-11T06:50:31.184 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.185 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.185 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.185 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:31.185 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041537+0000", 2024-09-11T06:50:31.185 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3e-06 2024-09-11T06:50:31.185 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.185 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.185 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:31.185 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041543+0000", 2024-09-11T06:50:31.185 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2360000000000001e-06 2024-09-11T06:50:31.185 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.185 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.185 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:31.186 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041558+0000", 2024-09-11T06:50:31.186 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4909000000000001e-05 2024-09-11T06:50:31.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.186 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.186 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.186 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041569+0000", 2024-09-11T06:50:31.186 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0946e-05 2024-09-11T06:50:31.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.186 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.186 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:31.186 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.042138+0000", 2024-09-11T06:50:31.186 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000568996 2024-09-11T06:50:31.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.187 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.187 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:31.187 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.042190+0000", 2024-09-11T06:50:31.187 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1609e-05 2024-09-11T06:50:31.187 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.187 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.187 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:31.187 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.042735+0000", 2024-09-11T06:50:31.187 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00054562399999999998 2024-09-11T06:50:31.187 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.187 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:31.187 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.187 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.187 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.187 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:166 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 790528~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:31.188 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.043564+0000", 2024-09-11T06:50:31.188 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 10.103571325000001, 2024-09-11T06:50:31.188 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011042000000000001, 2024-09-11T06:50:31.188 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:31.188 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:31.188 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:31.188 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:31.188 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:31.188 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:31.188 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 166 2024-09-11T06:50:31.188 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.188 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:31.188 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.188 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:31.189 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043564+0000", 2024-09-11T06:50:31.189 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.189 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.189 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:31.189 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043564+0000", 2024-09-11T06:50:31.189 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.189 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.189 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:31.189 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043565+0000", 2024-09-11T06:50:31.189 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0449999999999999e-06 2024-09-11T06:50:31.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.189 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.189 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:31.190 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043571+0000", 2024-09-11T06:50:31.190 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.6849999999999998e-06 2024-09-11T06:50:31.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.190 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.190 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:31.190 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043586+0000", 2024-09-11T06:50:31.190 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4902000000000001e-05 2024-09-11T06:50:31.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.190 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.190 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.190 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043593+0000", 2024-09-11T06:50:31.190 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9240000000000001e-06 2024-09-11T06:50:31.191 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.191 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.191 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:31.191 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.044073+0000", 2024-09-11T06:50:31.191 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000479813 2024-09-11T06:50:31.191 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.191 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.191 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:31.191 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.044123+0000", 2024-09-11T06:50:31.191 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0908000000000001e-05 2024-09-11T06:50:31.191 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.191 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.191 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:31.191 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.044668+0000", 2024-09-11T06:50:31.192 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000544923 2024-09-11T06:50:31.192 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.192 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:31.192 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.192 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.192 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.192 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:178 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 2363392~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:31.192 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.061599+0000", 2024-09-11T06:50:31.192 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 10.08553661, 2024-09-11T06:50:31.192 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012194529999999999, 2024-09-11T06:50:31.192 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:31.192 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:31.192 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:31.192 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:31.193 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:31.193 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:31.193 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 178 2024-09-11T06:50:31.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.193 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:31.193 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.193 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:31.193 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061599+0000", 2024-09-11T06:50:31.193 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.193 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.193 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:31.193 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061599+0000", 2024-09-11T06:50:31.193 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.194 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.194 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:31.194 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061600+0000", 2024-09-11T06:50:31.194 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3600000000000003e-07 2024-09-11T06:50:31.194 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.194 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.194 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:31.194 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061603+0000", 2024-09-11T06:50:31.194 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.6550000000000002e-06 2024-09-11T06:50:31.194 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.194 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.194 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:31.194 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061618+0000", 2024-09-11T06:50:31.194 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4377e-05 2024-09-11T06:50:31.195 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.195 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.195 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.195 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061626+0000", 2024-09-11T06:50:31.195 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.0679999999999997e-06 2024-09-11T06:50:31.195 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.195 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.195 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:31.195 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.062098+0000", 2024-09-11T06:50:31.195 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00047251299999999998 2024-09-11T06:50:31.195 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.195 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.195 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:31.195 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.062253+0000", 2024-09-11T06:50:31.195 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015533100000000001 2024-09-11T06:50:31.196 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.196 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.196 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:31.196 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.062818+0000", 2024-09-11T06:50:31.196 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00056487300000000002 2024-09-11T06:50:31.196 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.196 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:31.196 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.196 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.196 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.196 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:184 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 3149824~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:31.197 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:50:31.197 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 10.080926907, 2024-09-11T06:50:31.197 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010685969999999999, 2024-09-11T06:50:31.197 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:31.197 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:31.197 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:31.197 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:31.197 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:31.197 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:31.197 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 184 2024-09-11T06:50:31.197 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.197 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:31.197 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.197 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:31.197 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:50:31.198 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.198 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.198 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.198 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:31.198 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:50:31.198 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.198 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.198 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.198 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:31.198 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:50:31.198 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.6700000000000003e-07 2024-09-11T06:50:31.198 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.198 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.198 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:31.198 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066212+0000", 2024-09-11T06:50:31.198 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.1669999999999999e-06 2024-09-11T06:50:31.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.199 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.199 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:31.199 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066219+0000", 2024-09-11T06:50:31.199 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5699999999999998e-06 2024-09-11T06:50:31.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.199 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.199 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.199 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066223+0000", 2024-09-11T06:50:31.199 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3540000000000002e-06 2024-09-11T06:50:31.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.199 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.199 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:31.199 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066713+0000", 2024-09-11T06:50:31.200 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00048985399999999996 2024-09-11T06:50:31.200 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.200 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.200 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:31.200 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066813+0000", 2024-09-11T06:50:31.200 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00010003 2024-09-11T06:50:31.200 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.200 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.200 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:31.200 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.067277+0000", 2024-09-11T06:50:31.200 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000463955 2024-09-11T06:50:31.200 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.200 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:31.200 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.200 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.200 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.201 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:186 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 3411968~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:31.201 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.071198+0000", 2024-09-11T06:50:31.201 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 10.075937563, 2024-09-11T06:50:31.201 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012515779999999999, 2024-09-11T06:50:31.201 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:31.201 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:31.201 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:31.201 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:31.201 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:31.201 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:31.201 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 186 2024-09-11T06:50:31.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.201 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:31.201 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.202 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:31.202 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071198+0000", 2024-09-11T06:50:31.202 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.202 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.202 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.202 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:31.202 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071198+0000", 2024-09-11T06:50:31.202 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.202 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.202 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.202 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:31.202 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071199+0000", 2024-09-11T06:50:31.202 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0299999999999999e-07 2024-09-11T06:50:31.202 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.202 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.202 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:31.203 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071201+0000", 2024-09-11T06:50:31.203 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8540000000000001e-06 2024-09-11T06:50:31.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.203 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.203 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:31.203 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071209+0000", 2024-09-11T06:50:31.203 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3629999999999996e-06 2024-09-11T06:50:31.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.203 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.203 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.203 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071214+0000", 2024-09-11T06:50:31.203 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2109999999999997e-06 2024-09-11T06:50:31.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.203 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.203 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:31.203 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071915+0000", 2024-09-11T06:50:31.204 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00070101799999999998 2024-09-11T06:50:31.204 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.204 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.204 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:31.204 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.072011+0000", 2024-09-11T06:50:31.204 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.5755999999999996e-05 2024-09-11T06:50:31.204 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.204 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.204 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:31.204 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.072450+0000", 2024-09-11T06:50:31.204 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000438773 2024-09-11T06:50:31.204 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.204 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:31.204 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.204 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.204 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.205 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:190 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 3936256~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:31.205 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.074875+0000", 2024-09-11T06:50:31.205 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 10.072260648, 2024-09-11T06:50:31.205 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0013119620000000001, 2024-09-11T06:50:31.205 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:31.205 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:31.205 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:31.205 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:31.205 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:31.205 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:31.205 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 190 2024-09-11T06:50:31.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.205 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:31.205 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.205 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:31.206 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074875+0000", 2024-09-11T06:50:31.206 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.206 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.206 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.206 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:31.206 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074875+0000", 2024-09-11T06:50:31.206 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.206 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.206 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.206 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:31.206 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074876+0000", 2024-09-11T06:50:31.206 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7400000000000003e-07 2024-09-11T06:50:31.206 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.206 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.206 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:31.207 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074887+0000", 2024-09-11T06:50:31.207 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1817999999999999e-05 2024-09-11T06:50:31.207 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.207 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.207 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:31.207 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074896+0000", 2024-09-11T06:50:31.207 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9120000000000004e-06 2024-09-11T06:50:31.207 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.207 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.207 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.207 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074900+0000", 2024-09-11T06:50:31.207 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.2220000000000004e-06 2024-09-11T06:50:31.207 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.207 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.208 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:31.208 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.075198+0000", 2024-09-11T06:50:31.208 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00029750399999999998 2024-09-11T06:50:31.208 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.208 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.208 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:31.208 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.075430+0000", 2024-09-11T06:50:31.208 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023200400000000001 2024-09-11T06:50:31.208 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.208 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.208 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:31.208 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.076187+0000", 2024-09-11T06:50:31.208 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00075682799999999997 2024-09-11T06:50:31.208 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.208 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:31.209 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.209 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.209 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.209 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:359 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 135168~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:31.209 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.298832+0000", 2024-09-11T06:50:31.209 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 9.8483030140000007, 2024-09-11T06:50:31.209 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001078575, 2024-09-11T06:50:31.209 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:31.209 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:31.209 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:31.209 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:31.209 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:31.209 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:31.210 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 359 2024-09-11T06:50:31.210 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.210 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:31.210 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.210 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:31.210 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298832+0000", 2024-09-11T06:50:31.210 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.210 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.210 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.210 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:31.210 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298832+0000", 2024-09-11T06:50:31.210 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.210 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.211 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.211 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:31.211 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298834+0000", 2024-09-11T06:50:31.211 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.646e-06 2024-09-11T06:50:31.211 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.211 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.211 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:31.211 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298845+0000", 2024-09-11T06:50:31.211 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.081e-05 2024-09-11T06:50:31.211 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.211 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.211 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:31.211 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298872+0000", 2024-09-11T06:50:31.212 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7382e-05 2024-09-11T06:50:31.212 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.212 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.212 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.212 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298884+0000", 2024-09-11T06:50:31.212 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1956e-05 2024-09-11T06:50:31.212 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.212 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.212 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:31.212 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298959+0000", 2024-09-11T06:50:31.212 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5214000000000002e-05 2024-09-11T06:50:31.212 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.212 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.212 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:31.213 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.299090+0000", 2024-09-11T06:50:31.213 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000130247 2024-09-11T06:50:31.213 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.213 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.213 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:31.213 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.299911+0000", 2024-09-11T06:50:31.213 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00082131999999999995 2024-09-11T06:50:31.213 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.213 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:31.213 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.213 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.213 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.213 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:360 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:31.213 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.298914+0000", 2024-09-11T06:50:31.213 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 9.8482213440000006, 2024-09-11T06:50:31.214 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0023571690000000001, 2024-09-11T06:50:31.214 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:31.214 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:31.214 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:31.214 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:31.214 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:31.214 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:31.214 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 360 2024-09-11T06:50:31.214 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.214 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:31.214 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.214 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:31.214 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298914+0000", 2024-09-11T06:50:31.214 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.214 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.214 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.215 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:31.215 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298914+0000", 2024-09-11T06:50:31.215 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.215 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.215 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.215 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:31.215 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298915+0000", 2024-09-11T06:50:31.215 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9300000000000006e-07 2024-09-11T06:50:31.215 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.215 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.215 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:31.215 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298921+0000", 2024-09-11T06:50:31.215 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.6030000000000004e-06 2024-09-11T06:50:31.215 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.216 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.216 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:31.216 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298932+0000", 2024-09-11T06:50:31.216 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.082e-05 2024-09-11T06:50:31.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.216 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.216 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.216 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298939+0000", 2024-09-11T06:50:31.216 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7919999999999993e-06 2024-09-11T06:50:31.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.216 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.216 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:31.216 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.300011+0000", 2024-09-11T06:50:31.217 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001071179 2024-09-11T06:50:31.217 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.217 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.217 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:31.217 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.300245+0000", 2024-09-11T06:50:31.217 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000234434 2024-09-11T06:50:31.217 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.217 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.217 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:31.217 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.301271+0000", 2024-09-11T06:50:31.217 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010263480000000001 2024-09-11T06:50:31.217 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.217 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:31.217 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.218 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.218 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.218 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:362 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 528384~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:31.218 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.302376+0000", 2024-09-11T06:50:31.218 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 9.8447590209999998, 2024-09-11T06:50:31.218 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014132909999999999, 2024-09-11T06:50:31.218 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:31.218 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:31.218 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:31.218 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:31.218 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:31.218 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:31.218 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 362 2024-09-11T06:50:31.218 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.218 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:31.218 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.219 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:31.219 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302376+0000", 2024-09-11T06:50:31.219 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.219 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.219 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:31.219 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302376+0000", 2024-09-11T06:50:31.219 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.219 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.219 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:31.219 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302378+0000", 2024-09-11T06:50:31.219 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1629999999999999e-06 2024-09-11T06:50:31.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.219 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.219 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:31.220 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302383+0000", 2024-09-11T06:50:31.220 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.463e-06 2024-09-11T06:50:31.220 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.220 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.220 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:31.220 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302401+0000", 2024-09-11T06:50:31.220 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8219e-05 2024-09-11T06:50:31.220 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.220 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.220 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.220 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302411+0000", 2024-09-11T06:50:31.220 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9000000000000001e-06 2024-09-11T06:50:31.220 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.220 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.221 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:31.221 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302524+0000", 2024-09-11T06:50:31.221 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000113199 2024-09-11T06:50:31.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.221 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.221 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:31.221 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302745+0000", 2024-09-11T06:50:31.221 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022057399999999999 2024-09-11T06:50:31.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.221 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.221 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:31.221 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.303790+0000", 2024-09-11T06:50:31.221 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010447729999999999 2024-09-11T06:50:31.221 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.221 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:31.222 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.222 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.222 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:363 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 659456~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:31.222 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.302938+0000", 2024-09-11T06:50:31.222 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 9.8441974279999993, 2024-09-11T06:50:31.222 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00185012, 2024-09-11T06:50:31.222 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:31.222 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:31.222 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:31.222 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:31.222 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:31.222 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:31.222 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 363 2024-09-11T06:50:31.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.222 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:31.222 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.223 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:31.223 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302938+0000", 2024-09-11T06:50:31.223 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.223 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:31.223 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302938+0000", 2024-09-11T06:50:31.223 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.223 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:31.223 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302939+0000", 2024-09-11T06:50:31.223 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2300000000000001e-06 2024-09-11T06:50:31.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.224 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.224 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:31.224 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302944+0000", 2024-09-11T06:50:31.224 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.493e-06 2024-09-11T06:50:31.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.224 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.224 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:31.224 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302960+0000", 2024-09-11T06:50:31.224 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.632e-05 2024-09-11T06:50:31.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.224 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.224 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.224 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302972+0000", 2024-09-11T06:50:31.224 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1885000000000001e-05 2024-09-11T06:50:31.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.225 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:31.225 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.303835+0000", 2024-09-11T06:50:31.225 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000863141 2024-09-11T06:50:31.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.225 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:31.225 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.303965+0000", 2024-09-11T06:50:31.225 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013032199999999999 2024-09-11T06:50:31.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.226 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:31.226 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.304788+0000", 2024-09-11T06:50:31.226 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00082272900000000002 2024-09-11T06:50:31.226 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.226 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:31.226 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.226 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.226 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:366 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 1052672~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:31.226 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.306960+0000", 2024-09-11T06:50:31.226 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 9.8401755590000004, 2024-09-11T06:50:31.226 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010286749999999999, 2024-09-11T06:50:31.226 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:31.226 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:31.226 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:31.226 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:31.227 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:31.227 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:31.227 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 366 2024-09-11T06:50:31.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.227 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:31.227 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.227 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:31.227 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.306960+0000", 2024-09-11T06:50:31.227 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.227 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.227 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:31.227 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.306960+0000", 2024-09-11T06:50:31.227 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.228 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:31.228 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.306961+0000", 2024-09-11T06:50:31.228 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3939999999999999e-06 2024-09-11T06:50:31.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.228 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:31.228 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.306970+0000", 2024-09-11T06:50:31.228 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6859999999999995e-06 2024-09-11T06:50:31.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.228 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:31.229 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.306991+0000", 2024-09-11T06:50:31.229 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0772e-05 2024-09-11T06:50:31.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.229 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.229 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.229 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.307002+0000", 2024-09-11T06:50:31.229 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0757000000000001e-05 2024-09-11T06:50:31.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.229 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.229 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:31.229 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.307077+0000", 2024-09-11T06:50:31.230 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5337000000000001e-05 2024-09-11T06:50:31.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.230 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.230 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:31.230 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.307185+0000", 2024-09-11T06:50:31.230 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000108335 2024-09-11T06:50:31.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.230 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.230 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:31.230 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.307989+0000", 2024-09-11T06:50:31.230 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00080339400000000003 2024-09-11T06:50:31.230 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.230 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:31.231 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.231 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.231 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.231 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:367 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 1183744~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:31.231 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.312392+0000", 2024-09-11T06:50:31.231 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 9.8347437880000008, 2024-09-11T06:50:31.231 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011343460000000001, 2024-09-11T06:50:31.231 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:31.231 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:31.231 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:31.231 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:31.231 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:31.231 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:31.231 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 367 2024-09-11T06:50:31.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.232 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:31.232 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.232 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:31.232 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312392+0000", 2024-09-11T06:50:31.232 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.232 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.232 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:31.232 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312392+0000", 2024-09-11T06:50:31.232 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.232 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.232 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:31.233 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312393+0000", 2024-09-11T06:50:31.233 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6199999999999999e-06 2024-09-11T06:50:31.233 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.233 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:31.233 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312400+0000", 2024-09-11T06:50:31.233 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0700000000000001e-06 2024-09-11T06:50:31.233 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.233 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:31.233 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312420+0000", 2024-09-11T06:50:31.233 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9375999999999999e-05 2024-09-11T06:50:31.233 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.234 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.234 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312430+0000", 2024-09-11T06:50:31.234 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0209e-05 2024-09-11T06:50:31.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.234 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.234 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:31.234 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312549+0000", 2024-09-11T06:50:31.234 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000118779 2024-09-11T06:50:31.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.234 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.234 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:31.234 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312767+0000", 2024-09-11T06:50:31.234 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021786400000000001 2024-09-11T06:50:31.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.234 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.235 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:31.235 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313526+0000", 2024-09-11T06:50:31.235 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00075942800000000003 2024-09-11T06:50:31.235 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.235 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:31.235 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.235 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.235 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.235 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:368 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 1314816~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:31.235 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.312454+0000", 2024-09-11T06:50:31.235 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 9.8346812640000003, 2024-09-11T06:50:31.235 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014704550000000001, 2024-09-11T06:50:31.235 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:31.235 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:31.235 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:31.235 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:31.236 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:31.236 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:31.236 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 368 2024-09-11T06:50:31.236 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.236 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:31.236 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.236 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:31.236 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312454+0000", 2024-09-11T06:50:31.236 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.236 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.236 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.236 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:31.236 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312454+0000", 2024-09-11T06:50:31.237 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.237 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.237 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.237 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:31.237 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312455+0000", 2024-09-11T06:50:31.237 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4399999999999999e-07 2024-09-11T06:50:31.237 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.237 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.237 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:31.237 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312474+0000", 2024-09-11T06:50:31.237 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8899e-05 2024-09-11T06:50:31.237 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.237 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.237 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:31.238 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312485+0000", 2024-09-11T06:50:31.238 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0793e-05 2024-09-11T06:50:31.238 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.238 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.238 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.238 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312492+0000", 2024-09-11T06:50:31.238 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.593e-06 2024-09-11T06:50:31.238 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.238 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.238 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:31.238 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313576+0000", 2024-09-11T06:50:31.238 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010837970000000001 2024-09-11T06:50:31.238 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.238 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.238 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:31.238 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313620+0000", 2024-09-11T06:50:31.239 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3640000000000002e-05 2024-09-11T06:50:31.239 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.239 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.239 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:31.239 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313925+0000", 2024-09-11T06:50:31.239 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00030498899999999998 2024-09-11T06:50:31.239 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.239 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:31.239 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.239 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.239 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.239 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:370 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 1576960~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:31.239 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.314817+0000", 2024-09-11T06:50:31.239 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 9.8323185550000005, 2024-09-11T06:50:31.239 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011606190000000001, 2024-09-11T06:50:31.239 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:31.240 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:31.240 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:31.240 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:31.240 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:31.240 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:31.240 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 370 2024-09-11T06:50:31.240 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.240 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:31.240 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.240 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:31.240 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314817+0000", 2024-09-11T06:50:31.240 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.240 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.240 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.240 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:31.241 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314817+0000", 2024-09-11T06:50:31.241 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:31.241 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.241 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.241 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:31.241 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314818+0000", 2024-09-11T06:50:31.241 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.463e-06 2024-09-11T06:50:31.241 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.241 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.241 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:31.241 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314826+0000", 2024-09-11T06:50:31.241 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3860000000000001e-06 2024-09-11T06:50:31.241 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.241 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.241 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:31.242 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314842+0000", 2024-09-11T06:50:31.242 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6220000000000001e-05 2024-09-11T06:50:31.242 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.242 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.242 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.242 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314854+0000", 2024-09-11T06:50:31.242 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1562e-05 2024-09-11T06:50:31.242 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.242 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.242 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:31.242 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.315212+0000", 2024-09-11T06:50:31.242 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000358012 2024-09-11T06:50:31.242 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.242 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.242 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:31.242 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.315433+0000", 2024-09-11T06:50:31.243 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022103800000000001 2024-09-11T06:50:31.243 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:31.243 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:31.243 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:31.243 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.315978+0000", 2024-09-11T06:50:31.243 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00054493800000000002 2024-09-11T06:50:31.243 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.243 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:31.243 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.243 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:31.243 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:31.243 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:31.243 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T06:50:31.245 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:31.243+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:31.245 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:50:31.345 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:31.343+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:31.379 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:31.379 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:31.379 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:50:31.379 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:31.394 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:50:31.447 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:31.443+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:31.548 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:31.547+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:31.553 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:31.553 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:31.553 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:50:31.554 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:50:31.554 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:31.554 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo rbd --device-type nbd map --read-only rbd/testrbdnbd21607 2024-09-11T06:50:31.554 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:50:31.554 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:50:31.555 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:50:31.556 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which rbd 2024-09-11T06:50:31.558 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/rbd 2024-09-11T06:50:31.558 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:50:31.559 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/rbd --device-type nbd map --read-only rbd/testrbdnbd21607 2024-09-11T06:50:31.568 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:50:31.649 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:31.647+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:31.696 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV=/dev/nbd0 2024-09-11T06:50:31.696 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd list 2024-09-11T06:50:31.696 INFO:tasks.workunit.client.0.smithi176.stderr:++ awk -v pool=rbd -v img=testrbdnbd21607 -v dev=/dev/nbd0 '$2 == pool && $3 == img && $5 == dev {print $1}' 2024-09-11T06:50:31.701 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:31.701 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:50:31.701 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:50:31.701 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:50:31.701 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.702 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:54 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'32, mlcod=13'29)", 2024-09-11T06:50:31.702 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:50:31.702 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 85.488989238000002, 2024-09-11T06:50:31.702 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034271670000000001, 2024-09-11T06:50:31.702 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:31.702 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:31.702 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:31.702 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:31.702 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.702 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:31.702 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:50:31.702 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.702 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.702 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.702 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:31.703 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:50:31.703 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.703 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.703 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.703 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:31.703 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210361+0000", 2024-09-11T06:50:31.703 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1260000000000001e-06 2024-09-11T06:50:31.703 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.703 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.703 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:31.703 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210370+0000", 2024-09-11T06:50:31.703 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8200000000000003e-06 2024-09-11T06:50:31.703 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.703 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.704 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:31.704 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210390+0000", 2024-09-11T06:50:31.704 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0126e-05 2024-09-11T06:50:31.704 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.704 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.704 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.704 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210404+0000", 2024-09-11T06:50:31.704 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4589999999999999e-05 2024-09-11T06:50:31.704 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.704 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.704 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:31.704 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210538+0000", 2024-09-11T06:50:31.704 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000133719 2024-09-11T06:50:31.704 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.704 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.705 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:31.705 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210593+0000", 2024-09-11T06:50:31.705 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5016999999999999e-05 2024-09-11T06:50:31.705 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.705 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.705 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:31.705 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213695+0000", 2024-09-11T06:50:31.705 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003101866 2024-09-11T06:50:31.705 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.705 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.705 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:31.705 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213787+0000", 2024-09-11T06:50:31.705 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1903000000000004e-05 2024-09-11T06:50:31.706 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.706 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:31.706 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.706 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.706 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.706 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:55 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'33, mlcod=13'29)", 2024-09-11T06:50:31.706 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:50:31.706 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 85.487970000000004, 2024-09-11T06:50:31.706 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.005086091, 2024-09-11T06:50:31.706 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:31.706 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:31.706 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:31.706 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:31.706 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.707 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:31.707 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:50:31.707 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.707 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.707 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:31.707 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:50:31.707 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.707 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.707 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:31.707 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211380+0000", 2024-09-11T06:50:31.707 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.097e-06 2024-09-11T06:50:31.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.707 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.707 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:31.708 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211537+0000", 2024-09-11T06:50:31.708 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000157369 2024-09-11T06:50:31.708 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.708 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.708 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:31.708 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211559+0000", 2024-09-11T06:50:31.708 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000001e-05 2024-09-11T06:50:31.708 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.708 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.708 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.708 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211573+0000", 2024-09-11T06:50:31.708 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4946000000000001e-05 2024-09-11T06:50:31.708 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.708 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.708 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:31.709 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211676+0000", 2024-09-11T06:50:31.709 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0001026 2024-09-11T06:50:31.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.709 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.709 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:31.709 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211709+0000", 2024-09-11T06:50:31.709 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3071999999999998e-05 2024-09-11T06:50:31.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.709 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.709 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:31.709 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216379+0000", 2024-09-11T06:50:31.709 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0046693780000000001 2024-09-11T06:50:31.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.709 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.709 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:31.710 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216465+0000", 2024-09-11T06:50:31.710 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6469000000000002e-05 2024-09-11T06:50:31.710 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.710 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:31.710 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.710 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.710 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.710 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:56 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'34, mlcod=13'29)", 2024-09-11T06:50:31.710 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:50:31.710 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 85.486536141000002, 2024-09-11T06:50:31.710 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0060310629999999997, 2024-09-11T06:50:31.710 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:31.710 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:31.710 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:31.710 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:31.711 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.711 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:31.711 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:50:31.711 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.711 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.711 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.711 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:31.711 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:50:31.711 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.711 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.711 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.711 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:31.711 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212814+0000", 2024-09-11T06:50:31.711 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0869999999999999e-06 2024-09-11T06:50:31.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.712 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.712 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:31.712 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212822+0000", 2024-09-11T06:50:31.712 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0870000000000007e-06 2024-09-11T06:50:31.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.712 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.712 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:31.712 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212840+0000", 2024-09-11T06:50:31.712 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:50:31.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.712 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.712 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.713 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212854+0000", 2024-09-11T06:50:31.713 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4003e-05 2024-09-11T06:50:31.713 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.713 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.713 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:31.713 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213026+0000", 2024-09-11T06:50:31.713 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017207499999999999 2024-09-11T06:50:31.713 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.713 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.713 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:31.713 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213063+0000", 2024-09-11T06:50:31.713 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7187000000000002e-05 2024-09-11T06:50:31.713 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.713 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.713 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:31.714 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218762+0000", 2024-09-11T06:50:31.714 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0056993290000000004 2024-09-11T06:50:31.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.714 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.714 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:31.714 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218844+0000", 2024-09-11T06:50:31.714 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1701999999999998e-05 2024-09-11T06:50:31.714 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.714 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:31.714 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.714 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.714 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:60 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'38, mlcod=13'36)", 2024-09-11T06:50:31.714 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:50:31.714 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 85.472594791999995, 2024-09-11T06:50:31.715 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039254270000000004, 2024-09-11T06:50:31.715 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:31.715 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:31.715 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:31.715 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:31.715 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.715 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:31.715 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:50:31.715 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.715 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.715 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.715 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:31.715 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:50:31.715 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.715 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.716 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.716 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:31.716 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226756+0000", 2024-09-11T06:50:31.716 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-11T06:50:31.716 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.716 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.716 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:31.716 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226768+0000", 2024-09-11T06:50:31.716 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1919999999999999e-05 2024-09-11T06:50:31.716 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.716 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.716 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:31.716 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226797+0000", 2024-09-11T06:50:31.716 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9573e-05 2024-09-11T06:50:31.716 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.717 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.717 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.717 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226818+0000", 2024-09-11T06:50:31.717 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0432999999999999e-05 2024-09-11T06:50:31.717 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.717 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.717 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:31.717 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226909+0000", 2024-09-11T06:50:31.717 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0920000000000004e-05 2024-09-11T06:50:31.717 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.717 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.717 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:31.717 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226936+0000", 2024-09-11T06:50:31.717 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7472999999999999e-05 2024-09-11T06:50:31.718 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.718 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.718 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:31.718 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230596+0000", 2024-09-11T06:50:31.718 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0036602520000000001 2024-09-11T06:50:31.718 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.718 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.718 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:31.718 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230680+0000", 2024-09-11T06:50:31.718 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3106e-05 2024-09-11T06:50:31.718 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.718 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:31.718 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.718 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.718 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.719 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:107 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'69, mlcod=13'64)", 2024-09-11T06:50:31.719 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:50:31.719 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 85.256147334999994, 2024-09-11T06:50:31.719 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040728830000000002, 2024-09-11T06:50:31.719 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:31.719 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:31.719 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:31.719 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:31.719 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.719 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:31.719 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:50:31.719 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.719 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.719 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.719 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:31.720 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:50:31.720 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.720 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.720 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.720 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:31.720 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443203+0000", 2024-09-11T06:50:31.720 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1480000000000001e-06 2024-09-11T06:50:31.720 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.720 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.720 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:31.720 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443222+0000", 2024-09-11T06:50:31.720 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9040000000000001e-05 2024-09-11T06:50:31.720 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.720 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.721 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:31.721 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443246+0000", 2024-09-11T06:50:31.721 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4606000000000001e-05 2024-09-11T06:50:31.721 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.721 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.721 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.721 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443277+0000", 2024-09-11T06:50:31.721 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0207999999999999e-05 2024-09-11T06:50:31.721 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.721 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.721 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:31.721 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443340+0000", 2024-09-11T06:50:31.721 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3602000000000005e-05 2024-09-11T06:50:31.721 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.722 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.722 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:31.722 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443363+0000", 2024-09-11T06:50:31.722 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2331e-05 2024-09-11T06:50:31.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.722 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.722 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:31.722 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447216+0000", 2024-09-11T06:50:31.722 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0038539690000000001 2024-09-11T06:50:31.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.722 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.722 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:31.722 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447274+0000", 2024-09-11T06:50:31.722 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7979000000000002e-05 2024-09-11T06:50:31.723 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.723 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:31.723 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.723 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.723 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.723 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:111 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'73, mlcod=13'67)", 2024-09-11T06:50:31.723 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:50:31.723 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 85.249635705000003, 2024-09-11T06:50:31.723 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034372360000000002, 2024-09-11T06:50:31.723 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:31.723 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:31.723 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:31.723 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:31.724 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.724 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:31.724 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:50:31.724 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.724 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.724 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.724 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:31.724 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:50:31.724 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.724 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.724 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.724 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:31.724 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449714+0000", 2024-09-11T06:50:31.724 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5000000000000001e-07 2024-09-11T06:50:31.725 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.725 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.725 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:31.725 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449721+0000", 2024-09-11T06:50:31.725 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8530000000000003e-06 2024-09-11T06:50:31.725 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.725 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.725 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:31.725 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449736+0000", 2024-09-11T06:50:31.725 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5114e-05 2024-09-11T06:50:31.725 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.725 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.725 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.725 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449743+0000", 2024-09-11T06:50:31.726 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0210000000000002e-06 2024-09-11T06:50:31.726 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.726 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.726 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:31.726 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449804+0000", 2024-09-11T06:50:31.726 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0479999999999997e-05 2024-09-11T06:50:31.726 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.726 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.726 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:31.726 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449826+0000", 2024-09-11T06:50:31.726 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1922000000000001e-05 2024-09-11T06:50:31.726 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.726 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.726 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:31.726 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453104+0000", 2024-09-11T06:50:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003278652 2024-09-11T06:50:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453150+0000", 2024-09-11T06:50:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6244e-05 2024-09-11T06:50:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:114 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'76, mlcod=13'71)", 2024-09-11T06:50:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:50:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 85.244957634000002, 2024-09-11T06:50:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0042576660000000002, 2024-09-11T06:50:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:50:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:50:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454392+0000", 2024-09-11T06:50:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5999999999999991e-07 2024-09-11T06:50:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454399+0000", 2024-09-11T06:50:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.878e-06 2024-09-11T06:50:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454415+0000", 2024-09-11T06:50:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5659999999999999e-05 2024-09-11T06:50:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454422+0000", 2024-09-11T06:50:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:50:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454474+0000", 2024-09-11T06:50:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1860999999999997e-05 2024-09-11T06:50:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454497+0000", 2024-09-11T06:50:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3686e-05 2024-09-11T06:50:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458552+0000", 2024-09-11T06:50:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040544550000000002 2024-09-11T06:50:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458649+0000", 2024-09-11T06:50:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7005999999999999e-05 2024-09-11T06:50:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:115 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'77, mlcod=13'72)", 2024-09-11T06:50:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 85.243617021999995, 2024-09-11T06:50:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003355584, 2024-09-11T06:50:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5999999999999999e-07 2024-09-11T06:50:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455803+0000", 2024-09-11T06:50:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0310999999999998e-05 2024-09-11T06:50:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455815+0000", 2024-09-11T06:50:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2787e-05 2024-09-11T06:50:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455823+0000", 2024-09-11T06:50:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5959999999999998e-06 2024-09-11T06:50:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455875+0000", 2024-09-11T06:50:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2054e-05 2024-09-11T06:50:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455896+0000", 2024-09-11T06:50:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0554999999999999e-05 2024-09-11T06:50:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459000+0000", 2024-09-11T06:50:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003104842 2024-09-11T06:50:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459087+0000", 2024-09-11T06:50:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6978999999999995e-05 2024-09-11T06:50:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:120 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'82, mlcod=13'78)", 2024-09-11T06:50:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:50:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 85.232939736999995, 2024-09-11T06:50:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0064612549999999999, 2024-09-11T06:50:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:31.737 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:50:31.737 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.737 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.737 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.737 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:31.737 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:50:31.737 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.737 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.737 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.737 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:31.737 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466410+0000", 2024-09-11T06:50:31.737 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0389999999999999e-06 2024-09-11T06:50:31.737 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.737 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.737 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:31.737 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466496+0000", 2024-09-11T06:50:31.738 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5519000000000006e-05 2024-09-11T06:50:31.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.738 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.738 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:31.738 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466510+0000", 2024-09-11T06:50:31.738 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3842999999999999e-05 2024-09-11T06:50:31.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.738 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.738 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.738 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466517+0000", 2024-09-11T06:50:31.738 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5229999999999998e-06 2024-09-11T06:50:31.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.738 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.739 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:31.739 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466595+0000", 2024-09-11T06:50:31.739 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8019e-05 2024-09-11T06:50:31.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.739 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:31.739 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466612+0000", 2024-09-11T06:50:31.739 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7013000000000001e-05 2024-09-11T06:50:31.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.739 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:31.739 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472787+0000", 2024-09-11T06:50:31.739 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0061750269999999996 2024-09-11T06:50:31.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.739 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:31.740 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472870+0000", 2024-09-11T06:50:31.740 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3271999999999998e-05 2024-09-11T06:50:31.740 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.740 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:31.740 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.740 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.740 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:121 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'83, mlcod=13'78)", 2024-09-11T06:50:31.740 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:50:31.740 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 85.231714941000007, 2024-09-11T06:50:31.740 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055613069999999997, 2024-09-11T06:50:31.740 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:31.740 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:31.740 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:31.740 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:31.741 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.741 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:31.741 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:50:31.741 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.741 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.741 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.741 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:31.741 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:50:31.741 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.741 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.741 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.741 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:31.741 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467635+0000", 2024-09-11T06:50:31.741 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6899999999999996e-07 2024-09-11T06:50:31.741 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.742 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.742 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:31.742 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467727+0000", 2024-09-11T06:50:31.742 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1800999999999997e-05 2024-09-11T06:50:31.742 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.742 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.742 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:31.742 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467740+0000", 2024-09-11T06:50:31.742 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3747999999999999e-05 2024-09-11T06:50:31.742 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.742 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.742 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.742 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467753+0000", 2024-09-11T06:50:31.743 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2807e-05 2024-09-11T06:50:31.743 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.743 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.743 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:31.743 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467812+0000", 2024-09-11T06:50:31.743 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8381999999999998e-05 2024-09-11T06:50:31.743 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.743 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.743 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:31.743 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467828+0000", 2024-09-11T06:50:31.743 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6728e-05 2024-09-11T06:50:31.743 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.743 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.743 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:31.744 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473112+0000", 2024-09-11T06:50:31.744 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052833619999999998 2024-09-11T06:50:31.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.744 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.744 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:31.744 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473195+0000", 2024-09-11T06:50:31.744 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3609999999999994e-05 2024-09-11T06:50:31.744 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.744 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:31.744 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.744 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.744 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:131 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'89, mlcod=13'85)", 2024-09-11T06:50:31.744 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:50:31.745 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 84.880309225999994, 2024-09-11T06:50:31.745 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052492260000000001, 2024-09-11T06:50:31.745 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:31.745 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:31.745 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:31.745 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:31.745 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.745 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:31.745 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:50:31.745 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.745 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.745 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.745 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:31.745 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:50:31.745 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.745 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.746 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.746 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:31.746 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819041+0000", 2024-09-11T06:50:31.746 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4360000000000001e-06 2024-09-11T06:50:31.746 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.746 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.746 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:31.746 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819100+0000", 2024-09-11T06:50:31.746 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9225999999999998e-05 2024-09-11T06:50:31.746 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.746 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.746 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:31.746 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819128+0000", 2024-09-11T06:50:31.746 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.779e-05 2024-09-11T06:50:31.746 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.746 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.747 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.747 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819142+0000", 2024-09-11T06:50:31.747 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3573e-05 2024-09-11T06:50:31.747 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.747 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.747 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:31.747 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819263+0000", 2024-09-11T06:50:31.747 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000121086 2024-09-11T06:50:31.747 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.747 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.747 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:31.747 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819292+0000", 2024-09-11T06:50:31.747 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9363e-05 2024-09-11T06:50:31.747 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.747 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.747 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:31.747 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824185+0000", 2024-09-11T06:50:31.748 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0048924620000000002 2024-09-11T06:50:31.748 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.748 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.748 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:31.748 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824289+0000", 2024-09-11T06:50:31.748 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010429000000000001 2024-09-11T06:50:31.748 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.748 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:31.748 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.748 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.748 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.748 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:132 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'90, mlcod=13'85)", 2024-09-11T06:50:31.748 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:50:31.748 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 84.879534535999994, 2024-09-11T06:50:31.748 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.004532342, 2024-09-11T06:50:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:50:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:50:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:31.750 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819816+0000", 2024-09-11T06:50:31.750 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.61e-06 2024-09-11T06:50:31.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.750 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.750 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:31.750 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819827+0000", 2024-09-11T06:50:31.750 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1e-05 2024-09-11T06:50:31.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.750 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.750 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:31.750 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819851+0000", 2024-09-11T06:50:31.750 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4267e-05 2024-09-11T06:50:31.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.750 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.750 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.750 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819864+0000", 2024-09-11T06:50:31.751 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2767e-05 2024-09-11T06:50:31.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.751 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.751 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:31.751 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820071+0000", 2024-09-11T06:50:31.751 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000207061 2024-09-11T06:50:31.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.751 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.751 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:31.751 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820103+0000", 2024-09-11T06:50:31.751 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2326999999999997e-05 2024-09-11T06:50:31.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.751 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.751 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:31.751 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824302+0000", 2024-09-11T06:50:31.751 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0041986000000000002 2024-09-11T06:50:31.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.752 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.752 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:31.752 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824347+0000", 2024-09-11T06:50:31.752 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4709999999999997e-05 2024-09-11T06:50:31.752 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.752 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:31.752 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.752 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.752 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:133 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'91, mlcod=13'86)", 2024-09-11T06:50:31.752 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:50:31.752 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 84.878902877000002, 2024-09-11T06:50:31.752 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039421459999999997, 2024-09-11T06:50:31.753 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:31.753 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:31.753 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:31.753 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:31.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.753 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:31.753 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:50:31.753 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.753 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:31.753 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:50:31.753 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.754 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:31.754 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820448+0000", 2024-09-11T06:50:31.754 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.84e-06 2024-09-11T06:50:31.754 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.754 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.754 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:31.754 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820463+0000", 2024-09-11T06:50:31.754 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5356e-05 2024-09-11T06:50:31.754 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.754 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.754 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:31.754 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820499+0000", 2024-09-11T06:50:31.754 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.625e-05 2024-09-11T06:50:31.754 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.754 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.754 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.755 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820546+0000", 2024-09-11T06:50:31.755 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7017000000000001e-05 2024-09-11T06:50:31.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.755 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:31.755 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820882+0000", 2024-09-11T06:50:31.755 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033512 2024-09-11T06:50:31.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.755 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:31.755 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820941+0000", 2024-09-11T06:50:31.755 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8922999999999998e-05 2024-09-11T06:50:31.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.756 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:31.756 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824351+0000", 2024-09-11T06:50:31.756 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034101940000000001 2024-09-11T06:50:31.756 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.756 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.756 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:31.756 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824388+0000", 2024-09-11T06:50:31.756 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7446e-05 2024-09-11T06:50:31.756 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.756 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:31.756 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.756 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.756 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.756 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:134 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'92, mlcod=13'86)", 2024-09-11T06:50:31.756 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:50:31.757 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 84.878243759, 2024-09-11T06:50:31.757 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033248179999999998, 2024-09-11T06:50:31.757 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:31.757 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:31.757 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:31.757 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:31.757 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.757 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:31.757 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:50:31.757 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.757 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.757 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:31.757 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:50:31.758 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.758 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.758 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:31.758 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821107+0000", 2024-09-11T06:50:31.758 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7430000000000001e-06 2024-09-11T06:50:31.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.758 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.758 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:31.758 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821205+0000", 2024-09-11T06:50:31.758 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7666999999999998e-05 2024-09-11T06:50:31.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.759 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:31.759 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821234+0000", 2024-09-11T06:50:31.759 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9756000000000001e-05 2024-09-11T06:50:31.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.759 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.759 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821254+0000", 2024-09-11T06:50:31.759 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9446e-05 2024-09-11T06:50:31.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.759 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:31.759 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821846+0000", 2024-09-11T06:50:31.759 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00059187599999999997 2024-09-11T06:50:31.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.760 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.760 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:31.760 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821894+0000", 2024-09-11T06:50:31.760 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8766e-05 2024-09-11T06:50:31.760 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.760 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.760 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:31.760 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824392+0000", 2024-09-11T06:50:31.760 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0024977509999999999 2024-09-11T06:50:31.760 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.760 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.760 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:31.760 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824430+0000", 2024-09-11T06:50:31.760 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7812999999999999e-05 2024-09-11T06:50:31.761 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.761 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:31.761 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.761 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.761 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:139 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'97, mlcod=13'86)", 2024-09-11T06:50:31.761 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:50:31.761 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 84.873492898999999, 2024-09-11T06:50:31.761 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003732221, 2024-09-11T06:50:31.761 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:31.761 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:31.762 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:31.762 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:31.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.762 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:31.762 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:50:31.762 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.762 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:31.762 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:50:31.762 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.762 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:31.763 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825858+0000", 2024-09-11T06:50:31.763 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5829999999999999e-06 2024-09-11T06:50:31.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.763 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.763 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:31.763 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825874+0000", 2024-09-11T06:50:31.763 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5967000000000001e-05 2024-09-11T06:50:31.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.763 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.763 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:31.763 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825909+0000", 2024-09-11T06:50:31.763 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5692999999999999e-05 2024-09-11T06:50:31.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.763 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.764 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.764 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825923+0000", 2024-09-11T06:50:31.764 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4080000000000001e-05 2024-09-11T06:50:31.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.764 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:31.764 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826010+0000", 2024-09-11T06:50:31.764 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6248999999999994e-05 2024-09-11T06:50:31.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.764 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:31.764 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826038+0000", 2024-09-11T06:50:31.764 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8010000000000001e-05 2024-09-11T06:50:31.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.765 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.765 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:31.765 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829499+0000", 2024-09-11T06:50:31.765 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003461783 2024-09-11T06:50:31.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.765 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.765 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:31.765 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829588+0000", 2024-09-11T06:50:31.765 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8856000000000004e-05 2024-09-11T06:50:31.765 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.765 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:31.765 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.765 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.765 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:140 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'98, mlcod=13'93)", 2024-09-11T06:50:31.765 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:50:31.766 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 84.871642170000001, 2024-09-11T06:50:31.766 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0057995929999999996, 2024-09-11T06:50:31.766 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:31.766 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:31.766 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:31.766 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:31.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.766 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:31.766 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:50:31.766 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.766 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:31.766 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:50:31.767 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.767 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:31.767 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827708+0000", 2024-09-11T06:50:31.767 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.547e-06 2024-09-11T06:50:31.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.767 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:31.767 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827765+0000", 2024-09-11T06:50:31.767 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6266e-05 2024-09-11T06:50:31.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.767 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:31.767 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827791+0000", 2024-09-11T06:50:31.768 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6098999999999999e-05 2024-09-11T06:50:31.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.768 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.768 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.768 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827806+0000", 2024-09-11T06:50:31.768 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5537e-05 2024-09-11T06:50:31.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.768 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.768 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:31.768 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827928+0000", 2024-09-11T06:50:31.768 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000122012 2024-09-11T06:50:31.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.768 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.768 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:31.768 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827961+0000", 2024-09-11T06:50:31.769 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2663999999999998e-05 2024-09-11T06:50:31.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.769 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.769 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:31.769 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833409+0000", 2024-09-11T06:50:31.769 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054479350000000001 2024-09-11T06:50:31.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.769 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.769 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:31.769 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833506+0000", 2024-09-11T06:50:31.769 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7533000000000003e-05 2024-09-11T06:50:31.769 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.769 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:31.769 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.770 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.770 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:141 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'99, mlcod=13'93)", 2024-09-11T06:50:31.770 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:50:31.770 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 84.870748786999997, 2024-09-11T06:50:31.770 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055394709999999998, 2024-09-11T06:50:31.770 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:31.770 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:31.770 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:31.770 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:31.770 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.770 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:31.770 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:50:31.770 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.770 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.771 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:31.771 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:50:31.771 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.771 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:31.771 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828602+0000", 2024-09-11T06:50:31.771 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.46e-06 2024-09-11T06:50:31.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.771 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:31.771 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828618+0000", 2024-09-11T06:50:31.771 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6592999999999999e-05 2024-09-11T06:50:31.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.772 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:31.772 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828646+0000", 2024-09-11T06:50:31.772 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7552999999999999e-05 2024-09-11T06:50:31.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.772 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.772 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828663+0000", 2024-09-11T06:50:31.772 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.719e-05 2024-09-11T06:50:31.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.772 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:31.772 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829023+0000", 2024-09-11T06:50:31.772 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036054799999999998 2024-09-11T06:50:31.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.773 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:31.773 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829062+0000", 2024-09-11T06:50:31.773 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8865999999999997e-05 2024-09-11T06:50:31.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.773 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:31.773 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834078+0000", 2024-09-11T06:50:31.773 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050157480000000004 2024-09-11T06:50:31.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.773 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:31.773 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834140+0000", 2024-09-11T06:50:31.773 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1513000000000005e-05 2024-09-11T06:50:31.774 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.774 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:31.774 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.774 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:144 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'102, mlcod=13'98)", 2024-09-11T06:50:31.774 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:50:31.774 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 84.861588174000005, 2024-09-11T06:50:31.774 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003210348, 2024-09-11T06:50:31.774 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:31.774 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:31.774 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:31.774 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:31.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.774 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:31.775 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:50:31.775 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.775 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.775 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:31.775 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:50:31.775 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.775 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.775 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:31.775 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837768+0000", 2024-09-11T06:50:31.775 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9330000000000004e-06 2024-09-11T06:50:31.775 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.775 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:31.775 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837858+0000", 2024-09-11T06:50:31.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0185999999999993e-05 2024-09-11T06:50:31.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:31.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837890+0000", 2024-09-11T06:50:31.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2437000000000001e-05 2024-09-11T06:50:31.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837905+0000", 2024-09-11T06:50:31.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:50:31.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:31.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838019+0000", 2024-09-11T06:50:31.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000114336 2024-09-11T06:50:31.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.777 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:31.777 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838052+0000", 2024-09-11T06:50:31.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3139000000000002e-05 2024-09-11T06:50:31.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.777 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:31.777 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840914+0000", 2024-09-11T06:50:31.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028612780000000001 2024-09-11T06:50:31.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.777 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:31.778 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840971+0000", 2024-09-11T06:50:31.778 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7402999999999999e-05 2024-09-11T06:50:31.778 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.778 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:31.778 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.778 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:145 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'103, mlcod=13'98)", 2024-09-11T06:50:31.778 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:50:31.778 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 84.860796289999996, 2024-09-11T06:50:31.778 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054455409999999999, 2024-09-11T06:50:31.778 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:31.778 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:31.778 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:31.778 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:31.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.779 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:31.779 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:50:31.779 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.779 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.779 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.779 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:31.779 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:50:31.779 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.779 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.779 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.779 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:31.779 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838554+0000", 2024-09-11T06:50:31.779 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3430000000000001e-06 2024-09-11T06:50:31.779 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.779 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.779 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:31.780 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838640+0000", 2024-09-11T06:50:31.780 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5569999999999996e-05 2024-09-11T06:50:31.780 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.780 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.780 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:31.780 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838660+0000", 2024-09-11T06:50:31.780 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0576000000000001e-05 2024-09-11T06:50:31.780 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.780 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.780 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.780 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838674+0000", 2024-09-11T06:50:31.780 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3529999999999999e-05 2024-09-11T06:50:31.780 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.780 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.780 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:31.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838838+0000", 2024-09-11T06:50:31.781 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016468799999999999 2024-09-11T06:50:31.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:31.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838885+0000", 2024-09-11T06:50:31.781 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6884e-05 2024-09-11T06:50:31.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:31.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843901+0000", 2024-09-11T06:50:31.781 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050162310000000003 2024-09-11T06:50:31.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.782 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:31.782 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843998+0000", 2024-09-11T06:50:31.782 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.6718999999999993e-05 2024-09-11T06:50:31.782 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.782 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:31.782 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.782 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4558.0:1 3.3 e56/55 3:c4c92e5a:::rbd_trash:head v 56'1, mlcod=0'0)", 2024-09-11T06:50:31.782 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:50:31.782 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.607993228, 2024-09-11T06:50:31.782 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0053791810000000002, 2024-09-11T06:50:31.782 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:31.782 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:31.783 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:31.783 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:31.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.783 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:31.783 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:50:31.783 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.783 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:31.783 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:50:31.783 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:31.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.784 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:31.784 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091357+0000", 2024-09-11T06:50:31.784 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.54e-07 2024-09-11T06:50:31.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.784 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.784 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:31.784 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091376+0000", 2024-09-11T06:50:31.784 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9769e-05 2024-09-11T06:50:31.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.784 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.784 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:31.784 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091396+0000", 2024-09-11T06:50:31.784 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0152999999999998e-05 2024-09-11T06:50:31.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.784 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.784 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:31.785 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091416+0000", 2024-09-11T06:50:31.785 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0006000000000001e-05 2024-09-11T06:50:31.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.785 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.785 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:31.785 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091506+0000", 2024-09-11T06:50:31.785 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9325000000000003e-05 2024-09-11T06:50:31.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.785 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.785 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:31.785 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091523+0000", 2024-09-11T06:50:31.785 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.77e-05 2024-09-11T06:50:31.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.785 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.785 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:31.785 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.096685+0000", 2024-09-11T06:50:31.786 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0051618569999999997 2024-09-11T06:50:31.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:31.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:31.786 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:31.786 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.096735+0000", 2024-09-11T06:50:31.786 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9517e-05 2024-09-11T06:50:31.786 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.786 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:31.786 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.786 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:31.786 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:31.786 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:31.787 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=22559 2024-09-11T06:50:31.788 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 22559 2024-09-11T06:50:31.788 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 22559 -C rbd-nbd 2024-09-11T06:50:31.788 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:50:31.788 INFO:tasks.workunit.client.0.smithi176.stdout: 22559 ? 00:00:00 rbd-nbd 2024-09-11T06:50:31.789 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:50:31.790 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo dd if=/dev/nbd0 of=/dev/null bs=1M 2024-09-11T06:50:31.790 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:31.790 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:31.790 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:31.790 INFO:tasks.workunit.client.0.smithi176.stderr:++ which dd 2024-09-11T06:50:31.790 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/dd 2024-09-11T06:50:31.790 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:31.790 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/dd if=/dev/nbd0 of=/dev/null bs=1M 2024-09-11T06:50:31.790 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:31.787+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:31.888 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:31.883+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:31.962 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:31.962 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:31.962 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:50:31.962 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:31.977 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:50:31.989 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:31.987+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:32.037 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:50:32.113 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:32.111+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:32.117 INFO:tasks.workunit.client.0.smithi176.stderr:64+0 records in 2024-09-11T06:50:32.117 INFO:tasks.workunit.client.0.smithi176.stderr:64+0 records out 2024-09-11T06:50:32.117 INFO:tasks.workunit.client.0.smithi176.stderr:67108864 bytes (67 MB, 64 MiB) copied, 0.336668 s, 199 MB/s 2024-09-11T06:50:32.118 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false _sudo dd if=/tmp/tmp.IGWYJYR7sz/data of=/dev/nbd0 bs=1M oflag=direct 2024-09-11T06:50:32.119 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo dd if=/tmp/tmp.IGWYJYR7sz/data of=/dev/nbd0 bs=1M oflag=direct 2024-09-11T06:50:32.119 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:32.119 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:32.121 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:32.122 INFO:tasks.workunit.client.0.smithi176.stderr:++ which dd 2024-09-11T06:50:32.124 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/dd 2024-09-11T06:50:32.124 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:32.124 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/dd if=/tmp/tmp.IGWYJYR7sz/data of=/dev/nbd0 bs=1M oflag=direct 2024-09-11T06:50:32.132 INFO:tasks.workunit.client.0.smithi176.stderr:/usr/bin/dd: error writing '/dev/nbd0': Operation not permitted 2024-09-11T06:50:32.132 INFO:tasks.workunit.client.0.smithi176.stderr:1+0 records in 2024-09-11T06:50:32.132 INFO:tasks.workunit.client.0.smithi176.stderr:0+0 records out 2024-09-11T06:50:32.132 INFO:tasks.workunit.client.0.smithi176.stderr:0 bytes copied, 0.00073302 s, 0.0 kB/s 2024-09-11T06:50:32.134 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:32.134 INFO:tasks.workunit.client.0.smithi176.stderr:+ unmap_device /dev/nbd0 22559 2024-09-11T06:50:32.134 INFO:tasks.workunit.client.0.smithi176.stderr:+ local args=/dev/nbd0 2024-09-11T06:50:32.134 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pid=22559 2024-09-11T06:50:32.134 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd0 2024-09-11T06:50:32.134 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:32.134 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:32.135 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:32.135 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:50:32.138 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:50:32.138 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:32.138 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd0 2024-09-11T06:50:32.153 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:32.153 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:32.154 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:50:32.154 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:50:32.154 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:32.170 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:50:32.190 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is set 2024-09-11T06:50:32.191 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:32.187+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:32.216 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:50:32.216 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:50:32.224 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:50:32.225 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^22559\b' 2024-09-11T06:50:32.225 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^22559\b' 2024-09-11T06:50:32.233 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:50:32.233 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:50:32.250 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:50:32.250 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:50:32.271 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:50:32.271 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:50:32.281 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:32.281 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -C rbd-nbd 2024-09-11T06:50:32.281 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^ *22559\b' 2024-09-11T06:50:32.281 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^ *22559\b' 2024-09-11T06:50:32.287 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:32.287 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 0.5 2024-09-11T06:50:32.291 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:32.287+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:32.292 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:50:32.292 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:50:32.309 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:32.309 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:32.313 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:50:32.313 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:50:32.325 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:50:32.393 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:32.391+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:32.494 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:32.491+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:32.497 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:32.497 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:32.512 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:50:32.595 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:32.591+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:32.670 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:32.670 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:32.684 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:50:32.696 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:32.695+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:32.789 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo rbd --device-type nbd map --exclusive rbd/testrbdnbd21607 2024-09-11T06:50:32.789 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:50:32.790 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:50:32.790 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:50:32.791 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which rbd 2024-09-11T06:50:32.793 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/rbd 2024-09-11T06:50:32.794 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:50:32.794 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/rbd --device-type nbd map --exclusive rbd/testrbdnbd21607 2024-09-11T06:50:32.796 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:32.791+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:32.805 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:32.805 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:32.819 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:50:32.897 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:32.895+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:32.938 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV=/dev/nbd0 2024-09-11T06:50:32.938 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd 2024-09-11T06:50:32.939 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:50:32.939 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:50:32.939 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:50:32.939 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd0'\'']/id' 2024-09-11T06:50:32.949 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:32.949 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:32.964 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:50:32.997 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:32.995+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:33.005 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=22631 2024-09-11T06:50:33.005 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 22631 2024-09-11T06:50:33.005 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 22631 -C rbd-nbd 2024-09-11T06:50:33.013 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:50:33.013 INFO:tasks.workunit.client.0.smithi176.stdout: 22631 ? 00:00:00 rbd-nbd 2024-09-11T06:50:33.013 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo dd if=/tmp/tmp.IGWYJYR7sz/data of=/dev/nbd0 bs=1M oflag=direct 2024-09-11T06:50:33.013 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:33.013 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:33.014 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:33.014 INFO:tasks.workunit.client.0.smithi176.stderr:++ which dd 2024-09-11T06:50:33.016 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/dd 2024-09-11T06:50:33.016 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:33.016 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/dd if=/tmp/tmp.IGWYJYR7sz/data of=/dev/nbd0 bs=1M oflag=direct 2024-09-11T06:50:33.098 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:33.095+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:33.139 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:33.139 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:33.154 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:50:33.189 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 1] out_osds: [3, 5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T06:50:33.189 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T06:50:33.189 INFO:tasks.thrashosds.thrasher:Adding osd 3 2024-09-11T06:50:33.189 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 3 2024-09-11T06:50:33.241 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:33.239+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:33.273 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:33.273 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:33.289 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:50:33.335 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:33.335+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:33.400 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:33.395+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:33.422 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:33.422 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:33.437 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:50:33.524 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:33.523+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:33.546 INFO:tasks.ceph.mon.a.smithi071.stderr:2024-09-11T06:50:33.543+0000 7f1296554640 -1 mon.a@0(leader).osd e68 definitely_dead 0 2024-09-11T06:50:33.602 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:33.599+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:33.617 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:33.617 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:33.632 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:50:33.720 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:33.719+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:33.769 INFO:tasks.workunit.client.0.smithi176.stderr:64+0 records in 2024-09-11T06:50:33.769 INFO:tasks.workunit.client.0.smithi176.stderr:64+0 records out 2024-09-11T06:50:33.769 INFO:tasks.workunit.client.0.smithi176.stderr:67108864 bytes (67 MB, 64 MiB) copied, 0.747532 s, 89.8 MB/s 2024-09-11T06:50:33.771 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false timeout 10 rbd bench testrbdnbd21607 --io-type write --io-size=1024 --io-total=1024 2024-09-11T06:50:33.771 INFO:tasks.workunit.client.0.smithi176.stderr:+ timeout 10 rbd bench testrbdnbd21607 --io-type write --io-size=1024 --io-total=1024 2024-09-11T06:50:33.798 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:33.798 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:33.804 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:33.799+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:33.812 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:50:33.845 INFO:tasks.workunit.client.0.smithi176.stderr:rbd: failed to flush: 2024-09-11T06:50:33.839+0000 7fcd04ec3640 -1 librbd::exclusive_lock::ImageDispatch: 0x7fcce0000f70 handle_acquire_lock: failed to acquire exclusive lock: (30) Read-only file system 2024-09-11T06:50:33.845 INFO:tasks.workunit.client.0.smithi176.stderr:2024-09-11T06:50:33.839+0000 7fcd04ec3640 -1 librbd::io::AioCompletion: 0x55f8fd788a00 fail: (30) Read-only file system 2024-09-11T06:50:33.845 INFO:tasks.workunit.client.0.smithi176.stderr:(30) Read-only file system 2024-09-11T06:50:33.845 INFO:tasks.workunit.client.0.smithi176.stderr:bench failed: (30) Read-only file system 2024-09-11T06:50:33.853 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:33.854 INFO:tasks.workunit.client.0.smithi176.stderr:+ unmap_device /dev/nbd0 22631 2024-09-11T06:50:33.854 INFO:tasks.workunit.client.0.smithi176.stderr:+ local args=/dev/nbd0 2024-09-11T06:50:33.854 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pid=22631 2024-09-11T06:50:33.854 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd0 2024-09-11T06:50:33.854 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:33.854 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:33.854 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:33.855 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:50:33.856 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:50:33.856 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:33.856 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd0 2024-09-11T06:50:33.905 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:33.903+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:33.952 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:50:33.952 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^22631\b' 2024-09-11T06:50:33.952 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^22631\b' 2024-09-11T06:50:33.976 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:33.976 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:33.990 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:50:34.000 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:34.000 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -C rbd-nbd 2024-09-11T06:50:34.000 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^ *22631\b' 2024-09-11T06:50:34.000 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^ *22631\b' 2024-09-11T06:50:34.006 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:34.006 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 0.5 2024-09-11T06:50:34.036 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:34.035+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:34.106 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:34.103+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:34.153 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:34.153 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:34.168 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:50:34.207 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:34.203+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:34.215 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T06:50:34.215 INFO:teuthology.orchestra.run.smithi071.stderr:marked in osd.3. 2024-09-11T06:50:34.244 INFO:tasks.thrashosds.thrasher:Added osd 3 2024-09-11T06:50:34.292 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:34.292 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:34.306 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:50:34.307 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:34.307+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:34.332 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:50:34.408 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:34.403+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:34.438 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:34.439 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:34.454 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:50:34.489 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:50:34.490 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:50:34.507 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:50:34.507 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:50:34.508 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV= 2024-09-11T06:50:34.508 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd bench testrbdnbd21607 --io-type write --io-size=1024 --io-total=1024 2024-09-11T06:50:34.509 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:34.507+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:34.525 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:50:34.525 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:50:34.546 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:50:34.546 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:50:34.566 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:50:34.566 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:50:34.586 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:50:34.587 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:50:34.605 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:34.605 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:34.605 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:50:34.605 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:50:34.605 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:34.610 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:34.607+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:34.618 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:50:34.711 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:34.707+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:34.786 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:34.786 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:50:34.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:50:34.787 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:50:34.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.787 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:54 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'32, mlcod=13'29)", 2024-09-11T06:50:34.787 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:50:34.787 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 88.574320503999999, 2024-09-11T06:50:34.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034271670000000001, 2024-09-11T06:50:34.787 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:34.787 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:34.787 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:34.787 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:34.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.787 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:34.788 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:50:34.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.788 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:34.788 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:50:34.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.788 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:34.788 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210361+0000", 2024-09-11T06:50:34.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1260000000000001e-06 2024-09-11T06:50:34.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.788 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:34.789 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210370+0000", 2024-09-11T06:50:34.789 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8200000000000003e-06 2024-09-11T06:50:34.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.789 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.789 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:34.789 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210390+0000", 2024-09-11T06:50:34.789 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0126e-05 2024-09-11T06:50:34.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.789 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.789 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:34.789 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210404+0000", 2024-09-11T06:50:34.789 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4589999999999999e-05 2024-09-11T06:50:34.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.790 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.790 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:34.790 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210538+0000", 2024-09-11T06:50:34.790 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000133719 2024-09-11T06:50:34.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.790 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.790 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:34.790 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210593+0000", 2024-09-11T06:50:34.790 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5016999999999999e-05 2024-09-11T06:50:34.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.790 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.790 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:34.790 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213695+0000", 2024-09-11T06:50:34.790 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003101866 2024-09-11T06:50:34.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.791 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.791 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:34.791 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213787+0000", 2024-09-11T06:50:34.791 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1903000000000004e-05 2024-09-11T06:50:34.791 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.791 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:34.791 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.791 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.791 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:55 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'33, mlcod=13'29)", 2024-09-11T06:50:34.791 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:50:34.791 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 88.573301266000001, 2024-09-11T06:50:34.791 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.005086091, 2024-09-11T06:50:34.791 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:34.792 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:34.792 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:34.792 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:34.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.792 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:34.792 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:50:34.792 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.792 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:34.792 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:50:34.792 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.793 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:34.793 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211380+0000", 2024-09-11T06:50:34.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.097e-06 2024-09-11T06:50:34.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.793 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.793 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:34.793 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211537+0000", 2024-09-11T06:50:34.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000157369 2024-09-11T06:50:34.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.793 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.793 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:34.793 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211559+0000", 2024-09-11T06:50:34.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000001e-05 2024-09-11T06:50:34.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.794 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:34.794 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211573+0000", 2024-09-11T06:50:34.794 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4946000000000001e-05 2024-09-11T06:50:34.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.794 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:34.794 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211676+0000", 2024-09-11T06:50:34.794 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0001026 2024-09-11T06:50:34.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.794 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:34.794 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211709+0000", 2024-09-11T06:50:34.794 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3071999999999998e-05 2024-09-11T06:50:34.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.795 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:34.795 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216379+0000", 2024-09-11T06:50:34.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0046693780000000001 2024-09-11T06:50:34.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.795 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:34.795 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216465+0000", 2024-09-11T06:50:34.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6469000000000002e-05 2024-09-11T06:50:34.795 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.795 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:34.795 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.795 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:56 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'34, mlcod=13'29)", 2024-09-11T06:50:34.796 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:50:34.796 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 88.571867406999999, 2024-09-11T06:50:34.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0060310629999999997, 2024-09-11T06:50:34.796 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:34.796 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:34.796 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:34.796 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:34.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:34.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:50:34.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:34.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:50:34.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:34.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212814+0000", 2024-09-11T06:50:34.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0869999999999999e-06 2024-09-11T06:50:34.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:34.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212822+0000", 2024-09-11T06:50:34.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0870000000000007e-06 2024-09-11T06:50:34.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:34.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212840+0000", 2024-09-11T06:50:34.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:50:34.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:34.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212854+0000", 2024-09-11T06:50:34.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4003e-05 2024-09-11T06:50:34.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:34.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213026+0000", 2024-09-11T06:50:34.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017207499999999999 2024-09-11T06:50:34.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.799 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:34.799 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213063+0000", 2024-09-11T06:50:34.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7187000000000002e-05 2024-09-11T06:50:34.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.799 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:34.799 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218762+0000", 2024-09-11T06:50:34.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0056993290000000004 2024-09-11T06:50:34.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.799 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:34.799 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218844+0000", 2024-09-11T06:50:34.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1701999999999998e-05 2024-09-11T06:50:34.799 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.799 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:34.800 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.800 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.800 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:60 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'38, mlcod=13'36)", 2024-09-11T06:50:34.800 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:50:34.800 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 88.557926058000007, 2024-09-11T06:50:34.800 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039254270000000004, 2024-09-11T06:50:34.800 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:34.800 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:34.800 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:34.800 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:34.800 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.800 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:34.800 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:50:34.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.801 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:34.801 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:50:34.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.801 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:34.801 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226756+0000", 2024-09-11T06:50:34.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-11T06:50:34.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.801 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:34.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226768+0000", 2024-09-11T06:50:34.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1919999999999999e-05 2024-09-11T06:50:34.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:34.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226797+0000", 2024-09-11T06:50:34.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9573e-05 2024-09-11T06:50:34.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:34.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226818+0000", 2024-09-11T06:50:34.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0432999999999999e-05 2024-09-11T06:50:34.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:34.803 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226909+0000", 2024-09-11T06:50:34.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0920000000000004e-05 2024-09-11T06:50:34.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.803 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:34.803 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226936+0000", 2024-09-11T06:50:34.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7472999999999999e-05 2024-09-11T06:50:34.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.803 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:34.803 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230596+0000", 2024-09-11T06:50:34.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0036602520000000001 2024-09-11T06:50:34.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.803 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:34.804 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230680+0000", 2024-09-11T06:50:34.804 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3106e-05 2024-09-11T06:50:34.804 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.804 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:34.804 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.804 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:107 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'69, mlcod=13'64)", 2024-09-11T06:50:34.804 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:50:34.804 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 88.341478601000006, 2024-09-11T06:50:34.804 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040728830000000002, 2024-09-11T06:50:34.804 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:34.804 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:34.804 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:34.805 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:34.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.805 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:34.805 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:50:34.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.805 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:34.805 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:50:34.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.805 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:34.805 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443203+0000", 2024-09-11T06:50:34.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1480000000000001e-06 2024-09-11T06:50:34.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:34.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443222+0000", 2024-09-11T06:50:34.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9040000000000001e-05 2024-09-11T06:50:34.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:34.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443246+0000", 2024-09-11T06:50:34.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4606000000000001e-05 2024-09-11T06:50:34.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:34.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443277+0000", 2024-09-11T06:50:34.807 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0207999999999999e-05 2024-09-11T06:50:34.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.807 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:34.807 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443340+0000", 2024-09-11T06:50:34.807 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3602000000000005e-05 2024-09-11T06:50:34.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.807 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:34.807 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443363+0000", 2024-09-11T06:50:34.807 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2331e-05 2024-09-11T06:50:34.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.808 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.808 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:34.808 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447216+0000", 2024-09-11T06:50:34.808 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0038539690000000001 2024-09-11T06:50:34.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.808 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.808 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:34.808 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447274+0000", 2024-09-11T06:50:34.808 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7979000000000002e-05 2024-09-11T06:50:34.808 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.808 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:34.808 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.808 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.809 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:111 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'73, mlcod=13'67)", 2024-09-11T06:50:34.809 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:50:34.809 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 88.334966971, 2024-09-11T06:50:34.809 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034372360000000002, 2024-09-11T06:50:34.809 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:34.809 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:34.809 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:34.809 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:34.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.809 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:34.809 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:50:34.809 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:34.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:50:34.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:34.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449714+0000", 2024-09-11T06:50:34.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5000000000000001e-07 2024-09-11T06:50:34.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:34.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449721+0000", 2024-09-11T06:50:34.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8530000000000003e-06 2024-09-11T06:50:34.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.811 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:34.811 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449736+0000", 2024-09-11T06:50:34.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5114e-05 2024-09-11T06:50:34.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.811 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:34.811 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449743+0000", 2024-09-11T06:50:34.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0210000000000002e-06 2024-09-11T06:50:34.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.811 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:34.811 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449804+0000", 2024-09-11T06:50:34.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0479999999999997e-05 2024-09-11T06:50:34.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.812 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:34.812 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449826+0000", 2024-09-11T06:50:34.812 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1922000000000001e-05 2024-09-11T06:50:34.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.812 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:34.812 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453104+0000", 2024-09-11T06:50:34.812 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003278652 2024-09-11T06:50:34.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.812 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:34.812 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453150+0000", 2024-09-11T06:50:34.812 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6244e-05 2024-09-11T06:50:34.812 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.812 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:34.813 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.813 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:114 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'76, mlcod=13'71)", 2024-09-11T06:50:34.813 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:50:34.813 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 88.330288899999999, 2024-09-11T06:50:34.813 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0042576660000000002, 2024-09-11T06:50:34.813 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:34.813 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:34.813 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:34.813 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:34.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.813 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:34.813 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:50:34.813 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.814 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:34.814 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:50:34.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.814 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:34.814 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454392+0000", 2024-09-11T06:50:34.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5999999999999991e-07 2024-09-11T06:50:34.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.814 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:34.814 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454399+0000", 2024-09-11T06:50:34.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.878e-06 2024-09-11T06:50:34.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:34.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454415+0000", 2024-09-11T06:50:34.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5659999999999999e-05 2024-09-11T06:50:34.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:34.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454422+0000", 2024-09-11T06:50:34.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:50:34.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:34.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454474+0000", 2024-09-11T06:50:34.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1860999999999997e-05 2024-09-11T06:50:34.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.816 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:34.816 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454497+0000", 2024-09-11T06:50:34.816 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3686e-05 2024-09-11T06:50:34.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.816 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:34.816 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458552+0000", 2024-09-11T06:50:34.816 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040544550000000002 2024-09-11T06:50:34.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.816 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:34.816 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458649+0000", 2024-09-11T06:50:34.816 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7005999999999999e-05 2024-09-11T06:50:34.817 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.817 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:34.817 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.817 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.817 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:115 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'77, mlcod=13'72)", 2024-09-11T06:50:34.817 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:34.817 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 88.328948288000007, 2024-09-11T06:50:34.817 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003355584, 2024-09-11T06:50:34.817 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:34.817 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:34.817 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:34.817 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:34.817 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.817 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:34.817 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:34.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.818 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:34.818 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:34.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.818 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:34.818 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:34.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5999999999999999e-07 2024-09-11T06:50:34.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.818 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:34.818 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455803+0000", 2024-09-11T06:50:34.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0310999999999998e-05 2024-09-11T06:50:34.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:34.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455815+0000", 2024-09-11T06:50:34.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2787e-05 2024-09-11T06:50:34.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:34.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455823+0000", 2024-09-11T06:50:34.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5959999999999998e-06 2024-09-11T06:50:34.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:34.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455875+0000", 2024-09-11T06:50:34.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2054e-05 2024-09-11T06:50:34.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:34.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455896+0000", 2024-09-11T06:50:34.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0554999999999999e-05 2024-09-11T06:50:34.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:34.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459000+0000", 2024-09-11T06:50:34.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003104842 2024-09-11T06:50:34.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:34.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459087+0000", 2024-09-11T06:50:34.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6978999999999995e-05 2024-09-11T06:50:34.820 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.821 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:34.821 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.821 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:120 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'82, mlcod=13'78)", 2024-09-11T06:50:34.821 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:50:34.821 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 88.318271003000007, 2024-09-11T06:50:34.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0064612549999999999, 2024-09-11T06:50:34.821 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:34.821 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:34.821 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:34.821 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:34.822 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.822 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:34.822 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:50:34.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.822 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.822 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:34.822 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:50:34.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.822 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.822 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:34.822 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466410+0000", 2024-09-11T06:50:34.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0389999999999999e-06 2024-09-11T06:50:34.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.822 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.823 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:34.823 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466496+0000", 2024-09-11T06:50:34.823 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5519000000000006e-05 2024-09-11T06:50:34.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.823 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:34.823 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466510+0000", 2024-09-11T06:50:34.823 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3842999999999999e-05 2024-09-11T06:50:34.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.824 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:34.824 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466517+0000", 2024-09-11T06:50:34.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5229999999999998e-06 2024-09-11T06:50:34.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.824 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:34.824 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466595+0000", 2024-09-11T06:50:34.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8019e-05 2024-09-11T06:50:34.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.824 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:34.824 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466612+0000", 2024-09-11T06:50:34.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7013000000000001e-05 2024-09-11T06:50:34.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:34.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472787+0000", 2024-09-11T06:50:34.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0061750269999999996 2024-09-11T06:50:34.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:34.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472870+0000", 2024-09-11T06:50:34.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3271999999999998e-05 2024-09-11T06:50:34.825 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.825 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:34.825 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.825 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:121 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'83, mlcod=13'78)", 2024-09-11T06:50:34.825 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:50:34.826 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 88.317046207000004, 2024-09-11T06:50:34.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055613069999999997, 2024-09-11T06:50:34.826 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:34.826 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:34.826 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:34.826 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:34.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.826 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:34.826 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:50:34.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.826 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:34.826 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:50:34.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.827 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.827 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:34.827 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467635+0000", 2024-09-11T06:50:34.827 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6899999999999996e-07 2024-09-11T06:50:34.827 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.827 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.827 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:34.827 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467727+0000", 2024-09-11T06:50:34.827 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1800999999999997e-05 2024-09-11T06:50:34.827 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.827 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.827 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:34.827 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467740+0000", 2024-09-11T06:50:34.827 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3747999999999999e-05 2024-09-11T06:50:34.827 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:34.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467753+0000", 2024-09-11T06:50:34.828 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2807e-05 2024-09-11T06:50:34.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:34.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467812+0000", 2024-09-11T06:50:34.828 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8381999999999998e-05 2024-09-11T06:50:34.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:34.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467828+0000", 2024-09-11T06:50:34.828 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6728e-05 2024-09-11T06:50:34.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:34.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473112+0000", 2024-09-11T06:50:34.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052833619999999998 2024-09-11T06:50:34.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:34.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473195+0000", 2024-09-11T06:50:34.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3609999999999994e-05 2024-09-11T06:50:34.829 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.829 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:34.829 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.829 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:131 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'89, mlcod=13'85)", 2024-09-11T06:50:34.830 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:50:34.830 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 87.965640492000006, 2024-09-11T06:50:34.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052492260000000001, 2024-09-11T06:50:34.830 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:34.830 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:34.830 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:34.830 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:34.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.830 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:34.830 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:50:34.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.830 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:34.830 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:50:34.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:34.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819041+0000", 2024-09-11T06:50:34.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4360000000000001e-06 2024-09-11T06:50:34.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:34.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819100+0000", 2024-09-11T06:50:34.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9225999999999998e-05 2024-09-11T06:50:34.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:34.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819128+0000", 2024-09-11T06:50:34.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.779e-05 2024-09-11T06:50:34.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:34.832 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819142+0000", 2024-09-11T06:50:34.832 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3573e-05 2024-09-11T06:50:34.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:34.832 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819263+0000", 2024-09-11T06:50:34.832 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000121086 2024-09-11T06:50:34.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:34.832 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819292+0000", 2024-09-11T06:50:34.832 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9363e-05 2024-09-11T06:50:34.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.833 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:34.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824185+0000", 2024-09-11T06:50:34.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0048924620000000002 2024-09-11T06:50:34.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.833 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:34.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824289+0000", 2024-09-11T06:50:34.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010429000000000001 2024-09-11T06:50:34.833 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.833 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:34.833 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.833 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:132 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'90, mlcod=13'85)", 2024-09-11T06:50:34.834 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:50:34.834 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 87.964865802000006, 2024-09-11T06:50:34.834 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.004532342, 2024-09-11T06:50:34.834 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:34.834 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:34.834 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:34.834 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:34.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.834 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:34.834 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:50:34.834 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.834 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:34.834 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:50:34.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:34.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819816+0000", 2024-09-11T06:50:34.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.61e-06 2024-09-11T06:50:34.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:34.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819827+0000", 2024-09-11T06:50:34.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1e-05 2024-09-11T06:50:34.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:34.836 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819851+0000", 2024-09-11T06:50:34.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4267e-05 2024-09-11T06:50:34.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:34.836 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819864+0000", 2024-09-11T06:50:34.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2767e-05 2024-09-11T06:50:34.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:34.836 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820071+0000", 2024-09-11T06:50:34.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000207061 2024-09-11T06:50:34.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:34.836 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820103+0000", 2024-09-11T06:50:34.837 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2326999999999997e-05 2024-09-11T06:50:34.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.837 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.837 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:34.837 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824302+0000", 2024-09-11T06:50:34.837 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0041986000000000002 2024-09-11T06:50:34.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.837 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.837 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:34.837 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824347+0000", 2024-09-11T06:50:34.837 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4709999999999997e-05 2024-09-11T06:50:34.837 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.837 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:34.837 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.838 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:133 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'91, mlcod=13'86)", 2024-09-11T06:50:34.838 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:50:34.838 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 87.964234142999999, 2024-09-11T06:50:34.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039421459999999997, 2024-09-11T06:50:34.838 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:34.838 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:34.838 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:34.838 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:34.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:34.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:50:34.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.839 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:34.839 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:50:34.839 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.839 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:34.839 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820448+0000", 2024-09-11T06:50:34.839 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.84e-06 2024-09-11T06:50:34.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.839 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:34.839 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820463+0000", 2024-09-11T06:50:34.839 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5356e-05 2024-09-11T06:50:34.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.840 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:34.840 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820499+0000", 2024-09-11T06:50:34.840 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.625e-05 2024-09-11T06:50:34.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.840 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:34.840 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820546+0000", 2024-09-11T06:50:34.840 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7017000000000001e-05 2024-09-11T06:50:34.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.840 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:34.840 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820882+0000", 2024-09-11T06:50:34.840 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033512 2024-09-11T06:50:34.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:34.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820941+0000", 2024-09-11T06:50:34.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8922999999999998e-05 2024-09-11T06:50:34.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:34.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824351+0000", 2024-09-11T06:50:34.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034101940000000001 2024-09-11T06:50:34.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:34.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824388+0000", 2024-09-11T06:50:34.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7446e-05 2024-09-11T06:50:34.841 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.842 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:34.842 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.842 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:134 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'92, mlcod=13'86)", 2024-09-11T06:50:34.842 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:50:34.842 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 87.963575024999997, 2024-09-11T06:50:34.842 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033248179999999998, 2024-09-11T06:50:34.842 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:34.842 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:34.842 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:34.842 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:34.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.842 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:34.842 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:50:34.843 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.843 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.843 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:34.843 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:50:34.843 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.843 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.843 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:34.843 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821107+0000", 2024-09-11T06:50:34.843 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7430000000000001e-06 2024-09-11T06:50:34.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.843 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.843 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:34.843 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821205+0000", 2024-09-11T06:50:34.843 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7666999999999998e-05 2024-09-11T06:50:34.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.844 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:34.844 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821234+0000", 2024-09-11T06:50:34.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9756000000000001e-05 2024-09-11T06:50:34.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.844 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:34.844 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821254+0000", 2024-09-11T06:50:34.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9446e-05 2024-09-11T06:50:34.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.844 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:34.844 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821846+0000", 2024-09-11T06:50:34.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00059187599999999997 2024-09-11T06:50:34.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.845 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:34.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821894+0000", 2024-09-11T06:50:34.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8766e-05 2024-09-11T06:50:34.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.845 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:34.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824392+0000", 2024-09-11T06:50:34.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0024977509999999999 2024-09-11T06:50:34.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.845 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:34.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824430+0000", 2024-09-11T06:50:34.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7812999999999999e-05 2024-09-11T06:50:34.846 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.846 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:34.846 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.846 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:139 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'97, mlcod=13'86)", 2024-09-11T06:50:34.846 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:50:34.846 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 87.958824164999996, 2024-09-11T06:50:34.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003732221, 2024-09-11T06:50:34.846 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:34.846 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:34.846 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:34.846 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:34.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:34.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:50:34.847 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.847 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:34.847 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:50:34.847 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.847 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:34.847 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825858+0000", 2024-09-11T06:50:34.847 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5829999999999999e-06 2024-09-11T06:50:34.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.847 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:34.847 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825874+0000", 2024-09-11T06:50:34.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5967000000000001e-05 2024-09-11T06:50:34.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.848 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:34.848 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825909+0000", 2024-09-11T06:50:34.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5692999999999999e-05 2024-09-11T06:50:34.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.848 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:34.848 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825923+0000", 2024-09-11T06:50:34.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4080000000000001e-05 2024-09-11T06:50:34.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.848 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:34.848 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826010+0000", 2024-09-11T06:50:34.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6248999999999994e-05 2024-09-11T06:50:34.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.849 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:34.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826038+0000", 2024-09-11T06:50:34.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8010000000000001e-05 2024-09-11T06:50:34.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.849 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:34.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829499+0000", 2024-09-11T06:50:34.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003461783 2024-09-11T06:50:34.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.849 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:34.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829588+0000", 2024-09-11T06:50:34.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8856000000000004e-05 2024-09-11T06:50:34.850 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.850 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:34.850 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.850 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:140 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'98, mlcod=13'93)", 2024-09-11T06:50:34.850 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:50:34.850 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 87.956973435999998, 2024-09-11T06:50:34.850 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0057995929999999996, 2024-09-11T06:50:34.850 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:34.850 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:34.850 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:34.850 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:34.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.850 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:34.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:50:34.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:34.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:50:34.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:34.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827708+0000", 2024-09-11T06:50:34.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.547e-06 2024-09-11T06:50:34.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:34.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827765+0000", 2024-09-11T06:50:34.852 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6266e-05 2024-09-11T06:50:34.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.852 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.852 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:34.852 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827791+0000", 2024-09-11T06:50:34.852 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6098999999999999e-05 2024-09-11T06:50:34.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.852 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.852 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:34.852 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827806+0000", 2024-09-11T06:50:34.852 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5537e-05 2024-09-11T06:50:34.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.852 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.852 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:34.852 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827928+0000", 2024-09-11T06:50:34.852 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000122012 2024-09-11T06:50:34.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.853 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.853 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:34.853 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827961+0000", 2024-09-11T06:50:34.853 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2663999999999998e-05 2024-09-11T06:50:34.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.853 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.853 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:34.853 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833409+0000", 2024-09-11T06:50:34.853 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054479350000000001 2024-09-11T06:50:34.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.853 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.853 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:34.853 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833506+0000", 2024-09-11T06:50:34.853 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7533000000000003e-05 2024-09-11T06:50:34.854 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.854 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:34.854 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.854 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:141 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'99, mlcod=13'93)", 2024-09-11T06:50:34.854 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:50:34.854 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 87.956080052999994, 2024-09-11T06:50:34.854 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055394709999999998, 2024-09-11T06:50:34.854 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:34.854 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:34.854 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:34.854 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:34.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.854 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:34.855 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:50:34.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.855 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:34.855 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:50:34.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.855 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:34.855 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828602+0000", 2024-09-11T06:50:34.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.46e-06 2024-09-11T06:50:34.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.855 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:34.856 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828618+0000", 2024-09-11T06:50:34.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6592999999999999e-05 2024-09-11T06:50:34.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.856 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:34.856 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828646+0000", 2024-09-11T06:50:34.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7552999999999999e-05 2024-09-11T06:50:34.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.856 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:34.856 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828663+0000", 2024-09-11T06:50:34.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.719e-05 2024-09-11T06:50:34.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.856 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:34.857 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829023+0000", 2024-09-11T06:50:34.857 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036054799999999998 2024-09-11T06:50:34.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.857 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:34.857 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829062+0000", 2024-09-11T06:50:34.857 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8865999999999997e-05 2024-09-11T06:50:34.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.857 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:34.857 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834078+0000", 2024-09-11T06:50:34.857 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050157480000000004 2024-09-11T06:50:34.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.857 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:34.858 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834140+0000", 2024-09-11T06:50:34.858 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1513000000000005e-05 2024-09-11T06:50:34.858 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.858 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:34.858 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.858 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.858 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:144 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'102, mlcod=13'98)", 2024-09-11T06:50:34.858 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:50:34.858 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 87.946919440000002, 2024-09-11T06:50:34.858 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003210348, 2024-09-11T06:50:34.858 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:34.858 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:34.858 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:34.858 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:34.859 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.859 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:34.859 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:50:34.859 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.859 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.859 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:34.859 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:50:34.859 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.859 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.859 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:34.859 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837768+0000", 2024-09-11T06:50:34.859 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9330000000000004e-06 2024-09-11T06:50:34.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.860 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:34.860 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837858+0000", 2024-09-11T06:50:34.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0185999999999993e-05 2024-09-11T06:50:34.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.860 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:34.860 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837890+0000", 2024-09-11T06:50:34.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2437000000000001e-05 2024-09-11T06:50:34.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.860 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:34.860 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837905+0000", 2024-09-11T06:50:34.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:50:34.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.861 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:34.861 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838019+0000", 2024-09-11T06:50:34.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000114336 2024-09-11T06:50:34.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.861 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:34.861 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838052+0000", 2024-09-11T06:50:34.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3139000000000002e-05 2024-09-11T06:50:34.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.861 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:34.861 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840914+0000", 2024-09-11T06:50:34.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028612780000000001 2024-09-11T06:50:34.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.862 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:34.862 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840971+0000", 2024-09-11T06:50:34.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7402999999999999e-05 2024-09-11T06:50:34.862 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.862 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:34.862 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.862 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:145 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'103, mlcod=13'98)", 2024-09-11T06:50:34.862 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:50:34.862 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 87.946127555999993, 2024-09-11T06:50:34.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054455409999999999, 2024-09-11T06:50:34.862 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:34.862 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:34.862 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:34.862 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:34.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:34.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:50:34.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:34.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:50:34.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:34.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838554+0000", 2024-09-11T06:50:34.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3430000000000001e-06 2024-09-11T06:50:34.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.864 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:34.864 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838640+0000", 2024-09-11T06:50:34.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5569999999999996e-05 2024-09-11T06:50:34.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.864 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:34.864 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838660+0000", 2024-09-11T06:50:34.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0576000000000001e-05 2024-09-11T06:50:34.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.864 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:34.864 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838674+0000", 2024-09-11T06:50:34.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3529999999999999e-05 2024-09-11T06:50:34.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:34.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838838+0000", 2024-09-11T06:50:34.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016468799999999999 2024-09-11T06:50:34.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:34.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838885+0000", 2024-09-11T06:50:34.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6884e-05 2024-09-11T06:50:34.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:34.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843901+0000", 2024-09-11T06:50:34.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050162310000000003 2024-09-11T06:50:34.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.866 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:34.866 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843998+0000", 2024-09-11T06:50:34.866 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.6718999999999993e-05 2024-09-11T06:50:34.866 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.866 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:34.866 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.866 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4558.0:1 3.3 e56/55 3:c4c92e5a:::rbd_trash:head v 56'1, mlcod=0'0)", 2024-09-11T06:50:34.866 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:50:34.866 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 17.693324493999999, 2024-09-11T06:50:34.866 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0053791810000000002, 2024-09-11T06:50:34.866 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:34.867 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:34.867 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:34.867 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:34.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.867 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:34.867 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:50:34.867 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.867 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:34.867 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:50:34.867 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:34.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.867 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:34.868 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091357+0000", 2024-09-11T06:50:34.868 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.54e-07 2024-09-11T06:50:34.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.868 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:34.868 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091376+0000", 2024-09-11T06:50:34.868 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9769e-05 2024-09-11T06:50:34.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.868 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:34.868 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091396+0000", 2024-09-11T06:50:34.868 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0152999999999998e-05 2024-09-11T06:50:34.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.868 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:34.869 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091416+0000", 2024-09-11T06:50:34.869 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0006000000000001e-05 2024-09-11T06:50:34.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.869 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:34.869 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091506+0000", 2024-09-11T06:50:34.869 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9325000000000003e-05 2024-09-11T06:50:34.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.869 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:34.869 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091523+0000", 2024-09-11T06:50:34.869 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.77e-05 2024-09-11T06:50:34.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.870 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.870 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:34.870 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.096685+0000", 2024-09-11T06:50:34.870 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0051618569999999997 2024-09-11T06:50:34.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:34.870 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:34.870 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:34.870 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.096735+0000", 2024-09-11T06:50:34.870 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9517e-05 2024-09-11T06:50:34.870 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.870 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:34.870 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.870 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:34.870 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:34.871 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:34.872 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:50:34.872 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:34.867+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:34.973 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:34.971+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:35.041 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:35.042 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:35.042 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:50:35.042 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:35.055 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:50:35.097 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:35.095+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:35.175 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:35.171+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:35.217 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:35.217 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:35.217 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:50:35.218 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:50:35.218 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:35.223 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is set 2024-09-11T06:50:35.237 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:50:35.260 INFO:tasks.workunit.client.0.smithi176.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2024-09-11T06:50:35.294 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:35.291+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:35.309 INFO:tasks.workunit.client.0.smithi176.stdout: SEC OPS OPS/SEC BYTES/SEC 2024-09-11T06:50:35.309 INFO:tasks.workunit.client.0.smithi176.stdout:elapsed: 0 ops: 1 ops/sec: 19.231 bytes/sec: 19 KiB/s 2024-09-11T06:50:35.322 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo rbd device --device-type nbd map rbd/testrbdnbd21607 2024-09-11T06:50:35.322 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:50:35.323 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:50:35.325 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:50:35.325 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which rbd 2024-09-11T06:50:35.327 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/rbd 2024-09-11T06:50:35.327 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:50:35.327 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map rbd/testrbdnbd21607 2024-09-11T06:50:35.376 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:35.371+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:35.416 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:35.416 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:50:35.417 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:50:35.417 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:50:35.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.417 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:5 2.2 e17/15 2:5cea7035:::rbd_info:head v 17'1, mlcod=0'0)", 2024-09-11T06:50:35.417 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:50:35.417 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 83.176134206, 2024-09-11T06:50:35.417 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0019446719999999999, 2024-09-11T06:50:35.417 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:35.417 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:35.417 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:35.417 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:35.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.417 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:35.417 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:50:35.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.418 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:35.418 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:50:35.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.418 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:35.418 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236085+0000", 2024-09-11T06:50:35.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6000000000000003e-07 2024-09-11T06:50:35.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.418 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:35.418 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236108+0000", 2024-09-11T06:50:35.419 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3113e-05 2024-09-11T06:50:35.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.419 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:35.419 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236133+0000", 2024-09-11T06:50:35.419 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5723000000000001e-05 2024-09-11T06:50:35.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.419 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:35.419 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236154+0000", 2024-09-11T06:50:35.419 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0333999999999999e-05 2024-09-11T06:50:35.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.419 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:35.419 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236297+0000", 2024-09-11T06:50:35.420 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014278500000000001 2024-09-11T06:50:35.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.420 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:35.420 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236383+0000", 2024-09-11T06:50:35.420 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5927999999999995e-05 2024-09-11T06:50:35.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.420 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:35.420 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.237950+0000", 2024-09-11T06:50:35.420 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015673690000000001 2024-09-11T06:50:35.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.420 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:35.420 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.238029+0000", 2024-09-11T06:50:35.421 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8659999999999996e-05 2024-09-11T06:50:35.421 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.421 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:35.421 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.421 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.421 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:7 2.2 e18/15 2:5cea7035:::rbd_info:head v 18'2, mlcod=0'0)", 2024-09-11T06:50:35.421 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:50:35.421 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 82.161808999000002, 2024-09-11T06:50:35.421 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001907192, 2024-09-11T06:50:35.421 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:35.421 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:35.421 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:35.421 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:35.421 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.421 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:35.422 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:50:35.422 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.422 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:35.422 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:50:35.422 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.422 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:35.422 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250410+0000", 2024-09-11T06:50:35.422 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7700000000000004e-07 2024-09-11T06:50:35.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.423 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:35.423 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250445+0000", 2024-09-11T06:50:35.423 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4749000000000002e-05 2024-09-11T06:50:35.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.423 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:35.423 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250478+0000", 2024-09-11T06:50:35.423 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3269999999999998e-05 2024-09-11T06:50:35.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.423 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:35.423 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250501+0000", 2024-09-11T06:50:35.423 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3632999999999999e-05 2024-09-11T06:50:35.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.424 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:35.424 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250633+0000", 2024-09-11T06:50:35.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013143200000000001 2024-09-11T06:50:35.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.424 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:35.424 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250700+0000", 2024-09-11T06:50:35.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7002999999999995e-05 2024-09-11T06:50:35.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.424 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:35.424 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.252172+0000", 2024-09-11T06:50:35.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014719550000000001 2024-09-11T06:50:35.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.425 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:35.425 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.252316+0000", 2024-09-11T06:50:35.425 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144473 2024-09-11T06:50:35.425 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.425 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:35.425 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.425 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4558.0:5 3.2 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-11T06:50:35.425 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:50:35.425 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 17.323799769000001, 2024-09-11T06:50:35.425 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0062353959999999998, 2024-09-11T06:50:35.426 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:35.426 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:35.426 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:35.426 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:35.426 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4558", 2024-09-11T06:50:35.426 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1478388898", 2024-09-11T06:50:35.426 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 5 2024-09-11T06:50:35.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.426 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:35.426 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.426 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:35.426 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:50:35.426 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.426 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.427 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:35.427 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:50:35.427 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.427 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:35.427 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088422+0000", 2024-09-11T06:50:35.427 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7859999999999998e-06 2024-09-11T06:50:35.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.427 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:35.427 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088440+0000", 2024-09-11T06:50:35.427 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8185999999999998e-05 2024-09-11T06:50:35.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.428 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:35.428 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088479+0000", 2024-09-11T06:50:35.428 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8248999999999999e-05 2024-09-11T06:50:35.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.428 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:35.428 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088497+0000", 2024-09-11T06:50:35.428 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8889000000000002e-05 2024-09-11T06:50:35.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.428 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:35.428 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088658+0000", 2024-09-11T06:50:35.428 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000160271 2024-09-11T06:50:35.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.428 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op must wait for map", 2024-09-11T06:50:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088668+0000", 2024-09-11T06:50:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0409e-05 2024-09-11T06:50:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092347+0000", 2024-09-11T06:50:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00367915 2024-09-11T06:50:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092541+0000", 2024-09-11T06:50:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000193833 2024-09-11T06:50:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-11T06:50:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092732+0000", 2024-09-11T06:50:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000190772 2024-09-11T06:50:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094136+0000", 2024-09-11T06:50:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014040809999999999 2024-09-11T06:50:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094506+0000", 2024-09-11T06:50:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000369959 2024-09-11T06:50:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094590+0000", 2024-09-11T06:50:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4033e-05 2024-09-11T06:50:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094654+0000", 2024-09-11T06:50:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3777999999999998e-05 2024-09-11T06:50:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:259 2.6 2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head [sparse-read 4096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.147544+0000", 2024-09-11T06:50:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.264674384999999, 2024-09-11T06:50:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0019824170000000002, 2024-09-11T06:50:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 259 2024-09-11T06:50:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.147544+0000", 2024-09-11T06:50:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:35.433 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.147544+0000", 2024-09-11T06:50:35.433 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.433 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.433 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:35.433 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.147545+0000", 2024-09-11T06:50:35.433 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4600000000000003e-07 2024-09-11T06:50:35.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.433 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.433 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:35.433 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.147549+0000", 2024-09-11T06:50:35.433 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8160000000000001e-06 2024-09-11T06:50:35.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.433 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.433 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.147565+0000", 2024-09-11T06:50:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5140000000000001e-05 2024-09-11T06:50:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.147574+0000", 2024-09-11T06:50:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2529999999999993e-06 2024-09-11T06:50:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.148551+0000", 2024-09-11T06:50:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0009775579999999999 2024-09-11T06:50:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.148694+0000", 2024-09-11T06:50:35.435 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014299399999999999 2024-09-11T06:50:35.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.435 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.435 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:35.435 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.149526+0000", 2024-09-11T06:50:35.435 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00083171000000000004 2024-09-11T06:50:35.435 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.435 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:35.435 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.435 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.435 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:261 2.6 2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:35.435 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.154411+0000", 2024-09-11T06:50:35.435 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.257806971000001, 2024-09-11T06:50:35.435 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002245024, 2024-09-11T06:50:35.436 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:35.436 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:35.436 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:35.436 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:35.436 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:35.436 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:35.436 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 261 2024-09-11T06:50:35.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.436 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:35.436 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.436 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:35.436 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154411+0000", 2024-09-11T06:50:35.436 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.436 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.436 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:35.437 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154411+0000", 2024-09-11T06:50:35.437 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.437 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:35.437 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154412+0000", 2024-09-11T06:50:35.437 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:50:35.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.437 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:35.437 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154416+0000", 2024-09-11T06:50:35.437 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3930000000000001e-06 2024-09-11T06:50:35.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.437 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154428+0000", 2024-09-11T06:50:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2099999999999999e-05 2024-09-11T06:50:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154438+0000", 2024-09-11T06:50:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2590000000000006e-06 2024-09-11T06:50:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.155643+0000", 2024-09-11T06:50:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012048689999999999 2024-09-11T06:50:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.155838+0000", 2024-09-11T06:50:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019531299999999999 2024-09-11T06:50:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.156656+0000", 2024-09-11T06:50:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00081824700000000003 2024-09-11T06:50:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:269 2.6 2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head [sparse-read 1314816~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.166906+0000", 2024-09-11T06:50:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.245311943000001, 2024-09-11T06:50:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001538906, 2024-09-11T06:50:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 269 2024-09-11T06:50:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.166906+0000", 2024-09-11T06:50:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.441 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.441 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:35.441 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.166906+0000", 2024-09-11T06:50:35.441 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.441 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.441 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:35.441 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.166907+0000", 2024-09-11T06:50:35.441 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2729999999999999e-06 2024-09-11T06:50:35.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.441 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.441 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:35.441 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.166914+0000", 2024-09-11T06:50:35.441 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3230000000000003e-06 2024-09-11T06:50:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.166934+0000", 2024-09-11T06:50:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.018e-05 2024-09-11T06:50:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.166951+0000", 2024-09-11T06:50:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7501999999999999e-05 2024-09-11T06:50:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.167677+0000", 2024-09-11T06:50:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00072579299999999997 2024-09-11T06:50:35.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.443 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.443 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:35.443 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.167865+0000", 2024-09-11T06:50:35.443 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000188023 2024-09-11T06:50:35.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.443 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.443 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:35.443 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.168445+0000", 2024-09-11T06:50:35.443 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00057981199999999997 2024-09-11T06:50:35.443 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.443 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:35.443 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:277 2.6 2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head [sparse-read 2363392~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.179016+0000", 2024-09-11T06:50:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.233201889, 2024-09-11T06:50:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015587540000000001, 2024-09-11T06:50:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 277 2024-09-11T06:50:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.179016+0000", 2024-09-11T06:50:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.179016+0000", 2024-09-11T06:50:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.179017+0000", 2024-09-11T06:50:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.78e-07 2024-09-11T06:50:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.179021+0000", 2024-09-11T06:50:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9469999999999996e-06 2024-09-11T06:50:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.179032+0000", 2024-09-11T06:50:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0763e-05 2024-09-11T06:50:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.179040+0000", 2024-09-11T06:50:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7360000000000001e-06 2024-09-11T06:50:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.179817+0000", 2024-09-11T06:50:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00077775700000000001 2024-09-11T06:50:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.179958+0000", 2024-09-11T06:50:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014041800000000001 2024-09-11T06:50:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.180575+0000", 2024-09-11T06:50:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00061725500000000004 2024-09-11T06:50:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:525 2.4 2:2a17619e:::rbd_data.1314f0a2bfa2.000000000000000e:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.530454+0000", 2024-09-11T06:50:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 13.881764285999999, 2024-09-11T06:50:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0019464650000000001, 2024-09-11T06:50:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 525 2024-09-11T06:50:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530454+0000", 2024-09-11T06:50:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530454+0000", 2024-09-11T06:50:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530455+0000", 2024-09-11T06:50:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4100000000000001e-06 2024-09-11T06:50:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530462+0000", 2024-09-11T06:50:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2760000000000003e-06 2024-09-11T06:50:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530486+0000", 2024-09-11T06:50:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3082999999999999e-05 2024-09-11T06:50:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530511+0000", 2024-09-11T06:50:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5065000000000001e-05 2024-09-11T06:50:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.531637+0000", 2024-09-11T06:50:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011263530000000001 2024-09-11T06:50:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.531716+0000", 2024-09-11T06:50:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8565000000000004e-05 2024-09-11T06:50:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.532400+0000", 2024-09-11T06:50:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00068471299999999999 2024-09-11T06:50:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:533 2.4 2:2a17619e:::rbd_data.1314f0a2bfa2.000000000000000e:head [sparse-read 1314816~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.542984+0000", 2024-09-11T06:50:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 13.869233942999999, 2024-09-11T06:50:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001565873, 2024-09-11T06:50:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 533 2024-09-11T06:50:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.542984+0000", 2024-09-11T06:50:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.542984+0000", 2024-09-11T06:50:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.542985+0000", 2024-09-11T06:50:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.217e-06 2024-09-11T06:50:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.542993+0000", 2024-09-11T06:50:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2799999999999998e-06 2024-09-11T06:50:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.543007+0000", 2024-09-11T06:50:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4046000000000001e-05 2024-09-11T06:50:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.543022+0000", 2024-09-11T06:50:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5206e-05 2024-09-11T06:50:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.543871+0000", 2024-09-11T06:50:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000849459 2024-09-11T06:50:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.544004+0000", 2024-09-11T06:50:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000132958 2024-09-11T06:50:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.544550+0000", 2024-09-11T06:50:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00054570699999999998 2024-09-11T06:50:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.456 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.456 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:549 2.4 2:2a17619e:::rbd_data.1314f0a2bfa2.000000000000000e:head [sparse-read 3411968~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:35.456 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.567982+0000", 2024-09-11T06:50:35.456 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 13.844235640000001, 2024-09-11T06:50:35.456 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002000216, 2024-09-11T06:50:35.456 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:35.456 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:35.456 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:35.456 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:35.456 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:35.456 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:35.456 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 549 2024-09-11T06:50:35.456 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.456 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.567982+0000", 2024-09-11T06:50:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.567982+0000", 2024-09-11T06:50:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.567984+0000", 2024-09-11T06:50:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7290000000000001e-06 2024-09-11T06:50:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.458 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.458 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:35.458 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.567994+0000", 2024-09-11T06:50:35.458 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4229999999999994e-06 2024-09-11T06:50:35.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.458 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.458 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:35.458 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.568015+0000", 2024-09-11T06:50:35.458 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1596000000000001e-05 2024-09-11T06:50:35.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.458 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.458 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:35.458 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.568032+0000", 2024-09-11T06:50:35.458 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7067000000000001e-05 2024-09-11T06:50:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.568713+0000", 2024-09-11T06:50:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00068109799999999999 2024-09-11T06:50:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.568866+0000", 2024-09-11T06:50:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015232399999999999 2024-09-11T06:50:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.569983+0000", 2024-09-11T06:50:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011169789999999999 2024-09-11T06:50:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:550 2.4 2:2a17619e:::rbd_data.1314f0a2bfa2.000000000000000e:head [sparse-read 3543040~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.569595+0000", 2024-09-11T06:50:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 13.842622613, 2024-09-11T06:50:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001587565, 2024-09-11T06:50:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 550 2024-09-11T06:50:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.569595+0000", 2024-09-11T06:50:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.569595+0000", 2024-09-11T06:50:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.569597+0000", 2024-09-11T06:50:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6130000000000001e-06 2024-09-11T06:50:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.569604+0000", 2024-09-11T06:50:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2429999999999999e-06 2024-09-11T06:50:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.569626+0000", 2024-09-11T06:50:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1478999999999999e-05 2024-09-11T06:50:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.569643+0000", 2024-09-11T06:50:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6973000000000002e-05 2024-09-11T06:50:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.570088+0000", 2024-09-11T06:50:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00044539999999999998 2024-09-11T06:50:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.570227+0000", 2024-09-11T06:50:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013836499999999999 2024-09-11T06:50:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.571183+0000", 2024-09-11T06:50:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00095649199999999998 2024-09-11T06:50:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:587 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-11T06:50:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:50:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 13.182967114, 2024-09-11T06:50:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0023436289999999999, 2024-09-11T06:50:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 587 2024-09-11T06:50:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:50:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:50:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229252+0000", 2024-09-11T06:50:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3799999999999999e-06 2024-09-11T06:50:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229259+0000", 2024-09-11T06:50:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4760000000000003e-06 2024-09-11T06:50:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229271+0000", 2024-09-11T06:50:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2313000000000001e-05 2024-09-11T06:50:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229279+0000", 2024-09-11T06:50:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1610000000000003e-06 2024-09-11T06:50:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229344+0000", 2024-09-11T06:50:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4375000000000006e-05 2024-09-11T06:50:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229404+0000", 2024-09-11T06:50:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0517999999999999e-05 2024-09-11T06:50:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229516+0000", 2024-09-11T06:50:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011129900000000001 2024-09-11T06:50:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231502+0000", 2024-09-11T06:50:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001986238 2024-09-11T06:50:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231537+0000", 2024-09-11T06:50:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4968000000000001e-05 2024-09-11T06:50:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231569+0000", 2024-09-11T06:50:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2459000000000002e-05 2024-09-11T06:50:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231595+0000", 2024-09-11T06:50:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5442000000000001e-05 2024-09-11T06:50:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:15 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.709418+0000", 2024-09-11T06:50:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.7027999610000002, 2024-09-11T06:50:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027504280000000001, 2024-09-11T06:50:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 15 2024-09-11T06:50:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709418+0000", 2024-09-11T06:50:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709418+0000", 2024-09-11T06:50:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709420+0000", 2024-09-11T06:50:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2299999999999998e-06 2024-09-11T06:50:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709432+0000", 2024-09-11T06:50:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1897e-05 2024-09-11T06:50:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709464+0000", 2024-09-11T06:50:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1819000000000001e-05 2024-09-11T06:50:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709479+0000", 2024-09-11T06:50:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4853e-05 2024-09-11T06:50:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709624+0000", 2024-09-11T06:50:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144912 2024-09-11T06:50:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709773+0000", 2024-09-11T06:50:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000149529 2024-09-11T06:50:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.710055+0000", 2024-09-11T06:50:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000281692 2024-09-11T06:50:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.711412+0000", 2024-09-11T06:50:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001356931 2024-09-11T06:50:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.711890+0000", 2024-09-11T06:50:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00047768700000000002 2024-09-11T06:50:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.712050+0000", 2024-09-11T06:50:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016041499999999999 2024-09-11T06:50:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.712169+0000", 2024-09-11T06:50:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011846299999999999 2024-09-11T06:50:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:18 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.713502+0000", 2024-09-11T06:50:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.698716535, 2024-09-11T06:50:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032323069999999998, 2024-09-11T06:50:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:35.476 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:35.476 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:35.476 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18 2024-09-11T06:50:35.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.476 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:35.476 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.476 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:35.476 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713502+0000", 2024-09-11T06:50:35.476 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.476 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.476 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:35.476 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713502+0000", 2024-09-11T06:50:35.476 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713503+0000", 2024-09-11T06:50:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8199999999999999e-06 2024-09-11T06:50:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713513+0000", 2024-09-11T06:50:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8539999999999992e-06 2024-09-11T06:50:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713540+0000", 2024-09-11T06:50:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6594999999999999e-05 2024-09-11T06:50:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713553+0000", 2024-09-11T06:50:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2843999999999999e-05 2024-09-11T06:50:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713686+0000", 2024-09-11T06:50:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013294899999999999 2024-09-11T06:50:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713937+0000", 2024-09-11T06:50:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025110600000000002 2024-09-11T06:50:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.714412+0000", 2024-09-11T06:50:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00047562 2024-09-11T06:50:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.715867+0000", 2024-09-11T06:50:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001455004 2024-09-11T06:50:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.716569+0000", 2024-09-11T06:50:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070159400000000005 2024-09-11T06:50:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.480 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:35.480 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.716660+0000", 2024-09-11T06:50:35.480 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1403000000000006e-05 2024-09-11T06:50:35.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.480 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.480 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:35.480 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.716734+0000", 2024-09-11T06:50:35.480 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3517999999999996e-05 2024-09-11T06:50:35.480 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.480 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:35.480 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.480 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.480 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:23 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:35.481 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.723392+0000", 2024-09-11T06:50:35.481 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.688825939, 2024-09-11T06:50:35.481 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027098489999999999, 2024-09-11T06:50:35.481 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:35.481 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:35.481 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:35.481 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:35.481 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:35.481 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:35.481 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23 2024-09-11T06:50:35.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.481 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:35.481 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.481 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723392+0000", 2024-09-11T06:50:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723392+0000", 2024-09-11T06:50:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723394+0000", 2024-09-11T06:50:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0669999999999999e-06 2024-09-11T06:50:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723406+0000", 2024-09-11T06:50:35.483 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1367e-05 2024-09-11T06:50:35.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.483 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.483 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:35.483 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723436+0000", 2024-09-11T06:50:35.483 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0874999999999998e-05 2024-09-11T06:50:35.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.483 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.483 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:35.483 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723451+0000", 2024-09-11T06:50:35.483 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4192999999999999e-05 2024-09-11T06:50:35.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.483 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.483 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723577+0000", 2024-09-11T06:50:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000126267 2024-09-11T06:50:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723718+0000", 2024-09-11T06:50:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000141206 2024-09-11T06:50:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723982+0000", 2024-09-11T06:50:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026338499999999999 2024-09-11T06:50:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:35.485 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.725377+0000", 2024-09-11T06:50:35.485 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013956999999999999 2024-09-11T06:50:35.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.485 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.485 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:35.485 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.725908+0000", 2024-09-11T06:50:35.485 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00053036200000000004 2024-09-11T06:50:35.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.485 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.485 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:35.485 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.725999+0000", 2024-09-11T06:50:35.485 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1016999999999999e-05 2024-09-11T06:50:35.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.485 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.486 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:35.486 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726102+0000", 2024-09-11T06:50:35.486 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010341 2024-09-11T06:50:35.486 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.486 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:35.486 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.486 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.486 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.486 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.5175.0:12 2.1 e67/41 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head v 67'57, mlcod=67'55)", 2024-09-11T06:50:35.486 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.862712+0000", 2024-09-11T06:50:35.486 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.549506053, 2024-09-11T06:50:35.486 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015435150000000001, 2024-09-11T06:50:35.486 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:35.487 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:35.487 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:35.487 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:35.487 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.487 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:35.487 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862712+0000", 2024-09-11T06:50:35.487 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.487 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.487 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.487 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:35.487 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862712+0000", 2024-09-11T06:50:35.487 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.487 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.487 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.487 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:35.487 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862714+0000", 2024-09-11T06:50:35.488 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8470000000000001e-06 2024-09-11T06:50:35.488 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.488 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.488 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:35.488 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862727+0000", 2024-09-11T06:50:35.488 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2702e-05 2024-09-11T06:50:35.488 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.488 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.488 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:35.488 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862753+0000", 2024-09-11T06:50:35.488 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6359999999999998e-05 2024-09-11T06:50:35.488 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.488 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.488 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:35.488 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862766+0000", 2024-09-11T06:50:35.489 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3059e-05 2024-09-11T06:50:35.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.489 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.489 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:35.489 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862858+0000", 2024-09-11T06:50:35.489 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2341000000000002e-05 2024-09-11T06:50:35.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.489 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.489 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:35.489 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862910+0000", 2024-09-11T06:50:35.489 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2132000000000002e-05 2024-09-11T06:50:35.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.489 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.489 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:35.489 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.864164+0000", 2024-09-11T06:50:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012534899999999999 2024-09-11T06:50:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.864256+0000", 2024-09-11T06:50:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1583999999999996e-05 2024-09-11T06:50:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:15 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.868921+0000", 2024-09-11T06:50:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.5432968980000004, 2024-09-11T06:50:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0026879, 2024-09-11T06:50:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 15 2024-09-11T06:50:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868921+0000", 2024-09-11T06:50:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868921+0000", 2024-09-11T06:50:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868923+0000", 2024-09-11T06:50:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8959999999999999e-06 2024-09-11T06:50:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868934+0000", 2024-09-11T06:50:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1083e-05 2024-09-11T06:50:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868965+0000", 2024-09-11T06:50:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0833000000000001e-05 2024-09-11T06:50:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868990+0000", 2024-09-11T06:50:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4545999999999998e-05 2024-09-11T06:50:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.869124+0000", 2024-09-11T06:50:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013489599999999999 2024-09-11T06:50:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.869270+0000", 2024-09-11T06:50:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000145976 2024-09-11T06:50:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.869538+0000", 2024-09-11T06:50:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026780599999999999 2024-09-11T06:50:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.870691+0000", 2024-09-11T06:50:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00115277 2024-09-11T06:50:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.871456+0000", 2024-09-11T06:50:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00076473800000000005 2024-09-11T06:50:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.871543+0000", 2024-09-11T06:50:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7040999999999996e-05 2024-09-11T06:50:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.871609+0000", 2024-09-11T06:50:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6315000000000005e-05 2024-09-11T06:50:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:20 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:35.496 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.877429+0000", 2024-09-11T06:50:35.496 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.5347886089999996, 2024-09-11T06:50:35.496 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032382629999999999, 2024-09-11T06:50:35.496 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:35.496 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:35.496 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:35.496 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:35.496 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:35.496 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:35.496 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 20 2024-09-11T06:50:35.496 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.496 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:35.496 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.496 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877429+0000", 2024-09-11T06:50:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877429+0000", 2024-09-11T06:50:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877431+0000", 2024-09-11T06:50:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9429999999999999e-06 2024-09-11T06:50:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877444+0000", 2024-09-11T06:50:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2396e-05 2024-09-11T06:50:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877474+0000", 2024-09-11T06:50:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0239999999999998e-05 2024-09-11T06:50:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877496+0000", 2024-09-11T06:50:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1914999999999999e-05 2024-09-11T06:50:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877616+0000", 2024-09-11T06:50:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000119814 2024-09-11T06:50:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877836+0000", 2024-09-11T06:50:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022029600000000001 2024-09-11T06:50:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.878482+0000", 2024-09-11T06:50:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00064605200000000004 2024-09-11T06:50:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:35.500 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.879804+0000", 2024-09-11T06:50:35.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013216460000000001 2024-09-11T06:50:35.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.500 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:35.500 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.880519+0000", 2024-09-11T06:50:35.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00071547600000000005 2024-09-11T06:50:35.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.500 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:35.500 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.880606+0000", 2024-09-11T06:50:35.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6907e-05 2024-09-11T06:50:35.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.880668+0000", 2024-09-11T06:50:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1577999999999999e-05 2024-09-11T06:50:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:23 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.885067+0000", 2024-09-11T06:50:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.5271509549999998, 2024-09-11T06:50:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028127320000000001, 2024-09-11T06:50:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23 2024-09-11T06:50:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885067+0000", 2024-09-11T06:50:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885067+0000", 2024-09-11T06:50:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885069+0000", 2024-09-11T06:50:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2000000000000001e-06 2024-09-11T06:50:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885081+0000", 2024-09-11T06:50:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1209999999999999e-05 2024-09-11T06:50:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885111+0000", 2024-09-11T06:50:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0318999999999999e-05 2024-09-11T06:50:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885132+0000", 2024-09-11T06:50:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0649000000000002e-05 2024-09-11T06:50:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885243+0000", 2024-09-11T06:50:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000111284 2024-09-11T06:50:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885387+0000", 2024-09-11T06:50:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144222 2024-09-11T06:50:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885699+0000", 2024-09-11T06:50:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00031239800000000003 2024-09-11T06:50:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.886817+0000", 2024-09-11T06:50:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001117191 2024-09-11T06:50:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.887684+0000", 2024-09-11T06:50:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00086699499999999996 2024-09-11T06:50:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.506 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.506 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:35.506 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.887772+0000", 2024-09-11T06:50:35.506 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8224000000000001e-05 2024-09-11T06:50:35.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.506 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.506 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:35.506 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.887880+0000", 2024-09-11T06:50:35.506 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010804 2024-09-11T06:50:35.506 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.506 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:35.506 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.506 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.5175.0:26 2.1 e67/41 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head v 67'61, mlcod=67'59)", 2024-09-11T06:50:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.893712+0000", 2024-09-11T06:50:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.5185065360000003, 2024-09-11T06:50:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001621379, 2024-09-11T06:50:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893712+0000", 2024-09-11T06:50:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:35.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893712+0000", 2024-09-11T06:50:35.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:35.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893713+0000", 2024-09-11T06:50:35.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7460000000000001e-06 2024-09-11T06:50:35.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:35.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893728+0000", 2024-09-11T06:50:35.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4673e-05 2024-09-11T06:50:35.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:35.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893760+0000", 2024-09-11T06:50:35.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2369000000000001e-05 2024-09-11T06:50:35.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:35.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893778+0000", 2024-09-11T06:50:35.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7770000000000001e-05 2024-09-11T06:50:35.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:35.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893965+0000", 2024-09-11T06:50:35.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00018686000000000001 2024-09-11T06:50:35.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894026+0000", 2024-09-11T06:50:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0636e-05 2024-09-11T06:50:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895258+0000", 2024-09-11T06:50:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001232007 2024-09-11T06:50:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895333+0000", 2024-09-11T06:50:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5318e-05 2024-09-11T06:50:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.511 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:35.511 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.511 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.511 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:35.511 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:35.513 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV=/dev/nbd0 2024-09-11T06:50:35.513 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd 2024-09-11T06:50:35.513 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:50:35.513 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:50:35.513 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:50:35.513 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd0'\'']/id' 2024-09-11T06:50:35.513 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:50:35.514 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:35.507+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:35.539 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=22744 2024-09-11T06:50:35.539 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 22744 2024-09-11T06:50:35.539 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 22744 -C rbd-nbd 2024-09-11T06:50:35.547 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:50:35.547 INFO:tasks.workunit.client.0.smithi176.stdout: 22744 ? 00:00:00 rbd-nbd 2024-09-11T06:50:35.547 INFO:tasks.workunit.client.0.smithi176.stderr:+ unmap_device testrbdnbd21607 22744 2024-09-11T06:50:35.548 INFO:tasks.workunit.client.0.smithi176.stderr:+ local args=testrbdnbd21607 2024-09-11T06:50:35.548 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pid=22744 2024-09-11T06:50:35.548 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device --device-type nbd unmap testrbdnbd21607 2024-09-11T06:50:35.548 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:35.548 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:35.548 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:35.548 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:50:35.550 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:50:35.550 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:35.550 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap testrbdnbd21607 2024-09-11T06:50:35.613 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:35.611+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:35.631 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:50:35.631 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^22744\b' 2024-09-11T06:50:35.631 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^22744\b' 2024-09-11T06:50:35.678 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:35.678 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -C rbd-nbd 2024-09-11T06:50:35.678 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^ *22744\b' 2024-09-11T06:50:35.678 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^ *22744\b' 2024-09-11T06:50:35.684 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:35.684 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 0.5 2024-09-11T06:50:35.692 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:35.692 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:35.692 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:50:35.692 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:35.707 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:50:35.714 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:35.711+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:35.815 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:35.811+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:35.829 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:35.829 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:35.829 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:50:35.830 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:50:35.830 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:35.843 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:50:35.916 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:35.911+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:35.968 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:35.968 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:50:35.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:50:35.969 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:50:35.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.969 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:574 2.1 2:88b29bb7:::rbd_data.1314f0a2bfa2.000000000000000f:head [sparse-read 2363392~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:35.969 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.609749+0000", 2024-09-11T06:50:35.969 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.354417305, 2024-09-11T06:50:35.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0024026590000000001, 2024-09-11T06:50:35.969 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:35.969 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:35.969 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:35.969 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:35.969 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:35.969 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:35.969 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 574 2024-09-11T06:50:35.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.970 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:35.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.970 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:35.970 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.609749+0000", 2024-09-11T06:50:35.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.970 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:35.970 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.609749+0000", 2024-09-11T06:50:35.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.970 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:35.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.609750+0000", 2024-09-11T06:50:35.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2640000000000001e-06 2024-09-11T06:50:35.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:35.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.609758+0000", 2024-09-11T06:50:35.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9929999999999995e-06 2024-09-11T06:50:35.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:35.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.609795+0000", 2024-09-11T06:50:35.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7410999999999998e-05 2024-09-11T06:50:35.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:35.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.609812+0000", 2024-09-11T06:50:35.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6396999999999999e-05 2024-09-11T06:50:35.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:35.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.610974+0000", 2024-09-11T06:50:35.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011627670000000001 2024-09-11T06:50:35.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:35.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.611152+0000", 2024-09-11T06:50:35.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000178013 2024-09-11T06:50:35.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:35.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.612151+0000", 2024-09-11T06:50:35.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000998814 2024-09-11T06:50:35.973 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.973 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:35.973 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.973 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch watch cookie 139735284585312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:35.973 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.521153+0000", 2024-09-11T06:50:35.973 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.4430132740000001, 2024-09-11T06:50:35.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028996400000000002, 2024-09-11T06:50:35.974 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:35.974 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:35.974 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:35.974 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:35.974 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:35.974 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:35.974 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:50:35.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.974 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:35.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:35.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521153+0000", 2024-09-11T06:50:35.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:35.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521153+0000", 2024-09-11T06:50:35.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:35.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521154+0000", 2024-09-11T06:50:35.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.77e-06 2024-09-11T06:50:35.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:35.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521162+0000", 2024-09-11T06:50:35.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6569999999999997e-06 2024-09-11T06:50:35.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.976 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:35.976 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521184+0000", 2024-09-11T06:50:35.976 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1736e-05 2024-09-11T06:50:35.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.976 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.976 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:35.976 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521200+0000", 2024-09-11T06:50:35.976 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5953000000000001e-05 2024-09-11T06:50:35.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.976 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.976 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:35.976 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521328+0000", 2024-09-11T06:50:35.976 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000128489 2024-09-11T06:50:35.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:35.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521565+0000", 2024-09-11T06:50:35.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000236357 2024-09-11T06:50:35.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:35.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521991+0000", 2024-09-11T06:50:35.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00042691099999999998 2024-09-11T06:50:35.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:35.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.523245+0000", 2024-09-11T06:50:35.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001253964 2024-09-11T06:50:35.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.978 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:35.978 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.523847+0000", 2024-09-11T06:50:35.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00060132899999999995 2024-09-11T06:50:35.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.978 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:35.978 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.523938+0000", 2024-09-11T06:50:35.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1577999999999996e-05 2024-09-11T06:50:35.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.978 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:35.978 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.524052+0000", 2024-09-11T06:50:35.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000113896 2024-09-11T06:50:35.978 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.979 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:35.979 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.979 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch watch cookie 140250278007648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:35.979 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.701683+0000", 2024-09-11T06:50:35.979 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.2624826589999998, 2024-09-11T06:50:35.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002794693, 2024-09-11T06:50:35.979 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:35.979 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:35.979 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:35.979 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:35.979 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:35.979 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:35.979 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:50:35.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.980 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:35.980 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.980 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:35.980 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.701683+0000", 2024-09-11T06:50:35.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.980 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.980 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:35.980 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.701683+0000", 2024-09-11T06:50:35.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.981 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:35.981 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.701685+0000", 2024-09-11T06:50:35.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6500000000000001e-06 2024-09-11T06:50:35.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.981 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:35.981 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.701694+0000", 2024-09-11T06:50:35.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0759999999999992e-06 2024-09-11T06:50:35.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.981 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:35.981 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.701717+0000", 2024-09-11T06:50:35.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.349e-05 2024-09-11T06:50:35.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.982 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:35.982 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.701730+0000", 2024-09-11T06:50:35.982 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2649e-05 2024-09-11T06:50:35.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.982 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:35.982 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.701853+0000", 2024-09-11T06:50:35.982 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012253000000000001 2024-09-11T06:50:35.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.982 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:35.982 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.702079+0000", 2024-09-11T06:50:35.982 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022646999999999999 2024-09-11T06:50:35.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:35.983 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.702508+0000", 2024-09-11T06:50:35.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00042935500000000003 2024-09-11T06:50:35.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:35.983 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.703905+0000", 2024-09-11T06:50:35.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001396102 2024-09-11T06:50:35.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:35.983 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.704285+0000", 2024-09-11T06:50:35.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000380436 2024-09-11T06:50:35.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.984 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:35.984 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.704374+0000", 2024-09-11T06:50:35.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8813999999999994e-05 2024-09-11T06:50:35.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.984 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:35.984 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.704478+0000", 2024-09-11T06:50:35.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000104121 2024-09-11T06:50:35.984 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.984 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:35.984 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.984 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:14 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:35.985 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.706169+0000", 2024-09-11T06:50:35.985 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.2579966049999998, 2024-09-11T06:50:35.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0029019599999999999, 2024-09-11T06:50:35.985 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:35.985 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:35.985 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:35.985 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:35.985 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:35.985 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:35.985 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 14 2024-09-11T06:50:35.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.985 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:35.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:35.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706169+0000", 2024-09-11T06:50:35.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:35.986 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706169+0000", 2024-09-11T06:50:35.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:35.986 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706172+0000", 2024-09-11T06:50:35.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.34e-06 2024-09-11T06:50:35.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:35.986 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706183+0000", 2024-09-11T06:50:35.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1556000000000001e-05 2024-09-11T06:50:35.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.987 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:35.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706215+0000", 2024-09-11T06:50:35.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2116000000000002e-05 2024-09-11T06:50:35.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.987 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:35.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706232+0000", 2024-09-11T06:50:35.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7073000000000001e-05 2024-09-11T06:50:35.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.987 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:35.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706356+0000", 2024-09-11T06:50:35.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012411599999999999 2024-09-11T06:50:35.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.988 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.988 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:35.988 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706591+0000", 2024-09-11T06:50:35.988 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000234897 2024-09-11T06:50:35.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.988 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.988 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:35.988 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.707060+0000", 2024-09-11T06:50:35.988 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00046906299999999998 2024-09-11T06:50:35.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.988 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.988 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:35.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.708179+0000", 2024-09-11T06:50:35.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001118232 2024-09-11T06:50:35.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:35.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.708894+0000", 2024-09-11T06:50:35.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00071581900000000003 2024-09-11T06:50:35.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:35.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.708983+0000", 2024-09-11T06:50:35.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8270999999999996e-05 2024-09-11T06:50:35.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:35.990 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709071+0000", 2024-09-11T06:50:35.990 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8477e-05 2024-09-11T06:50:35.990 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.990 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:35.990 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.990 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.990 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:20 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:35.990 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.717149+0000", 2024-09-11T06:50:35.990 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.2470164370000001, 2024-09-11T06:50:35.990 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002913297, 2024-09-11T06:50:35.990 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:35.990 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:35.990 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:35.991 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:35.991 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:35.991 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:35.991 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 20 2024-09-11T06:50:35.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.991 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:35.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.991 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:35.991 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717149+0000", 2024-09-11T06:50:35.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.991 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:35.992 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717149+0000", 2024-09-11T06:50:35.992 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.992 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.992 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:35.992 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717152+0000", 2024-09-11T06:50:35.992 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.21e-06 2024-09-11T06:50:35.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.992 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.992 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:35.992 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717164+0000", 2024-09-11T06:50:35.992 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1912999999999999e-05 2024-09-11T06:50:35.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.992 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.992 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:35.993 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717191+0000", 2024-09-11T06:50:35.993 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7526e-05 2024-09-11T06:50:35.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.993 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:35.993 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717206+0000", 2024-09-11T06:50:35.993 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5113000000000001e-05 2024-09-11T06:50:35.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.993 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:35.993 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717355+0000", 2024-09-11T06:50:35.993 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014896500000000001 2024-09-11T06:50:35.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.993 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:35.993 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717607+0000", 2024-09-11T06:50:35.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000252017 2024-09-11T06:50:35.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.994 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:35.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.718128+0000", 2024-09-11T06:50:35.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00052114499999999998 2024-09-11T06:50:35.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.994 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:35.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.719432+0000", 2024-09-11T06:50:35.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013041629999999999 2024-09-11T06:50:35.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.994 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:35.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.719864+0000", 2024-09-11T06:50:35.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00043115299999999998 2024-09-11T06:50:35.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.995 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:35.995 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.719949+0000", 2024-09-11T06:50:35.995 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4862000000000002e-05 2024-09-11T06:50:35.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.995 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:35.995 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.720063+0000", 2024-09-11T06:50:35.995 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011423 2024-09-11T06:50:35.995 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.995 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:35.995 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:35.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.996 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:24 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:35.996 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.726441+0000", 2024-09-11T06:50:35.996 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.2377249399999997, 2024-09-11T06:50:35.996 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027674599999999998, 2024-09-11T06:50:35.996 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:35.996 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:35.996 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:35.996 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:35.996 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:35.996 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:35.996 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 24 2024-09-11T06:50:35.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.996 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:35.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.997 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:35.997 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726441+0000", 2024-09-11T06:50:35.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.997 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.997 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:35.997 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726441+0000", 2024-09-11T06:50:35.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:35.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.997 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.997 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:35.997 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726443+0000", 2024-09-11T06:50:35.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3769999999999999e-06 2024-09-11T06:50:35.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:35.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726456+0000", 2024-09-11T06:50:35.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2616000000000001e-05 2024-09-11T06:50:35.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:35.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726485+0000", 2024-09-11T06:50:35.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9195000000000001e-05 2024-09-11T06:50:35.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:35.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726499+0000", 2024-09-11T06:50:35.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4087000000000001e-05 2024-09-11T06:50:35.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.999 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:35.999 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726618+0000", 2024-09-11T06:50:35.999 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000118337 2024-09-11T06:50:35.999 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.999 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:35.999 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726772+0000", 2024-09-11T06:50:35.999 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015453799999999999 2024-09-11T06:50:35.999 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:35.999 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:35.999 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.727048+0000", 2024-09-11T06:50:35.999 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00027612900000000001 2024-09-11T06:50:35.999 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:35.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.000 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:36.000 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.728406+0000", 2024-09-11T06:50:36.000 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013579880000000001 2024-09-11T06:50:36.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.000 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:36.000 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.728878+0000", 2024-09-11T06:50:36.000 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00047132599999999999 2024-09-11T06:50:36.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.000 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:36.000 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729044+0000", 2024-09-11T06:50:36.000 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016672900000000001 2024-09-11T06:50:36.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.001 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:36.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729208+0000", 2024-09-11T06:50:36.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016413800000000001 2024-09-11T06:50:36.001 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.001 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:36.001 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.001 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:26 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 140250278007648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:36.001 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:50:36.001 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.234533302, 2024-09-11T06:50:36.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034645140000000001, 2024-09-11T06:50:36.001 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:36.001 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:36.001 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:36.002 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:36.002 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:36.002 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:36.002 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:50:36.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.002 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:36.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:36.002 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:50:36.002 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:36.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:36.002 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:50:36.002 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:36.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:36.003 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729634+0000", 2024-09-11T06:50:36.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0470000000000001e-06 2024-09-11T06:50:36.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:36.003 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729640+0000", 2024-09-11T06:50:36.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8860000000000001e-06 2024-09-11T06:50:36.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:36.003 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729654+0000", 2024-09-11T06:50:36.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:50:36.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.004 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.004 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729664+0000", 2024-09-11T06:50:36.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.594e-06 2024-09-11T06:50:36.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.004 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:36.004 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730422+0000", 2024-09-11T06:50:36.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000758648 2024-09-11T06:50:36.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.004 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:36.004 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730571+0000", 2024-09-11T06:50:36.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000148932 2024-09-11T06:50:36.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:36.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730936+0000", 2024-09-11T06:50:36.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036510900000000002 2024-09-11T06:50:36.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:36.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732159+0000", 2024-09-11T06:50:36.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001222302 2024-09-11T06:50:36.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:36.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732763+0000", 2024-09-11T06:50:36.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00060421599999999998 2024-09-11T06:50:36.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.006 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:36.006 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732916+0000", 2024-09-11T06:50:36.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015264 2024-09-11T06:50:36.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.006 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:36.006 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733097+0000", 2024-09-11T06:50:36.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000181504 2024-09-11T06:50:36.006 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.006 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:36.006 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.006 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:27 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:36.006 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:50:36.007 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.2335593999999999, 2024-09-11T06:50:36.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003072852, 2024-09-11T06:50:36.007 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:36.007 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:36.007 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:36.007 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:36.007 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:36.007 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:36.007 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27 2024-09-11T06:50:36.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.007 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:36.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.007 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:36.007 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:50:36.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:36.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.008 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:36.008 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:50:36.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:36.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.008 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:36.008 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730608+0000", 2024-09-11T06:50:36.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8369999999999999e-06 2024-09-11T06:50:36.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.008 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:36.008 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730617+0000", 2024-09-11T06:50:36.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1859999999999992e-06 2024-09-11T06:50:36.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.009 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:36.009 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730636+0000", 2024-09-11T06:50:36.009 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9296e-05 2024-09-11T06:50:36.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.009 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.009 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730674+0000", 2024-09-11T06:50:36.009 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8609000000000003e-05 2024-09-11T06:50:36.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.009 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:36.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731496+0000", 2024-09-11T06:50:36.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00082112299999999999 2024-09-11T06:50:36.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:36.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731722+0000", 2024-09-11T06:50:36.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022651400000000001 2024-09-11T06:50:36.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:36.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733215+0000", 2024-09-11T06:50:36.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001493257 2024-09-11T06:50:36.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.011 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:36.011 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733452+0000", 2024-09-11T06:50:36.011 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023631600000000001 2024-09-11T06:50:36.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.011 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:36.011 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733679+0000", 2024-09-11T06:50:36.011 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022771400000000001 2024-09-11T06:50:36.011 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.011 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:36.011 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.011 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:73 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:36.011 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:50:36.012 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.2334751160000001, 2024-09-11T06:50:36.012 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033454439999999999, 2024-09-11T06:50:36.012 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:36.012 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:36.012 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:36.012 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:36.012 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:36.012 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:36.012 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73 2024-09-11T06:50:36.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.012 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:36.012 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.012 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:36.012 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:50:36.012 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:36.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.013 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:36.013 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:50:36.013 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:36.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.013 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:36.013 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730694+0000", 2024-09-11T06:50:36.013 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5099999999999999e-06 2024-09-11T06:50:36.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.013 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:36.013 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730708+0000", 2024-09-11T06:50:36.013 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3237e-05 2024-09-11T06:50:36.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.014 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:36.014 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730738+0000", 2024-09-11T06:50:36.014 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0688999999999997e-05 2024-09-11T06:50:36.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.014 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.014 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730759+0000", 2024-09-11T06:50:36.014 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0818999999999998e-05 2024-09-11T06:50:36.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.014 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:36.014 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731841+0000", 2024-09-11T06:50:36.015 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010821819999999999 2024-09-11T06:50:36.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.015 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:36.015 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732092+0000", 2024-09-11T06:50:36.015 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025086299999999999 2024-09-11T06:50:36.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.015 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:36.015 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733730+0000", 2024-09-11T06:50:36.015 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0016374600000000001 2024-09-11T06:50:36.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.015 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:36.016 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733866+0000", 2024-09-11T06:50:36.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000136666 2024-09-11T06:50:36.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.016 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.016 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:36.016 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.734036+0000", 2024-09-11T06:50:36.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000170018 2024-09-11T06:50:36.016 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.016 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:36.016 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.016 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.016 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:77 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch reconnect cookie 139735284585312 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:36.016 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.735944+0000", 2024-09-11T06:50:36.016 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.2282216000000004, 2024-09-11T06:50:36.017 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002511091, 2024-09-11T06:50:36.017 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:36.017 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:36.017 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:36.017 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:36.017 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:36.017 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:36.017 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 77 2024-09-11T06:50:36.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.017 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:36.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.017 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:36.017 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.735944+0000", 2024-09-11T06:50:36.017 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:36.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.018 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:36.018 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.735944+0000", 2024-09-11T06:50:36.018 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:36.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.018 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.018 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:36.018 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.735946+0000", 2024-09-11T06:50:36.018 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0959999999999999e-06 2024-09-11T06:50:36.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.018 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.018 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:36.018 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.735963+0000", 2024-09-11T06:50:36.018 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6494000000000002e-05 2024-09-11T06:50:36.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.018 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.018 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:36.019 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.735996+0000", 2024-09-11T06:50:36.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2728999999999998e-05 2024-09-11T06:50:36.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.019 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.736011+0000", 2024-09-11T06:50:36.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5631999999999999e-05 2024-09-11T06:50:36.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:36.019 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.736161+0000", 2024-09-11T06:50:36.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015007599999999999 2024-09-11T06:50:36.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:36.020 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.736339+0000", 2024-09-11T06:50:36.020 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017737499999999999 2024-09-11T06:50:36.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.020 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:36.020 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.738455+0000", 2024-09-11T06:50:36.020 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0021166890000000002 2024-09-11T06:50:36.020 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.020 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:36.020 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.020 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch watch cookie 139740787512064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:36.020 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.861993+0000", 2024-09-11T06:50:36.020 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.1021729499999999, 2024-09-11T06:50:36.020 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027042189999999999, 2024-09-11T06:50:36.021 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:36.021 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:36.021 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:36.021 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:36.021 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:36.021 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:36.021 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:50:36.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.021 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:36.021 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.021 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:36.021 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.861993+0000", 2024-09-11T06:50:36.021 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:36.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.022 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:36.022 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.861993+0000", 2024-09-11T06:50:36.022 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:36.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.022 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:36.022 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.861995+0000", 2024-09-11T06:50:36.022 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2529999999999999e-06 2024-09-11T06:50:36.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.022 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:36.022 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862006+0000", 2024-09-11T06:50:36.022 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0509999999999999e-05 2024-09-11T06:50:36.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.023 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.023 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:36.023 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862037+0000", 2024-09-11T06:50:36.023 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1396000000000002e-05 2024-09-11T06:50:36.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.023 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.023 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.023 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862056+0000", 2024-09-11T06:50:36.023 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.859e-05 2024-09-11T06:50:36.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.023 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.023 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:36.023 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862168+0000", 2024-09-11T06:50:36.023 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000111996 2024-09-11T06:50:36.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.023 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.024 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:36.024 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862304+0000", 2024-09-11T06:50:36.024 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000136716 2024-09-11T06:50:36.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.024 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.024 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:36.024 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862544+0000", 2024-09-11T06:50:36.024 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024005000000000001 2024-09-11T06:50:36.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.024 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.024 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:36.024 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.863704+0000", 2024-09-11T06:50:36.024 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011596499999999999 2024-09-11T06:50:36.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.025 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.025 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:36.025 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.864508+0000", 2024-09-11T06:50:36.025 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00080411999999999997 2024-09-11T06:50:36.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.025 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.025 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:36.025 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.864597+0000", 2024-09-11T06:50:36.025 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8660999999999997e-05 2024-09-11T06:50:36.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.025 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.025 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:36.025 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.864697+0000", 2024-09-11T06:50:36.025 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000100277 2024-09-11T06:50:36.025 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.026 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:36.026 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.026 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:14 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:36.026 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.865981+0000", 2024-09-11T06:50:36.026 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.0981849910000001, 2024-09-11T06:50:36.026 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0025323239999999999, 2024-09-11T06:50:36.026 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:36.026 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:36.026 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:36.026 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:36.026 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:36.026 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:36.026 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 14 2024-09-11T06:50:36.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.027 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:36.027 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.027 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:36.027 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.865981+0000", 2024-09-11T06:50:36.027 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:36.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.027 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.027 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:36.027 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.865981+0000", 2024-09-11T06:50:36.027 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:36.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.027 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.027 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:36.027 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.865983+0000", 2024-09-11T06:50:36.027 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8870000000000001e-06 2024-09-11T06:50:36.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.028 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:36.028 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.865993+0000", 2024-09-11T06:50:36.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0525e-05 2024-09-11T06:50:36.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.028 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:36.028 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.866016+0000", 2024-09-11T06:50:36.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2764000000000001e-05 2024-09-11T06:50:36.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.028 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.028 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.866035+0000", 2024-09-11T06:50:36.029 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9025000000000002e-05 2024-09-11T06:50:36.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.029 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:36.029 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.866136+0000", 2024-09-11T06:50:36.029 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010080400000000001 2024-09-11T06:50:36.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.029 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:36.029 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.866277+0000", 2024-09-11T06:50:36.029 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000140839 2024-09-11T06:50:36.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.029 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:36.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.866561+0000", 2024-09-11T06:50:36.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00028413599999999999 2024-09-11T06:50:36.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:36.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.867767+0000", 2024-09-11T06:50:36.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012061789999999999 2024-09-11T06:50:36.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:36.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868321+0000", 2024-09-11T06:50:36.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00055425400000000001 2024-09-11T06:50:36.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.031 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:36.031 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868407+0000", 2024-09-11T06:50:36.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6044000000000005e-05 2024-09-11T06:50:36.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.031 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:36.031 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868513+0000", 2024-09-11T06:50:36.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010586699999999999 2024-09-11T06:50:36.031 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.031 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:36.031 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.031 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:18 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:36.032 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:50:36.032 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.0900579270000001, 2024-09-11T06:50:36.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0031646579999999999, 2024-09-11T06:50:36.032 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:36.032 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:36.032 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:36.032 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:36.032 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:36.032 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:36.032 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18 2024-09-11T06:50:36.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.032 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:36.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:36.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:50:36.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:36.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:36.033 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:50:36.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:36.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:36.033 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874109+0000", 2024-09-11T06:50:36.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.426e-06 2024-09-11T06:50:36.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:36.033 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874118+0000", 2024-09-11T06:50:36.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6929999999999996e-06 2024-09-11T06:50:36.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.034 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.034 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:36.034 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874134+0000", 2024-09-11T06:50:36.034 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5503e-05 2024-09-11T06:50:36.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.034 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.034 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.034 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874144+0000", 2024-09-11T06:50:36.034 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0207000000000001e-05 2024-09-11T06:50:36.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.034 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.034 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:36.034 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874545+0000", 2024-09-11T06:50:36.034 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040120800000000001 2024-09-11T06:50:36.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.035 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:36.035 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874686+0000", 2024-09-11T06:50:36.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014128599999999999 2024-09-11T06:50:36.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.035 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:36.035 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875023+0000", 2024-09-11T06:50:36.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033706600000000001 2024-09-11T06:50:36.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.035 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:36.035 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.876217+0000", 2024-09-11T06:50:36.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011938999999999999 2024-09-11T06:50:36.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:36.036 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877042+0000", 2024-09-11T06:50:36.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00082499899999999998 2024-09-11T06:50:36.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:36.036 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877128+0000", 2024-09-11T06:50:36.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5903999999999996e-05 2024-09-11T06:50:36.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:36.036 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877273+0000", 2024-09-11T06:50:36.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144466 2024-09-11T06:50:36.036 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.037 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:36.037 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.037 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:126 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:36.037 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.875489+0000", 2024-09-11T06:50:36.037 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.0886766410000002, 2024-09-11T06:50:36.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002819996, 2024-09-11T06:50:36.037 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:36.037 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:36.037 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:36.037 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:36.037 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:36.037 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:36.037 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 126 2024-09-11T06:50:36.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.038 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:36.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:36.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875489+0000", 2024-09-11T06:50:36.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:36.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:36.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875489+0000", 2024-09-11T06:50:36.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:36.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:36.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875492+0000", 2024-09-11T06:50:36.039 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2600000000000001e-06 2024-09-11T06:50:36.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.039 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.039 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:36.039 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875507+0000", 2024-09-11T06:50:36.039 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4619e-05 2024-09-11T06:50:36.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.039 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.039 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:36.039 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875537+0000", 2024-09-11T06:50:36.039 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.023e-05 2024-09-11T06:50:36.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.039 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.040 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.040 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875587+0000", 2024-09-11T06:50:36.040 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9694999999999998e-05 2024-09-11T06:50:36.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.040 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.040 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:36.040 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875834+0000", 2024-09-11T06:50:36.040 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024709999999999999 2024-09-11T06:50:36.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.040 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.040 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:36.040 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.876037+0000", 2024-09-11T06:50:36.040 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00020236000000000001 2024-09-11T06:50:36.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:36.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877776+0000", 2024-09-11T06:50:36.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00173942 2024-09-11T06:50:36.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:36.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877998+0000", 2024-09-11T06:50:36.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022171100000000001 2024-09-11T06:50:36.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:36.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.878309+0000", 2024-09-11T06:50:36.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000311601 2024-09-11T06:50:36.042 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.042 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:36.042 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.042 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:24 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:36.042 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:50:36.042 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.0755522209999997, 2024-09-11T06:50:36.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002980271, 2024-09-11T06:50:36.042 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:36.042 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:36.042 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:36.042 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:36.042 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:36.042 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:36.043 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 24 2024-09-11T06:50:36.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.043 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:36.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.043 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:36.043 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:50:36.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:36.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.043 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:36.043 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:50:36.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:36.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.044 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:36.044 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888616+0000", 2024-09-11T06:50:36.044 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0030000000000001e-06 2024-09-11T06:50:36.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.044 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.044 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:36.044 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888626+0000", 2024-09-11T06:50:36.044 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0502999999999999e-05 2024-09-11T06:50:36.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.044 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.044 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:36.044 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888655+0000", 2024-09-11T06:50:36.044 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8419999999999999e-05 2024-09-11T06:50:36.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.045 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.045 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.045 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888688+0000", 2024-09-11T06:50:36.045 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3212e-05 2024-09-11T06:50:36.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.045 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.045 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:36.045 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888829+0000", 2024-09-11T06:50:36.045 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014114299999999999 2024-09-11T06:50:36.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.045 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.045 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:36.045 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.889082+0000", 2024-09-11T06:50:36.045 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025279499999999999 2024-09-11T06:50:36.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:36.046 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.889577+0000", 2024-09-11T06:50:36.046 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00049536299999999997 2024-09-11T06:50:36.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:36.046 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.890811+0000", 2024-09-11T06:50:36.046 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001233858 2024-09-11T06:50:36.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:36.046 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891368+0000", 2024-09-11T06:50:36.046 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00055677400000000001 2024-09-11T06:50:36.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.047 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:36.047 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891456+0000", 2024-09-11T06:50:36.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8246999999999997e-05 2024-09-11T06:50:36.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.047 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:36.047 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891594+0000", 2024-09-11T06:50:36.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013795300000000001 2024-09-11T06:50:36.047 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.047 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:36.047 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.048 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.048 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:26 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 139740787512064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:36.048 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:50:36.048 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.0718495770000001, 2024-09-11T06:50:36.048 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003613733, 2024-09-11T06:50:36.048 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:36.048 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:36.048 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:36.048 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:36.048 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:36.048 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:36.048 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:50:36.048 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.048 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:36.049 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.049 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:36.049 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:50:36.049 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:36.049 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.049 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.049 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:36.049 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:50:36.049 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:36.049 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.049 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.049 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:36.049 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892318+0000", 2024-09-11T06:50:36.049 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.623e-06 2024-09-11T06:50:36.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.050 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.050 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:36.050 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892325+0000", 2024-09-11T06:50:36.050 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:50:36.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.050 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.050 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:36.050 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892346+0000", 2024-09-11T06:50:36.050 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1246e-05 2024-09-11T06:50:36.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.050 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.050 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.050 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892362+0000", 2024-09-11T06:50:36.050 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5409000000000001e-05 2024-09-11T06:50:36.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:36.051 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892883+0000", 2024-09-11T06:50:36.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00052087599999999998 2024-09-11T06:50:36.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:36.051 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893123+0000", 2024-09-11T06:50:36.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000240769 2024-09-11T06:50:36.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:36.051 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893526+0000", 2024-09-11T06:50:36.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040300200000000002 2024-09-11T06:50:36.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.052 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.052 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:36.052 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894820+0000", 2024-09-11T06:50:36.052 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012934159999999999 2024-09-11T06:50:36.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.052 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.052 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:36.052 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895608+0000", 2024-09-11T06:50:36.052 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00078788100000000004 2024-09-11T06:50:36.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.053 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.053 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:36.053 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895724+0000", 2024-09-11T06:50:36.053 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011595999999999999 2024-09-11T06:50:36.053 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.053 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.053 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:36.053 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895930+0000", 2024-09-11T06:50:36.053 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000206391 2024-09-11T06:50:36.053 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.053 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:36.053 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.053 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.053 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.053 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:133 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:36.054 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:50:36.054 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.0710515599999999, 2024-09-11T06:50:36.054 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035017360000000001, 2024-09-11T06:50:36.054 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:36.054 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:36.054 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:36.054 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:36.054 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:36.054 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:36.054 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 133 2024-09-11T06:50:36.054 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.054 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:36.054 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.054 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:36.054 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:50:36.055 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:36.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.055 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:36.055 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:50:36.055 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:36.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.055 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:36.055 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893116+0000", 2024-09-11T06:50:36.055 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1629999999999999e-06 2024-09-11T06:50:36.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.055 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:36.055 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893128+0000", 2024-09-11T06:50:36.055 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.11e-05 2024-09-11T06:50:36.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.056 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.056 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:36.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893156+0000", 2024-09-11T06:50:36.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8266e-05 2024-09-11T06:50:36.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.056 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.056 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893171+0000", 2024-09-11T06:50:36.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5426000000000001e-05 2024-09-11T06:50:36.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.056 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.056 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:36.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894561+0000", 2024-09-11T06:50:36.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013899959999999999 2024-09-11T06:50:36.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.057 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.057 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:36.057 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894781+0000", 2024-09-11T06:50:36.057 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021987100000000001 2024-09-11T06:50:36.057 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.057 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.057 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:36.057 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896349+0000", 2024-09-11T06:50:36.057 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015681779999999999 2024-09-11T06:50:36.057 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.057 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.057 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:36.057 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896458+0000", 2024-09-11T06:50:36.057 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000108187 2024-09-11T06:50:36.057 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.058 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.058 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:36.058 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896616+0000", 2024-09-11T06:50:36.058 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015854899999999999 2024-09-11T06:50:36.058 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.058 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:36.058 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.058 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.058 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.058 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:27 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:36.058 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:50:36.058 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.0710401799999998, 2024-09-11T06:50:36.058 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003151989, 2024-09-11T06:50:36.058 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:36.058 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:36.059 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:36.059 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:36.059 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:36.059 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:36.059 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27 2024-09-11T06:50:36.059 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.059 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:36.059 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.059 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:36.059 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:50:36.059 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:36.059 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.059 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.059 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:36.059 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:50:36.059 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:36.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.060 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.060 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:36.060 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893128+0000", 2024-09-11T06:50:36.060 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.013e-06 2024-09-11T06:50:36.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.060 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.060 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:36.060 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893136+0000", 2024-09-11T06:50:36.060 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4190000000000002e-06 2024-09-11T06:50:36.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.060 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.060 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:36.060 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893158+0000", 2024-09-11T06:50:36.060 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1506999999999999e-05 2024-09-11T06:50:36.061 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.061 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.061 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.061 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893170+0000", 2024-09-11T06:50:36.061 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2741999999999999e-05 2024-09-11T06:50:36.061 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.061 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.061 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:36.061 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894255+0000", 2024-09-11T06:50:36.061 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001084453 2024-09-11T06:50:36.061 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.061 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.061 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:36.061 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894461+0000", 2024-09-11T06:50:36.062 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00020586700000000001 2024-09-11T06:50:36.062 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.062 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.062 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:36.062 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896033+0000", 2024-09-11T06:50:36.062 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001572092 2024-09-11T06:50:36.062 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.062 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.062 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:36.062 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896149+0000", 2024-09-11T06:50:36.062 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116636 2024-09-11T06:50:36.062 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.062 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.062 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:36.062 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896278+0000", 2024-09-11T06:50:36.062 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012826 2024-09-11T06:50:36.063 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.063 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:36.063 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.063 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.063 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.063 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:176 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch reconnect cookie 139735284585312 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:36.063 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:50:36.063 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.0396733610000002, 2024-09-11T06:50:36.063 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033620960000000002, 2024-09-11T06:50:36.063 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:36.063 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:36.063 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:36.063 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:36.063 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:36.064 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:36.064 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 176 2024-09-11T06:50:36.064 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.064 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:36.064 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.064 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:36.064 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:50:36.064 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:36.064 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.064 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.064 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:36.064 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:50:36.064 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:36.064 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.064 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.064 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:36.065 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924494+0000", 2024-09-11T06:50:36.065 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.967e-06 2024-09-11T06:50:36.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.065 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.065 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:36.065 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924501+0000", 2024-09-11T06:50:36.065 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2659999999999998e-06 2024-09-11T06:50:36.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.065 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.065 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:36.065 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924520+0000", 2024-09-11T06:50:36.065 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9038999999999998e-05 2024-09-11T06:50:36.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.065 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.065 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.066 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924535+0000", 2024-09-11T06:50:36.066 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5359999999999999e-05 2024-09-11T06:50:36.066 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.066 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.066 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:36.066 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.925544+0000", 2024-09-11T06:50:36.066 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010085109999999999 2024-09-11T06:50:36.066 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.066 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.066 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:36.066 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.925671+0000", 2024-09-11T06:50:36.066 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012756599999999999 2024-09-11T06:50:36.066 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.066 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.066 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:36.067 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.927855+0000", 2024-09-11T06:50:36.067 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0021833870000000002 2024-09-11T06:50:36.067 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.067 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:36.067 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.067 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.067 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.067 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:186 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 139735284585312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:36.067 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.981668+0000", 2024-09-11T06:50:36.067 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.9824977800000001, 2024-09-11T06:50:36.067 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0026256180000000001, 2024-09-11T06:50:36.067 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:36.067 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:36.067 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:36.067 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:36.068 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:36.068 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:36.068 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 186 2024-09-11T06:50:36.068 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.068 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:36.068 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.068 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:36.068 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.981668+0000", 2024-09-11T06:50:36.068 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:36.068 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.068 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.068 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:36.068 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.981668+0000", 2024-09-11T06:50:36.068 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:36.068 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.068 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.069 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:36.069 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.981670+0000", 2024-09-11T06:50:36.069 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3640000000000002e-06 2024-09-11T06:50:36.069 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.069 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.069 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:36.069 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.981682+0000", 2024-09-11T06:50:36.069 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1036000000000001e-05 2024-09-11T06:50:36.069 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.069 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.069 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:36.069 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.981713+0000", 2024-09-11T06:50:36.069 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1569000000000002e-05 2024-09-11T06:50:36.069 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.069 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.070 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.070 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.981728+0000", 2024-09-11T06:50:36.070 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5302999999999999e-05 2024-09-11T06:50:36.070 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.070 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.070 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:36.070 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.981858+0000", 2024-09-11T06:50:36.070 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012991599999999999 2024-09-11T06:50:36.070 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.070 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.070 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:36.070 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.981992+0000", 2024-09-11T06:50:36.070 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000133996 2024-09-11T06:50:36.070 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.070 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.070 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:36.071 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.982247+0000", 2024-09-11T06:50:36.071 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025445300000000003 2024-09-11T06:50:36.071 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.071 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.071 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:36.071 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.983436+0000", 2024-09-11T06:50:36.071 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011896089999999999 2024-09-11T06:50:36.071 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.071 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.071 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:36.071 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.984108+0000", 2024-09-11T06:50:36.071 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00067153099999999995 2024-09-11T06:50:36.071 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.072 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.072 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:36.072 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.984195+0000", 2024-09-11T06:50:36.072 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6804000000000004e-05 2024-09-11T06:50:36.072 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:36.072 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:36.072 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:36.072 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.984294+0000", 2024-09-11T06:50:36.072 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9036999999999994e-05 2024-09-11T06:50:36.072 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.072 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:36.072 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.072 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:36.072 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:36.073 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:36.074 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:50:36.075 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:36.071+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:36.175 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:36.171+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:36.185 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV= 2024-09-11T06:50:36.185 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd snap create rbd/testrbdnbd21607@snap 2024-09-11T06:50:36.208 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:36.208 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:36.208 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:50:36.208 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:36.222 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:50:36.276 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:36.271+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:36.377 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:36.375+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:36.391 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:36.391 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:36.391 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:50:36.391 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:50:36.391 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:36.405 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:50:36.478 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:36.475+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:36.532 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:36.532 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:36.532 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:36.532 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:50:36.532 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.532 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:36.532 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:50:36.533 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 90.358706670999993, 2024-09-11T06:50:36.533 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011420669, 2024-09-11T06:50:36.533 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:36.533 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:36.533 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:36.533 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:36.533 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:36.533 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:36.533 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 37 2024-09-11T06:50:36.533 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.533 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:36.533 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.533 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:36.533 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:50:36.533 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.534 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.534 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.534 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:36.534 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:50:36.534 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.534 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.534 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.534 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:36.534 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170509+0000", 2024-09-11T06:50:36.534 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.13e-06 2024-09-11T06:50:36.534 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.534 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.534 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:36.534 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170521+0000", 2024-09-11T06:50:36.535 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1776999999999999e-05 2024-09-11T06:50:36.535 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.535 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.535 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:36.535 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170538+0000", 2024-09-11T06:50:36.535 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7413000000000001e-05 2024-09-11T06:50:36.535 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.535 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.535 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.535 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170570+0000", 2024-09-11T06:50:36.535 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2313e-05 2024-09-11T06:50:36.535 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.535 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.535 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:36.535 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179768+0000", 2024-09-11T06:50:36.536 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0091978010000000002 2024-09-11T06:50:36.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.536 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.536 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:36.536 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179951+0000", 2024-09-11T06:50:36.536 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018227900000000001 2024-09-11T06:50:36.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.536 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.536 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:36.536 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.180175+0000", 2024-09-11T06:50:36.536 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022432800000000001 2024-09-11T06:50:36.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.536 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.536 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:36.537 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181112+0000", 2024-09-11T06:50:36.537 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00093712500000000002 2024-09-11T06:50:36.537 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.537 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.537 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:36.537 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181789+0000", 2024-09-11T06:50:36.537 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00067705100000000002 2024-09-11T06:50:36.537 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.537 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.537 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:36.537 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181871+0000", 2024-09-11T06:50:36.537 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1816000000000004e-05 2024-09-11T06:50:36.538 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.538 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.538 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:36.538 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181929+0000", 2024-09-11T06:50:36.538 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7636000000000002e-05 2024-09-11T06:50:36.538 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.538 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:36.538 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.538 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.538 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.538 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:36.538 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:36.538 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 90.257447709999994, 2024-09-11T06:50:36.538 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283336, 2024-09-11T06:50:36.538 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:36.539 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:36.539 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:36.539 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:36.539 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:36.539 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:36.539 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 87 2024-09-11T06:50:36.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.539 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:36.539 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.539 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:36.539 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:36.539 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.539 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.539 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:36.540 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:36.540 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.540 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.540 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.540 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:36.540 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271768+0000", 2024-09-11T06:50:36.540 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7599999999999996e-07 2024-09-11T06:50:36.540 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.540 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.540 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:36.540 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271860+0000", 2024-09-11T06:50:36.540 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2639999999999994e-05 2024-09-11T06:50:36.540 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.540 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.540 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:36.541 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271873+0000", 2024-09-11T06:50:36.541 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3139000000000001e-05 2024-09-11T06:50:36.541 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.541 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.541 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.541 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271882+0000", 2024-09-11T06:50:36.541 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2369999999999992e-06 2024-09-11T06:50:36.541 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.541 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.541 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:36.541 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277150+0000", 2024-09-11T06:50:36.541 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052684419999999999 2024-09-11T06:50:36.541 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.541 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.541 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:36.542 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277273+0000", 2024-09-11T06:50:36.542 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012278899999999999 2024-09-11T06:50:36.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.542 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.542 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:36.542 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277466+0000", 2024-09-11T06:50:36.542 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193578 2024-09-11T06:50:36.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.542 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.542 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:36.542 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.280808+0000", 2024-09-11T06:50:36.542 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003341971 2024-09-11T06:50:36.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.542 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.542 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:36.542 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284419+0000", 2024-09-11T06:50:36.543 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003610982 2024-09-11T06:50:36.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.543 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.543 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:36.543 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284531+0000", 2024-09-11T06:50:36.543 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011121299999999999 2024-09-11T06:50:36.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.543 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.543 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:36.543 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284600+0000", 2024-09-11T06:50:36.543 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9592999999999996e-05 2024-09-11T06:50:36.543 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.543 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:36.543 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.543 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.544 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:36.544 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:36.544 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 90.257317458000003, 2024-09-11T06:50:36.544 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283314, 2024-09-11T06:50:36.544 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:36.544 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:36.544 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:36.544 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:36.544 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:36.544 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:36.544 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 88 2024-09-11T06:50:36.544 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.544 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:36.544 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.545 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:36.545 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:36.545 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.545 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.545 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:36.545 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:36.545 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.545 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.545 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:36.545 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271898+0000", 2024-09-11T06:50:36.545 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2999999999999999e-07 2024-09-11T06:50:36.546 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.546 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.546 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:36.546 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271904+0000", 2024-09-11T06:50:36.546 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.767e-06 2024-09-11T06:50:36.546 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.546 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.546 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:36.546 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271913+0000", 2024-09-11T06:50:36.546 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7200000000000001e-06 2024-09-11T06:50:36.546 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.546 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.546 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.546 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271920+0000", 2024-09-11T06:50:36.547 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7100000000000001e-06 2024-09-11T06:50:36.547 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.547 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.547 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:36.547 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278684+0000", 2024-09-11T06:50:36.547 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0067642650000000002 2024-09-11T06:50:36.547 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.547 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.547 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:36.547 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278838+0000", 2024-09-11T06:50:36.547 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015371499999999999 2024-09-11T06:50:36.547 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.547 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.547 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:36.547 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.279040+0000", 2024-09-11T06:50:36.548 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020207499999999999 2024-09-11T06:50:36.548 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.548 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.548 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:36.548 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.281448+0000", 2024-09-11T06:50:36.548 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002407686 2024-09-11T06:50:36.548 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.548 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.548 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:36.548 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284615+0000", 2024-09-11T06:50:36.548 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031675499999999999 2024-09-11T06:50:36.548 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.548 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.548 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:36.549 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284682+0000", 2024-09-11T06:50:36.549 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7032000000000005e-05 2024-09-11T06:50:36.549 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.549 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.549 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:36.549 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284730+0000", 2024-09-11T06:50:36.549 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7790000000000002e-05 2024-09-11T06:50:36.549 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.549 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:36.549 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.549 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.549 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.549 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:36.549 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:50:36.550 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 90.088825260999997, 2024-09-11T06:50:36.550 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011236556, 2024-09-11T06:50:36.550 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:36.550 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:36.550 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:36.550 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:36.550 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:36.550 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:36.550 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 109 2024-09-11T06:50:36.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.550 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:36.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.550 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:36.550 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:50:36.550 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.551 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.551 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:36.551 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:50:36.551 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.551 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.551 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:36.551 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440391+0000", 2024-09-11T06:50:36.551 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8360000000000001e-06 2024-09-11T06:50:36.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.551 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.551 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:36.551 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440560+0000", 2024-09-11T06:50:36.551 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000169085 2024-09-11T06:50:36.552 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.552 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.552 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:36.552 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440585+0000", 2024-09-11T06:50:36.552 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4530000000000001e-05 2024-09-11T06:50:36.552 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.552 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.552 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.552 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440606+0000", 2024-09-11T06:50:36.552 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1277e-05 2024-09-11T06:50:36.552 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.552 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.552 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:36.552 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446479+0000", 2024-09-11T06:50:36.552 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0058734989999999999 2024-09-11T06:50:36.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.553 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:36.553 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446649+0000", 2024-09-11T06:50:36.553 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016948199999999999 2024-09-11T06:50:36.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.553 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:36.553 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446854+0000", 2024-09-11T06:50:36.553 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020491500000000001 2024-09-11T06:50:36.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.553 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:36.553 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451380+0000", 2024-09-11T06:50:36.553 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0045264110000000001 2024-09-11T06:50:36.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.554 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.554 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:36.554 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451479+0000", 2024-09-11T06:50:36.554 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.8975999999999996e-05 2024-09-11T06:50:36.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.554 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.554 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:36.554 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451562+0000", 2024-09-11T06:50:36.554 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3213000000000004e-05 2024-09-11T06:50:36.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.554 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.554 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:36.554 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451626+0000", 2024-09-11T06:50:36.554 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3331999999999996e-05 2024-09-11T06:50:36.554 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.555 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:36.555 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.555 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.555 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:36.555 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:50:36.555 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 90.088576133000004, 2024-09-11T06:50:36.555 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011252866, 2024-09-11T06:50:36.555 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:36.555 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:36.555 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:36.555 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:36.555 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:36.555 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:36.555 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 110 2024-09-11T06:50:36.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.556 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:36.556 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.556 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:36.556 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:50:36.556 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.556 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.556 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.556 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:36.556 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:50:36.556 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.556 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.556 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.556 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:36.556 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440640+0000", 2024-09-11T06:50:36.556 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5430000000000001e-06 2024-09-11T06:50:36.557 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.557 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.557 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:36.557 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440651+0000", 2024-09-11T06:50:36.557 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0672999999999999e-05 2024-09-11T06:50:36.557 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.557 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.557 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:36.557 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440670+0000", 2024-09-11T06:50:36.557 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9726999999999999e-05 2024-09-11T06:50:36.557 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.557 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.557 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.557 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440691+0000", 2024-09-11T06:50:36.557 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0526e-05 2024-09-11T06:50:36.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.558 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.558 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:36.558 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448191+0000", 2024-09-11T06:50:36.558 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0075005990000000002 2024-09-11T06:50:36.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.558 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.558 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:36.558 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448390+0000", 2024-09-11T06:50:36.558 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000198815 2024-09-11T06:50:36.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.558 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.558 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:36.558 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448735+0000", 2024-09-11T06:50:36.559 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034424999999999999 2024-09-11T06:50:36.559 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.559 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.559 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:36.559 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451742+0000", 2024-09-11T06:50:36.559 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030077440000000001 2024-09-11T06:50:36.559 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.559 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.559 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:36.559 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451761+0000", 2024-09-11T06:50:36.559 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9196999999999999e-05 2024-09-11T06:50:36.559 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.559 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.559 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:36.559 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451834+0000", 2024-09-11T06:50:36.560 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2071999999999997e-05 2024-09-11T06:50:36.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.560 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:36.560 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451891+0000", 2024-09-11T06:50:36.560 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7720000000000003e-05 2024-09-11T06:50:36.560 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.560 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:36.560 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.560 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:36.560 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:36.560 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 90.088468258000006, 2024-09-11T06:50:36.560 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014944941, 2024-09-11T06:50:36.561 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:36.561 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:36.561 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:36.561 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:36.561 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:36.561 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:36.561 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 111 2024-09-11T06:50:36.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.561 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:36.561 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.561 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:36.562 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:36.562 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.562 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:36.562 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:36.562 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.562 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:36.562 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440748+0000", 2024-09-11T06:50:36.562 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:50:36.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.563 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:36.563 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440760+0000", 2024-09-11T06:50:36.563 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1897e-05 2024-09-11T06:50:36.563 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.563 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:36.563 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440782+0000", 2024-09-11T06:50:36.563 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2552999999999999e-05 2024-09-11T06:50:36.563 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.563 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.564 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440793+0000", 2024-09-11T06:50:36.564 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0643e-05 2024-09-11T06:50:36.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.564 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:36.564 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449755+0000", 2024-09-11T06:50:36.564 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089622620000000004 2024-09-11T06:50:36.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.564 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:36.564 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449918+0000", 2024-09-11T06:50:36.564 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163086 2024-09-11T06:50:36.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.565 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:36.565 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450150+0000", 2024-09-11T06:50:36.565 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023204500000000001 2024-09-11T06:50:36.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.565 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.565 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:36.565 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454533+0000", 2024-09-11T06:50:36.565 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0043831349999999998 2024-09-11T06:50:36.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.565 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.565 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:36.565 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455511+0000", 2024-09-11T06:50:36.565 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0009776050000000001 2024-09-11T06:50:36.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.566 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.566 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:36.566 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455597+0000", 2024-09-11T06:50:36.566 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6446000000000005e-05 2024-09-11T06:50:36.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.566 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.566 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:36.566 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455691+0000", 2024-09-11T06:50:36.566 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3832000000000006e-05 2024-09-11T06:50:36.566 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.566 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:36.566 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.566 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.567 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:36.567 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:36.567 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 90.088398495000007, 2024-09-11T06:50:36.567 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018885411000000001, 2024-09-11T06:50:36.567 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:36.567 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:36.567 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:36.567 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:36.567 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:36.567 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:36.567 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 112 2024-09-11T06:50:36.567 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.567 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:36.567 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.567 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:36.568 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:36.568 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.568 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.568 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:36.568 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:36.568 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.568 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.568 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:36.568 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440817+0000", 2024-09-11T06:50:36.568 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:50:36.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.568 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.568 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:36.568 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441003+0000", 2024-09-11T06:50:36.569 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018661200000000001 2024-09-11T06:50:36.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.569 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:36.569 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441027+0000", 2024-09-11T06:50:36.569 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3920000000000001e-05 2024-09-11T06:50:36.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.569 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.569 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441062+0000", 2024-09-11T06:50:36.569 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4663000000000003e-05 2024-09-11T06:50:36.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.569 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:36.570 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451981+0000", 2024-09-11T06:50:36.570 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010918926000000001 2024-09-11T06:50:36.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.570 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:36.570 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452111+0000", 2024-09-11T06:50:36.570 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129529 2024-09-11T06:50:36.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.570 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:36.570 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452311+0000", 2024-09-11T06:50:36.570 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020013799999999999 2024-09-11T06:50:36.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.571 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:36.571 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455809+0000", 2024-09-11T06:50:36.571 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003498443 2024-09-11T06:50:36.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.571 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:36.571 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459566+0000", 2024-09-11T06:50:36.571 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003757271 2024-09-11T06:50:36.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.571 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:36.571 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459640+0000", 2024-09-11T06:50:36.571 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3949000000000006e-05 2024-09-11T06:50:36.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.571 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:36.572 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459701+0000", 2024-09-11T06:50:36.572 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1117000000000005e-05 2024-09-11T06:50:36.572 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.572 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:36.572 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.572 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:36.572 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:36.572 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 90.088119644000002, 2024-09-11T06:50:36.572 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018760915, 2024-09-11T06:50:36.572 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:36.572 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:36.572 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:36.572 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:36.573 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:36.573 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:36.573 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 113 2024-09-11T06:50:36.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.573 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:36.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.573 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:36.573 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:36.573 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.573 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:36.573 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:36.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.574 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:36.574 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441096+0000", 2024-09-11T06:50:36.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4130000000000001e-06 2024-09-11T06:50:36.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.574 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:36.574 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441110+0000", 2024-09-11T06:50:36.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3900000000000001e-05 2024-09-11T06:50:36.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.574 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:36.574 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441131+0000", 2024-09-11T06:50:36.575 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0636999999999999e-05 2024-09-11T06:50:36.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.575 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.575 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441147+0000", 2024-09-11T06:50:36.575 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6512999999999999e-05 2024-09-11T06:50:36.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.575 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:36.575 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453274+0000", 2024-09-11T06:50:36.575 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012126764999999999 2024-09-11T06:50:36.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.575 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:36.575 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453498+0000", 2024-09-11T06:50:36.575 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000223568 2024-09-11T06:50:36.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.576 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:36.576 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453872+0000", 2024-09-11T06:50:36.576 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037467099999999998 2024-09-11T06:50:36.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.576 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:36.576 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458709+0000", 2024-09-11T06:50:36.576 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0048365680000000003 2024-09-11T06:50:36.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.576 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:36.576 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459716+0000", 2024-09-11T06:50:36.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010075240000000001 2024-09-11T06:50:36.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:36.577 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459779+0000", 2024-09-11T06:50:36.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2336999999999996e-05 2024-09-11T06:50:36.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:36.577 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459856+0000", 2024-09-11T06:50:36.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7019000000000002e-05 2024-09-11T06:50:36.577 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.577 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:36.577 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.578 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:36.578 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:36.578 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 90.088042904999995, 2024-09-11T06:50:36.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018963114, 2024-09-11T06:50:36.578 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:36.578 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:36.578 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:36.578 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:36.578 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:36.578 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:36.578 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 114 2024-09-11T06:50:36.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.579 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:36.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.579 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:36.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:36.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.579 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:36.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:36.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.579 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:36.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:36.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9299999999999996e-07 2024-09-11T06:50:36.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.580 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:36.580 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441180+0000", 2024-09-11T06:50:36.580 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7200000000000006e-06 2024-09-11T06:50:36.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.580 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:36.580 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441265+0000", 2024-09-11T06:50:36.580 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4753000000000007e-05 2024-09-11T06:50:36.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.580 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441288+0000", 2024-09-11T06:50:36.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2860000000000001e-05 2024-09-11T06:50:36.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.581 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:36.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454585+0000", 2024-09-11T06:50:36.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013297286 2024-09-11T06:50:36.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.581 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:36.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454714+0000", 2024-09-11T06:50:36.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000128465 2024-09-11T06:50:36.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:36.582 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454909+0000", 2024-09-11T06:50:36.582 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019522499999999999 2024-09-11T06:50:36.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:36.582 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459868+0000", 2024-09-11T06:50:36.582 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0049591239999999997 2024-09-11T06:50:36.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:36.582 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459988+0000", 2024-09-11T06:50:36.582 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000120532 2024-09-11T06:50:36.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:36.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460071+0000", 2024-09-11T06:50:36.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2256000000000007e-05 2024-09-11T06:50:36.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:36.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460135+0000", 2024-09-11T06:50:36.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3999999999999997e-05 2024-09-11T06:50:36.583 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.583 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:36.583 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.583 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:36.583 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:36.584 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 90.087885275999994, 2024-09-11T06:50:36.584 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.019036849000000002, 2024-09-11T06:50:36.584 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:36.584 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:36.584 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:36.584 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:36.584 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:36.584 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:36.584 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 115 2024-09-11T06:50:36.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.584 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:36.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.584 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:36.584 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:36.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.585 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:36.585 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:36.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.585 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:36.585 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441330+0000", 2024-09-11T06:50:36.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1930000000000001e-06 2024-09-11T06:50:36.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.585 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:36.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441544+0000", 2024-09-11T06:50:36.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021405799999999999 2024-09-11T06:50:36.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:36.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441572+0000", 2024-09-11T06:50:36.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7317e-05 2024-09-11T06:50:36.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441593+0000", 2024-09-11T06:50:36.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1702999999999999e-05 2024-09-11T06:50:36.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:36.587 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455891+0000", 2024-09-11T06:50:36.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01429708 2024-09-11T06:50:36.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.587 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:36.587 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456046+0000", 2024-09-11T06:50:36.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000155682 2024-09-11T06:50:36.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.587 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:36.587 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456242+0000", 2024-09-11T06:50:36.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000195692 2024-09-11T06:50:36.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.587 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:36.587 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459876+0000", 2024-09-11T06:50:36.588 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036336480000000002 2024-09-11T06:50:36.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.588 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:36.588 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460245+0000", 2024-09-11T06:50:36.588 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036901700000000002 2024-09-11T06:50:36.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.588 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:36.588 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460316+0000", 2024-09-11T06:50:36.588 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1846000000000003e-05 2024-09-11T06:50:36.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.588 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:36.589 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460366+0000", 2024-09-11T06:50:36.589 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9613000000000001e-05 2024-09-11T06:50:36.589 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.589 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:36.589 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.589 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:36.589 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:36.589 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 90.087563196000005, 2024-09-11T06:50:36.589 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01904432, 2024-09-11T06:50:36.589 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:36.589 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:36.589 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:36.590 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:36.590 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:36.590 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:36.590 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 116 2024-09-11T06:50:36.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.590 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:36.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:36.590 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:36.590 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:36.590 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:36.590 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.591 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:36.591 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441653+0000", 2024-09-11T06:50:36.591 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:50:36.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.591 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:36.591 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441665+0000", 2024-09-11T06:50:36.591 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1653e-05 2024-09-11T06:50:36.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.591 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:36.591 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441684+0000", 2024-09-11T06:50:36.591 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9490000000000001e-05 2024-09-11T06:50:36.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.592 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.592 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441699+0000", 2024-09-11T06:50:36.592 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5044e-05 2024-09-11T06:50:36.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.592 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:36.592 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457297+0000", 2024-09-11T06:50:36.592 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015598241000000001 2024-09-11T06:50:36.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.592 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:36.592 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457577+0000", 2024-09-11T06:50:36.592 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027930799999999998 2024-09-11T06:50:36.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.593 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:36.593 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457957+0000", 2024-09-11T06:50:36.593 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00038088 2024-09-11T06:50:36.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.593 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:36.593 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460432+0000", 2024-09-11T06:50:36.593 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024743349999999998 2024-09-11T06:50:36.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.593 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:36.593 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460584+0000", 2024-09-11T06:50:36.593 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015232900000000001 2024-09-11T06:50:36.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.594 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:36.594 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460646+0000", 2024-09-11T06:50:36.594 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1883000000000004e-05 2024-09-11T06:50:36.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.594 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:36.594 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460696+0000", 2024-09-11T06:50:36.594 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9589999999999998e-05 2024-09-11T06:50:36.594 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.594 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:36.594 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.594 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:36.594 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:36.595 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 90.087491130000004, 2024-09-11T06:50:36.595 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.021834473, 2024-09-11T06:50:36.595 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:36.595 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:36.595 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:36.595 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:36.595 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:36.595 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:36.595 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 117 2024-09-11T06:50:36.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.595 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:36.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.595 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:36.595 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:36.595 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.596 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:36.596 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:36.596 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.596 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:36.596 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441725+0000", 2024-09-11T06:50:36.596 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5069999999999999e-06 2024-09-11T06:50:36.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.596 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:36.596 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441738+0000", 2024-09-11T06:50:36.596 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2813000000000001e-05 2024-09-11T06:50:36.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.597 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:36.597 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441759+0000", 2024-09-11T06:50:36.597 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1246999999999998e-05 2024-09-11T06:50:36.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.597 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.597 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441779+0000", 2024-09-11T06:50:36.597 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0392999999999999e-05 2024-09-11T06:50:36.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.597 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:36.597 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458804+0000", 2024-09-11T06:50:36.597 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.017024182999999998 2024-09-11T06:50:36.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.598 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:36.598 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458977+0000", 2024-09-11T06:50:36.598 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000173926 2024-09-11T06:50:36.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.598 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:36.598 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459170+0000", 2024-09-11T06:50:36.598 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000192184 2024-09-11T06:50:36.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.598 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:36.598 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.462300+0000", 2024-09-11T06:50:36.598 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031304969999999999 2024-09-11T06:50:36.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.599 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:36.599 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463408+0000", 2024-09-11T06:50:36.599 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011077839999999999 2024-09-11T06:50:36.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.599 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:36.599 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463497+0000", 2024-09-11T06:50:36.599 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8828999999999999e-05 2024-09-11T06:50:36.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.599 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:36.599 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463558+0000", 2024-09-11T06:50:36.599 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1110000000000003e-05 2024-09-11T06:50:36.599 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.599 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:36.600 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.600 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:36.600 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:50:36.600 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 89.710983442, 2024-09-11T06:50:36.600 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012216898, 2024-09-11T06:50:36.600 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:36.600 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:36.600 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:36.600 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:36.600 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:36.600 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:36.600 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 137 2024-09-11T06:50:36.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.601 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:36.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.601 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:36.601 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:50:36.601 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.601 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:36.601 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:50:36.601 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.601 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:36.601 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818232+0000", 2024-09-11T06:50:36.601 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2839999999999999e-06 2024-09-11T06:50:36.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.602 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:36.602 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818326+0000", 2024-09-11T06:50:36.602 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3531999999999998e-05 2024-09-11T06:50:36.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.602 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:36.602 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818337+0000", 2024-09-11T06:50:36.602 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0655999999999999e-05 2024-09-11T06:50:36.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.602 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.602 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818342+0000", 2024-09-11T06:50:36.602 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4600000000000002e-06 2024-09-11T06:50:36.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.603 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:36.603 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823890+0000", 2024-09-11T06:50:36.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0055480700000000004 2024-09-11T06:50:36.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.603 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:36.603 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823953+0000", 2024-09-11T06:50:36.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2848999999999994e-05 2024-09-11T06:50:36.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.603 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:36.603 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824074+0000", 2024-09-11T06:50:36.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121309 2024-09-11T06:50:36.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.604 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:36.604 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828043+0000", 2024-09-11T06:50:36.604 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0039685240000000002 2024-09-11T06:50:36.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.604 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:36.604 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830293+0000", 2024-09-11T06:50:36.604 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0022503179999999998 2024-09-11T06:50:36.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.604 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:36.604 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830366+0000", 2024-09-11T06:50:36.604 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2539999999999993e-05 2024-09-11T06:50:36.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.605 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:36.605 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830448+0000", 2024-09-11T06:50:36.605 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2355999999999996e-05 2024-09-11T06:50:36.605 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.605 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:36.605 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.605 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:36.605 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:36.605 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 89.710849628000005, 2024-09-11T06:50:36.605 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013136297, 2024-09-11T06:50:36.605 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:36.605 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:36.605 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:36.605 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:36.606 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:36.606 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:36.606 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 138 2024-09-11T06:50:36.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.606 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:36.606 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.606 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:36.606 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:36.606 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.606 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.606 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:36.606 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:36.606 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.607 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:36.607 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:36.607 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.89e-07 2024-09-11T06:50:36.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.607 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:36.607 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818369+0000", 2024-09-11T06:50:36.607 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4170000000000001e-06 2024-09-11T06:50:36.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.607 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:36.607 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818376+0000", 2024-09-11T06:50:36.607 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4270000000000002e-06 2024-09-11T06:50:36.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.608 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.608 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818383+0000", 2024-09-11T06:50:36.608 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3060000000000003e-06 2024-09-11T06:50:36.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.608 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:36.608 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824783+0000", 2024-09-11T06:50:36.608 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064003020000000001 2024-09-11T06:50:36.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.608 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:36.608 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825013+0000", 2024-09-11T06:50:36.608 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023039499999999999 2024-09-11T06:50:36.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.609 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:36.609 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825484+0000", 2024-09-11T06:50:36.609 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000470562 2024-09-11T06:50:36.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.609 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:36.609 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828633+0000", 2024-09-11T06:50:36.609 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031496689999999999 2024-09-11T06:50:36.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.609 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:36.609 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831413+0000", 2024-09-11T06:50:36.609 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027793219999999999 2024-09-11T06:50:36.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.610 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:36.610 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831450+0000", 2024-09-11T06:50:36.610 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7370999999999998e-05 2024-09-11T06:50:36.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.610 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:36.610 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831501+0000", 2024-09-11T06:50:36.610 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1036999999999999e-05 2024-09-11T06:50:36.610 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.610 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:36.610 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.611 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:36.611 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:36.611 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 89.710825147999998, 2024-09-11T06:50:36.611 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013223011, 2024-09-11T06:50:36.611 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:36.611 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:36.611 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:36.611 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:36.611 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:36.611 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:36.611 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 139 2024-09-11T06:50:36.611 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.611 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:36.611 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.611 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:36.612 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:36.612 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.612 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.612 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.612 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:36.612 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:36.612 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.612 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.612 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.612 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:36.612 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818390+0000", 2024-09-11T06:50:36.612 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5600000000000001e-07 2024-09-11T06:50:36.612 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.613 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.613 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:36.613 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818394+0000", 2024-09-11T06:50:36.613 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5929999999999999e-06 2024-09-11T06:50:36.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.613 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.613 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:36.613 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818399+0000", 2024-09-11T06:50:36.613 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7740000000000004e-06 2024-09-11T06:50:36.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.613 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.613 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.613 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818412+0000", 2024-09-11T06:50:36.614 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2625e-05 2024-09-11T06:50:36.614 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.614 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.614 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:36.614 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826205+0000", 2024-09-11T06:50:36.614 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0077930730000000002 2024-09-11T06:50:36.614 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.614 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.614 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:36.614 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826281+0000", 2024-09-11T06:50:36.614 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5876000000000003e-05 2024-09-11T06:50:36.614 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.614 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.614 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:36.614 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826384+0000", 2024-09-11T06:50:36.614 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103838 2024-09-11T06:50:36.615 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.615 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.615 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:36.615 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830642+0000", 2024-09-11T06:50:36.615 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004257083 2024-09-11T06:50:36.615 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.615 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.615 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:36.615 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831509+0000", 2024-09-11T06:50:36.615 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086730900000000003 2024-09-11T06:50:36.615 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.615 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.615 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:36.615 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831547+0000", 2024-09-11T06:50:36.615 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7793000000000002e-05 2024-09-11T06:50:36.616 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.616 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.616 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:36.616 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831612+0000", 2024-09-11T06:50:36.616 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5691000000000005e-05 2024-09-11T06:50:36.616 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.616 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:36.616 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.616 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.616 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.616 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:36.616 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:36.616 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 89.710796454000004, 2024-09-11T06:50:36.616 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016098024999999998, 2024-09-11T06:50:36.617 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:36.617 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:36.617 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:36.617 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:36.617 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:36.617 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:36.617 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 140 2024-09-11T06:50:36.617 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.617 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:36.617 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.617 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:36.617 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:36.617 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.617 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.618 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.618 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:36.618 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:36.618 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.618 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.618 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.618 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:36.618 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:36.618 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5199999999999998e-07 2024-09-11T06:50:36.618 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.618 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.618 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:36.618 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818479+0000", 2024-09-11T06:50:36.618 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0368000000000002e-05 2024-09-11T06:50:36.619 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.619 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.619 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:36.619 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818487+0000", 2024-09-11T06:50:36.619 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2339999999999994e-06 2024-09-11T06:50:36.619 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.619 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.619 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.619 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818491+0000", 2024-09-11T06:50:36.619 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4329999999999997e-06 2024-09-11T06:50:36.619 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.619 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.619 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:36.619 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828080+0000", 2024-09-11T06:50:36.619 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0095890449999999992 2024-09-11T06:50:36.620 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.620 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.620 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:36.620 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828130+0000", 2024-09-11T06:50:36.620 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9239000000000001e-05 2024-09-11T06:50:36.620 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.620 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.620 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:36.620 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828208+0000", 2024-09-11T06:50:36.620 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.852e-05 2024-09-11T06:50:36.620 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.620 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.620 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:36.620 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831625+0000", 2024-09-11T06:50:36.620 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034169410000000002 2024-09-11T06:50:36.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.621 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.621 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:36.621 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834411+0000", 2024-09-11T06:50:36.621 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002786111 2024-09-11T06:50:36.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.621 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.621 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:36.621 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834457+0000", 2024-09-11T06:50:36.621 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6041000000000002e-05 2024-09-11T06:50:36.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.621 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.621 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:36.621 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834516+0000", 2024-09-11T06:50:36.622 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8740999999999999e-05 2024-09-11T06:50:36.622 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.622 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:36.622 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.622 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.622 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.622 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:36.622 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:36.622 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 89.710715386999993, 2024-09-11T06:50:36.622 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016588772000000002, 2024-09-11T06:50:36.622 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:36.622 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:36.622 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:36.622 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:36.622 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:36.623 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:36.623 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 141 2024-09-11T06:50:36.623 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.623 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:36.623 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.623 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:36.623 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:36.623 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.623 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.623 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.623 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:36.623 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:36.623 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.623 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.624 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.624 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:36.624 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818500+0000", 2024-09-11T06:50:36.624 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0800000000000005e-07 2024-09-11T06:50:36.624 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.624 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.624 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:36.624 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818505+0000", 2024-09-11T06:50:36.624 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2429999999999996e-06 2024-09-11T06:50:36.624 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.624 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.624 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:36.624 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818510+0000", 2024-09-11T06:50:36.624 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9339999999999997e-06 2024-09-11T06:50:36.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.625 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.625 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818514+0000", 2024-09-11T06:50:36.625 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7270000000000001e-06 2024-09-11T06:50:36.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.625 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:36.625 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828711+0000", 2024-09-11T06:50:36.625 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010197031 2024-09-11T06:50:36.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.625 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:36.626 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828853+0000", 2024-09-11T06:50:36.626 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014201200000000001 2024-09-11T06:50:36.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.626 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.626 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:36.626 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.829115+0000", 2024-09-11T06:50:36.626 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026257500000000003 2024-09-11T06:50:36.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.626 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.626 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:36.626 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831629+0000", 2024-09-11T06:50:36.626 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025138169999999998 2024-09-11T06:50:36.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.627 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:36.627 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834991+0000", 2024-09-11T06:50:36.627 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033624459999999998 2024-09-11T06:50:36.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.627 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:36.627 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835032+0000", 2024-09-11T06:50:36.627 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1001000000000003e-05 2024-09-11T06:50:36.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.627 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:36.627 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835088+0000", 2024-09-11T06:50:36.627 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5478e-05 2024-09-11T06:50:36.627 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.628 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:36.628 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.628 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.628 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:50:36.628 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:50:36.628 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 89.086894381999997, 2024-09-11T06:50:36.628 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011561900999999999, 2024-09-11T06:50:36.628 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:36.628 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:36.628 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:36.628 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:36.628 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:36.628 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:36.629 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 159 2024-09-11T06:50:36.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.629 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:36.629 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.629 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:36.629 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:50:36.629 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.629 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.629 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:36.629 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:50:36.629 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.629 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.630 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:36.630 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442321+0000", 2024-09-11T06:50:36.630 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8400000000000003e-07 2024-09-11T06:50:36.630 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.630 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.630 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:36.630 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442328+0000", 2024-09-11T06:50:36.630 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5130000000000002e-06 2024-09-11T06:50:36.630 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.630 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.630 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:36.630 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442341+0000", 2024-09-11T06:50:36.630 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.363e-05 2024-09-11T06:50:36.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.631 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.631 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442349+0000", 2024-09-11T06:50:36.631 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4760000000000001e-06 2024-09-11T06:50:36.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.631 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:36.631 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451257+0000", 2024-09-11T06:50:36.631 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089088569999999992 2024-09-11T06:50:36.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.631 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:36.631 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451389+0000", 2024-09-11T06:50:36.632 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013110899999999999 2024-09-11T06:50:36.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.632 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:36.632 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451608+0000", 2024-09-11T06:50:36.632 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000219588 2024-09-11T06:50:36.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.632 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:36.632 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453517+0000", 2024-09-11T06:50:36.632 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0019088779999999999 2024-09-11T06:50:36.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.633 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:36.633 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453727+0000", 2024-09-11T06:50:36.633 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020955100000000001 2024-09-11T06:50:36.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.633 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:36.633 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453815+0000", 2024-09-11T06:50:36.633 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8858999999999997e-05 2024-09-11T06:50:36.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.633 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:36.633 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453882+0000", 2024-09-11T06:50:36.633 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.6556000000000004e-05 2024-09-11T06:50:36.633 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.633 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:36.634 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.634 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.634 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5235.0:1 2.5 2:b2cc3336:::rbd_id.testrbdnbd21607:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-11T06:50:36.634 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:50:36.634 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 1.969913365, 2024-09-11T06:50:36.634 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.68353462499999995, 2024-09-11T06:50:36.634 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:36.634 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:36.634 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:36.634 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:36.634 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5235", 2024-09-11T06:50:36.634 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2008977164", 2024-09-11T06:50:36.634 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:50:36.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.635 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:36.635 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.635 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:36.635 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:50:36.635 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.635 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.635 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:36.635 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:50:36.635 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.635 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.635 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:36.635 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559303+0000", 2024-09-11T06:50:36.636 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1830000000000001e-06 2024-09-11T06:50:36.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.636 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.636 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:36.636 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559314+0000", 2024-09-11T06:50:36.636 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0396e-05 2024-09-11T06:50:36.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.636 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.636 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:36.636 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559335+0000", 2024-09-11T06:50:36.636 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1183e-05 2024-09-11T06:50:36.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.636 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.636 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.637 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559354+0000", 2024-09-11T06:50:36.637 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9225999999999999e-05 2024-09-11T06:50:36.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.637 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:36.637 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559522+0000", 2024-09-11T06:50:36.637 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000168021 2024-09-11T06:50:36.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.637 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:50:36.637 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559543+0000", 2024-09-11T06:50:36.637 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0616999999999999e-05 2024-09-11T06:50:36.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.638 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:36.638 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.238584+0000", 2024-09-11T06:50:36.638 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.67904107499999999 2024-09-11T06:50:36.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.638 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for missing object", 2024-09-11T06:50:36.638 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.238933+0000", 2024-09-11T06:50:36.638 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034879600000000001 2024-09-11T06:50:36.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.638 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:36.638 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.241968+0000", 2024-09-11T06:50:36.638 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030350830000000001 2024-09-11T06:50:36.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.639 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:36.639 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.242281+0000", 2024-09-11T06:50:36.639 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00031299499999999999 2024-09-11T06:50:36.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.639 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:36.639 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.242836+0000", 2024-09-11T06:50:36.639 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00055504999999999997 2024-09-11T06:50:36.639 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.639 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:36.639 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.639 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5235.0:18 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 0~1024 [fadvise_sequential]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T06:50:36.640 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:50:36.640 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 1.267420926, 2024-09-11T06:50:36.640 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.048993760999999997, 2024-09-11T06:50:36.640 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:36.640 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:36.640 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:36.640 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:36.640 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5235", 2024-09-11T06:50:36.640 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2008977164", 2024-09-11T06:50:36.640 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 18 2024-09-11T06:50:36.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.640 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:36.640 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.640 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:36.641 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:50:36.641 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.641 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:36.641 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:50:36.641 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:36.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.641 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:36.641 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261797+0000", 2024-09-11T06:50:36.641 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2059999999999999e-06 2024-09-11T06:50:36.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.642 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:36.642 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261812+0000", 2024-09-11T06:50:36.642 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5159e-05 2024-09-11T06:50:36.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.642 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:36.642 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261852+0000", 2024-09-11T06:50:36.642 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.9906e-05 2024-09-11T06:50:36.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.642 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:36.642 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261865+0000", 2024-09-11T06:50:36.642 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.312e-05 2024-09-11T06:50:36.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.643 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:36.643 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.263166+0000", 2024-09-11T06:50:36.643 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001301134 2024-09-11T06:50:36.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.643 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for missing object", 2024-09-11T06:50:36.643 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.263529+0000", 2024-09-11T06:50:36.643 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036279900000000002 2024-09-11T06:50:36.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.643 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:36.643 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.300567+0000", 2024-09-11T06:50:36.643 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.037038326000000003 2024-09-11T06:50:36.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.644 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:36.644 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.300800+0000", 2024-09-11T06:50:36.644 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023323300000000001 2024-09-11T06:50:36.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.644 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:36.644 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.306709+0000", 2024-09-11T06:50:36.644 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0059087690000000003 2024-09-11T06:50:36.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.644 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:36.644 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.306850+0000", 2024-09-11T06:50:36.644 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014106200000000001 2024-09-11T06:50:36.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:36.645 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.307121+0000", 2024-09-11T06:50:36.645 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027083599999999999 2024-09-11T06:50:36.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:36.645 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.308809+0000", 2024-09-11T06:50:36.645 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0016880759999999999 2024-09-11T06:50:36.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:36.645 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310382+0000", 2024-09-11T06:50:36.645 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0015732840000000001 2024-09-11T06:50:36.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.646 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:36.646 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310501+0000", 2024-09-11T06:50:36.646 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000118276 2024-09-11T06:50:36.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:36.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:36.646 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:36.646 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310787+0000", 2024-09-11T06:50:36.646 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00028657500000000001 2024-09-11T06:50:36.646 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.646 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:36.646 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.646 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:36.647 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:36.647 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:36.647 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:50:36.648 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:50:36.649 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:36.647+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:36.749 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:36.747+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:36.775 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:36.775 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:36.775 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:50:36.775 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:36.789 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:50:36.800 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:50:36.800 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:50:36.817 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:50:36.817 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:50:36.833 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:50:36.833 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:50:36.850 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:36.847+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:36.851 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:50:36.851 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:50:36.869 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:50:36.869 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:50:36.888 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:50:36.888 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:50:36.943 INFO:tasks.workunit.client.0.smithi176.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2024-09-11T06:50:36.951 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:36.947+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:36.957 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo rbd device --device-type nbd map rbd/testrbdnbd21607@snap 2024-09-11T06:50:36.957 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:50:36.957 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:50:36.958 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:36.958 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:36.959 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:50:36.959 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which rbd 2024-09-11T06:50:36.961 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/rbd 2024-09-11T06:50:36.961 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:50:36.961 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map rbd/testrbdnbd21607@snap 2024-09-11T06:50:36.973 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:50:37.052 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:37.047+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:37.117 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV=/dev/nbd0 2024-09-11T06:50:37.117 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd 2024-09-11T06:50:37.117 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:50:37.117 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:50:37.117 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:50:37.117 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd0'\'']/id' 2024-09-11T06:50:37.144 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:37.144 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:37.154 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:37.151+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:37.158 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:50:37.186 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=22841 2024-09-11T06:50:37.186 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 22841 2024-09-11T06:50:37.186 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 22841 -C rbd-nbd 2024-09-11T06:50:37.192 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:50:37.192 INFO:tasks.workunit.client.0.smithi176.stdout: 22841 ? 00:00:00 rbd-nbd 2024-09-11T06:50:37.193 INFO:tasks.workunit.client.0.smithi176.stderr:+ unmap_device testrbdnbd21607@snap 22841 2024-09-11T06:50:37.193 INFO:tasks.workunit.client.0.smithi176.stderr:+ local args=testrbdnbd21607@snap 2024-09-11T06:50:37.193 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pid=22841 2024-09-11T06:50:37.193 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device --device-type nbd unmap testrbdnbd21607@snap 2024-09-11T06:50:37.193 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:37.194 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:37.194 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:37.194 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:50:37.196 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:50:37.196 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:37.196 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap testrbdnbd21607@snap 2024-09-11T06:50:37.250 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T06:50:37.254 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:37.251+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:37.279 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:50:37.279 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^22841\b' 2024-09-11T06:50:37.279 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^22841\b' 2024-09-11T06:50:37.325 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:37.325 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:37.326 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:37.326 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -C rbd-nbd 2024-09-11T06:50:37.326 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^ *22841\b' 2024-09-11T06:50:37.327 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^ *22841\b' 2024-09-11T06:50:37.332 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:37.333 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 0.5 2024-09-11T06:50:37.339 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:50:37.380 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:37.379+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:37.455 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:37.451+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:37.504 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:37.585 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:37.586 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:50:37.586 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:37.583+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:37.687 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:37.683+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:37.755 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:37.755 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:37.770 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:50:37.812 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:37.811+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:37.833 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV= 2024-09-11T06:50:37.834 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd snap ls rbd/testrbdnbd21607 2024-09-11T06:50:37.835 INFO:tasks.workunit.client.0.smithi176.stderr:++ awk '$2 == "snap" {print $1}' 2024-09-11T06:50:37.889 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:37.887+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:37.898 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:37.898 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:37.899 INFO:tasks.workunit.client.0.smithi176.stderr:+ SNAPID=3 2024-09-11T06:50:37.900 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo rbd device --device-type nbd map --snap-id 3 rbd/testrbdnbd21607 2024-09-11T06:50:37.900 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:50:37.900 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:50:37.901 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:50:37.902 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which rbd 2024-09-11T06:50:37.903 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/rbd 2024-09-11T06:50:37.903 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:50:37.903 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map --snap-id 3 rbd/testrbdnbd21607 2024-09-11T06:50:37.912 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:50:37.931 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is unset 2024-09-11T06:50:37.997 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:37.995+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:38.037 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:38.038 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:38.041 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV=/dev/nbd0 2024-09-11T06:50:38.041 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd 2024-09-11T06:50:38.041 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:50:38.041 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:50:38.041 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:50:38.041 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd0'\'']/id' 2024-09-11T06:50:38.052 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:50:38.093 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:38.091+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:38.112 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=22940 2024-09-11T06:50:38.112 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 22940 2024-09-11T06:50:38.112 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 22940 -C rbd-nbd 2024-09-11T06:50:38.117 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:50:38.117 INFO:tasks.workunit.client.0.smithi176.stdout: 22940 ? 00:00:00 rbd-nbd 2024-09-11T06:50:38.118 INFO:tasks.workunit.client.0.smithi176.stderr:+ unmap_device '--snap-id 3 testrbdnbd21607' 22940 2024-09-11T06:50:38.118 INFO:tasks.workunit.client.0.smithi176.stderr:+ local 'args=--snap-id 3 testrbdnbd21607' 2024-09-11T06:50:38.118 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pid=22940 2024-09-11T06:50:38.118 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device --device-type nbd unmap --snap-id 3 testrbdnbd21607 2024-09-11T06:50:38.118 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:38.118 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:38.119 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:38.119 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:50:38.121 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:50:38.121 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:38.121 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap --snap-id 3 testrbdnbd21607 2024-09-11T06:50:38.192 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:38.191+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:38.209 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:38.209 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:38.212 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:50:38.212 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^22940\b' 2024-09-11T06:50:38.212 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^22940\b' 2024-09-11T06:50:38.223 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:50:38.263 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:38.264 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -C rbd-nbd 2024-09-11T06:50:38.264 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^ *22940\b' 2024-09-11T06:50:38.264 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^ *22940\b' 2024-09-11T06:50:38.271 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:38.271 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 0.5 2024-09-11T06:50:38.293 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:38.291+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:38.341 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:38.341 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:38.357 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:50:38.405 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:38.403+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:38.480 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:38.480 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:38.495 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:50:38.495 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:38.491+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:38.596 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:38.595+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:38.662 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:38.662 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:38.676 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:50:38.696 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:38.695+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:38.772 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV= 2024-09-11T06:50:38.772 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd snap create rbd/ns/testrbdnbd21607@snap 2024-09-11T06:50:38.797 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:38.795+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:38.798 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:38.798 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:38.813 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:50:38.898 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:38.895+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:38.905 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:50:38.933 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:38.933 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:38.948 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:50:38.999 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:38.995+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:39.045 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:50:39.046 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:50:39.062 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:50:39.063 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:50:39.080 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:50:39.080 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:50:39.100 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:39.099+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:39.101 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:50:39.101 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:50:39.112 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:39.112 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:50:39.112 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:50:39.113 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:50:39.113 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.113 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch watch cookie 139735284585312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:39.113 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.521153+0000", 2024-09-11T06:50:39.113 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.5890136459999997, 2024-09-11T06:50:39.113 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028996400000000002, 2024-09-11T06:50:39.113 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:39.113 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:39.113 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.113 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:39.113 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:39.113 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:39.113 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:50:39.114 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.114 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:39.114 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.114 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:39.114 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521153+0000", 2024-09-11T06:50:39.114 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.114 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.114 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.114 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:39.114 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521153+0000", 2024-09-11T06:50:39.114 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.114 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.114 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.115 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:39.115 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521154+0000", 2024-09-11T06:50:39.115 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.77e-06 2024-09-11T06:50:39.115 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.115 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.115 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:39.115 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521162+0000", 2024-09-11T06:50:39.115 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6569999999999997e-06 2024-09-11T06:50:39.115 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.115 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.115 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:39.115 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521184+0000", 2024-09-11T06:50:39.115 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1736e-05 2024-09-11T06:50:39.115 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.116 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.116 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.116 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521200+0000", 2024-09-11T06:50:39.116 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5953000000000001e-05 2024-09-11T06:50:39.116 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.116 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.116 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:39.116 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521328+0000", 2024-09-11T06:50:39.116 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000128489 2024-09-11T06:50:39.116 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.116 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.116 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:39.116 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521565+0000", 2024-09-11T06:50:39.116 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000236357 2024-09-11T06:50:39.117 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.117 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.117 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:39.117 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521991+0000", 2024-09-11T06:50:39.117 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00042691099999999998 2024-09-11T06:50:39.117 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.117 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.117 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:39.117 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.523245+0000", 2024-09-11T06:50:39.117 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001253964 2024-09-11T06:50:39.117 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.117 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.117 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.118 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.523847+0000", 2024-09-11T06:50:39.118 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00060132899999999995 2024-09-11T06:50:39.118 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.118 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.118 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:39.118 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.523938+0000", 2024-09-11T06:50:39.118 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1577999999999996e-05 2024-09-11T06:50:39.118 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.118 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.118 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:39.118 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.524052+0000", 2024-09-11T06:50:39.118 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000113896 2024-09-11T06:50:39.118 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.118 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:39.119 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.119 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.119 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.119 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch watch cookie 140250278007648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:39.119 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.701683+0000", 2024-09-11T06:50:39.119 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.4084830309999994, 2024-09-11T06:50:39.119 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002794693, 2024-09-11T06:50:39.119 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:39.119 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:39.119 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.119 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:39.119 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:39.119 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:39.119 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:50:39.120 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.120 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:39.120 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.120 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:39.120 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.701683+0000", 2024-09-11T06:50:39.120 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.120 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.120 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.120 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:39.120 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.701683+0000", 2024-09-11T06:50:39.120 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.120 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.120 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.121 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:39.121 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.701685+0000", 2024-09-11T06:50:39.121 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6500000000000001e-06 2024-09-11T06:50:39.121 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.121 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.121 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:39.121 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.701694+0000", 2024-09-11T06:50:39.121 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0759999999999992e-06 2024-09-11T06:50:39.121 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.121 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.121 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:39.121 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.701717+0000", 2024-09-11T06:50:39.121 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.349e-05 2024-09-11T06:50:39.122 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.122 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.122 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.122 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.701730+0000", 2024-09-11T06:50:39.122 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2649e-05 2024-09-11T06:50:39.122 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.122 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.122 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:39.122 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.701853+0000", 2024-09-11T06:50:39.122 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012253000000000001 2024-09-11T06:50:39.122 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.122 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.122 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:39.122 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.702079+0000", 2024-09-11T06:50:39.122 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022646999999999999 2024-09-11T06:50:39.123 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.123 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.123 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:39.123 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.702508+0000", 2024-09-11T06:50:39.123 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00042935500000000003 2024-09-11T06:50:39.123 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.123 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.123 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:39.123 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.703905+0000", 2024-09-11T06:50:39.123 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001396102 2024-09-11T06:50:39.123 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.123 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.123 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.124 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.704285+0000", 2024-09-11T06:50:39.124 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000380436 2024-09-11T06:50:39.124 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.124 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.124 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:39.124 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.704374+0000", 2024-09-11T06:50:39.124 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8813999999999994e-05 2024-09-11T06:50:39.124 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.124 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.124 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:39.124 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.704478+0000", 2024-09-11T06:50:39.124 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000104121 2024-09-11T06:50:39.124 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.124 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:39.124 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.125 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.125 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.125 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:14 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:39.125 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.706169+0000", 2024-09-11T06:50:39.125 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.4039969770000003, 2024-09-11T06:50:39.125 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0029019599999999999, 2024-09-11T06:50:39.125 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:39.125 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:39.125 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.125 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:39.125 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:39.125 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:39.125 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 14 2024-09-11T06:50:39.125 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.126 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:39.126 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.126 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:39.126 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706169+0000", 2024-09-11T06:50:39.126 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.126 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.126 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.126 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:39.126 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706169+0000", 2024-09-11T06:50:39.126 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.126 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.126 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.126 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:39.126 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706172+0000", 2024-09-11T06:50:39.127 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.34e-06 2024-09-11T06:50:39.127 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.127 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.127 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:39.127 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706183+0000", 2024-09-11T06:50:39.127 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1556000000000001e-05 2024-09-11T06:50:39.127 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.127 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.127 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:39.127 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706215+0000", 2024-09-11T06:50:39.127 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2116000000000002e-05 2024-09-11T06:50:39.127 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.127 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.127 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.128 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706232+0000", 2024-09-11T06:50:39.128 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7073000000000001e-05 2024-09-11T06:50:39.128 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.128 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.128 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:39.128 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706356+0000", 2024-09-11T06:50:39.128 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012411599999999999 2024-09-11T06:50:39.128 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.128 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.128 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:39.128 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706591+0000", 2024-09-11T06:50:39.128 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000234897 2024-09-11T06:50:39.128 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.129 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.129 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:39.129 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.707060+0000", 2024-09-11T06:50:39.129 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00046906299999999998 2024-09-11T06:50:39.129 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.129 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.129 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:39.129 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.708179+0000", 2024-09-11T06:50:39.129 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001118232 2024-09-11T06:50:39.129 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.129 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.129 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.129 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.708894+0000", 2024-09-11T06:50:39.129 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00071581900000000003 2024-09-11T06:50:39.130 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.130 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.130 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:39.130 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.708983+0000", 2024-09-11T06:50:39.130 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8270999999999996e-05 2024-09-11T06:50:39.130 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.130 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.130 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:39.130 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709071+0000", 2024-09-11T06:50:39.130 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8477e-05 2024-09-11T06:50:39.130 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.130 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:39.130 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.130 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.131 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.131 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:20 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:39.131 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.717149+0000", 2024-09-11T06:50:39.131 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.3930168090000006, 2024-09-11T06:50:39.131 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002913297, 2024-09-11T06:50:39.131 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:39.131 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:39.131 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.131 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:39.131 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:39.131 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:39.131 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 20 2024-09-11T06:50:39.131 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.132 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:39.132 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.132 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:39.132 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717149+0000", 2024-09-11T06:50:39.132 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.132 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.132 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.132 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:39.132 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717149+0000", 2024-09-11T06:50:39.132 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.132 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.132 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.132 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:39.133 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717152+0000", 2024-09-11T06:50:39.133 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.21e-06 2024-09-11T06:50:39.133 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.133 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.133 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:39.133 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717164+0000", 2024-09-11T06:50:39.133 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1912999999999999e-05 2024-09-11T06:50:39.133 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.133 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.133 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:39.133 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717191+0000", 2024-09-11T06:50:39.133 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7526e-05 2024-09-11T06:50:39.134 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.134 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.134 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.134 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717206+0000", 2024-09-11T06:50:39.134 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5113000000000001e-05 2024-09-11T06:50:39.134 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.134 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.134 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:39.134 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717355+0000", 2024-09-11T06:50:39.134 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014896500000000001 2024-09-11T06:50:39.134 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.134 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.134 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:39.135 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717607+0000", 2024-09-11T06:50:39.135 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000252017 2024-09-11T06:50:39.135 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.135 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.135 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:39.135 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.718128+0000", 2024-09-11T06:50:39.135 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00052114499999999998 2024-09-11T06:50:39.135 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.135 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.135 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:39.135 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.719432+0000", 2024-09-11T06:50:39.136 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013041629999999999 2024-09-11T06:50:39.136 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.136 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.136 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.136 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.719864+0000", 2024-09-11T06:50:39.136 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00043115299999999998 2024-09-11T06:50:39.136 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.136 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.136 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:39.136 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.719949+0000", 2024-09-11T06:50:39.136 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4862000000000002e-05 2024-09-11T06:50:39.137 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.137 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.137 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:39.137 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.720063+0000", 2024-09-11T06:50:39.137 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011423 2024-09-11T06:50:39.137 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.137 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:39.137 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.137 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.137 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.137 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:24 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:39.137 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.726441+0000", 2024-09-11T06:50:39.137 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.3837253119999993, 2024-09-11T06:50:39.138 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027674599999999998, 2024-09-11T06:50:39.138 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:39.138 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:39.138 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.138 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:39.138 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:39.138 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:39.138 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 24 2024-09-11T06:50:39.138 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.138 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:39.138 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.138 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:39.138 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726441+0000", 2024-09-11T06:50:39.139 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.139 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.139 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.139 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:39.139 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726441+0000", 2024-09-11T06:50:39.139 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.139 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.139 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.139 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:39.139 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726443+0000", 2024-09-11T06:50:39.139 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3769999999999999e-06 2024-09-11T06:50:39.139 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.139 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.139 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:39.139 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726456+0000", 2024-09-11T06:50:39.140 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2616000000000001e-05 2024-09-11T06:50:39.140 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.140 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.140 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:39.140 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726485+0000", 2024-09-11T06:50:39.140 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9195000000000001e-05 2024-09-11T06:50:39.140 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.140 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.140 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.140 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726499+0000", 2024-09-11T06:50:39.140 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4087000000000001e-05 2024-09-11T06:50:39.140 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.140 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.141 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:39.141 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726618+0000", 2024-09-11T06:50:39.141 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000118337 2024-09-11T06:50:39.141 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.141 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.141 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:39.141 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726772+0000", 2024-09-11T06:50:39.141 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015453799999999999 2024-09-11T06:50:39.141 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.141 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.141 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:39.141 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.727048+0000", 2024-09-11T06:50:39.141 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00027612900000000001 2024-09-11T06:50:39.141 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.142 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.142 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:39.142 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.728406+0000", 2024-09-11T06:50:39.142 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013579880000000001 2024-09-11T06:50:39.142 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.142 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.142 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.142 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.728878+0000", 2024-09-11T06:50:39.142 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00047132599999999999 2024-09-11T06:50:39.142 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.142 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.142 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:39.142 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729044+0000", 2024-09-11T06:50:39.143 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016672900000000001 2024-09-11T06:50:39.143 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.143 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.143 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:39.143 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729208+0000", 2024-09-11T06:50:39.143 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016413800000000001 2024-09-11T06:50:39.143 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.143 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:39.143 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.143 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.143 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.143 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:26 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 140250278007648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:39.143 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:50:39.143 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.3805336740000005, 2024-09-11T06:50:39.144 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034645140000000001, 2024-09-11T06:50:39.144 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:39.144 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:39.144 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.144 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:39.144 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:39.144 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:39.144 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:50:39.144 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.144 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:39.144 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.144 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:39.144 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:50:39.144 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.145 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.145 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.145 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:39.145 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:50:39.145 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.145 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.145 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.145 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:39.145 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729634+0000", 2024-09-11T06:50:39.145 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0470000000000001e-06 2024-09-11T06:50:39.145 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.145 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.145 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:39.145 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729640+0000", 2024-09-11T06:50:39.146 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8860000000000001e-06 2024-09-11T06:50:39.146 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.146 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.146 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:39.146 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729654+0000", 2024-09-11T06:50:39.146 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:50:39.146 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.146 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.146 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.146 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729664+0000", 2024-09-11T06:50:39.146 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.594e-06 2024-09-11T06:50:39.146 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.146 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.147 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:39.147 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730422+0000", 2024-09-11T06:50:39.147 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000758648 2024-09-11T06:50:39.147 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.147 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.147 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:39.147 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730571+0000", 2024-09-11T06:50:39.147 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000148932 2024-09-11T06:50:39.147 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.147 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.147 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:39.147 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730936+0000", 2024-09-11T06:50:39.147 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036510900000000002 2024-09-11T06:50:39.147 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.148 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.148 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:39.148 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732159+0000", 2024-09-11T06:50:39.148 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001222302 2024-09-11T06:50:39.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.148 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.148 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.148 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732763+0000", 2024-09-11T06:50:39.148 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00060421599999999998 2024-09-11T06:50:39.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.148 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.148 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:39.148 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732916+0000", 2024-09-11T06:50:39.149 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015264 2024-09-11T06:50:39.149 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.149 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.149 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:39.149 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733097+0000", 2024-09-11T06:50:39.149 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000181504 2024-09-11T06:50:39.149 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.149 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:39.149 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.149 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.149 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.149 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:27 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:39.149 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:50:39.149 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.3795597720000004, 2024-09-11T06:50:39.150 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003072852, 2024-09-11T06:50:39.150 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:39.150 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:39.150 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:39.150 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:39.150 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:39.150 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:39.150 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27 2024-09-11T06:50:39.150 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.150 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:39.150 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.150 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:39.150 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:50:39.150 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.151 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.151 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.151 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:39.151 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:50:39.151 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.151 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.151 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.151 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:39.151 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730608+0000", 2024-09-11T06:50:39.151 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8369999999999999e-06 2024-09-11T06:50:39.151 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.151 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.151 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:39.151 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730617+0000", 2024-09-11T06:50:39.152 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1859999999999992e-06 2024-09-11T06:50:39.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.152 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.152 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:39.152 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730636+0000", 2024-09-11T06:50:39.152 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9296e-05 2024-09-11T06:50:39.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.152 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.152 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.152 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730674+0000", 2024-09-11T06:50:39.152 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8609000000000003e-05 2024-09-11T06:50:39.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.152 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.152 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:39.153 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731496+0000", 2024-09-11T06:50:39.153 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00082112299999999999 2024-09-11T06:50:39.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.153 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.153 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:39.153 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731722+0000", 2024-09-11T06:50:39.153 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022651400000000001 2024-09-11T06:50:39.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.153 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.153 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:39.153 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733215+0000", 2024-09-11T06:50:39.153 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001493257 2024-09-11T06:50:39.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.153 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.154 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:39.154 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733452+0000", 2024-09-11T06:50:39.154 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023631600000000001 2024-09-11T06:50:39.154 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.154 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.154 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:39.154 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733679+0000", 2024-09-11T06:50:39.154 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022771400000000001 2024-09-11T06:50:39.154 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.154 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:39.154 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.154 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.154 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.154 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:73 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:39.155 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:50:39.155 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.3794754880000006, 2024-09-11T06:50:39.155 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033454439999999999, 2024-09-11T06:50:39.155 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:39.155 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:39.155 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:39.155 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:39.155 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:39.155 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:39.155 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73 2024-09-11T06:50:39.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.155 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:39.155 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.155 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:39.156 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:50:39.156 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.156 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.156 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.156 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:39.156 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:50:39.156 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.156 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.156 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.156 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:39.156 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730694+0000", 2024-09-11T06:50:39.156 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5099999999999999e-06 2024-09-11T06:50:39.156 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.157 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.157 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:39.157 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730708+0000", 2024-09-11T06:50:39.157 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3237e-05 2024-09-11T06:50:39.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.157 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.157 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:39.157 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730738+0000", 2024-09-11T06:50:39.157 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0688999999999997e-05 2024-09-11T06:50:39.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.157 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.157 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.157 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730759+0000", 2024-09-11T06:50:39.157 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0818999999999998e-05 2024-09-11T06:50:39.158 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.158 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.158 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:39.158 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731841+0000", 2024-09-11T06:50:39.158 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010821819999999999 2024-09-11T06:50:39.158 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.158 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.158 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:39.158 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732092+0000", 2024-09-11T06:50:39.158 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025086299999999999 2024-09-11T06:50:39.158 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.158 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.158 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:39.158 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733730+0000", 2024-09-11T06:50:39.159 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0016374600000000001 2024-09-11T06:50:39.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.159 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:39.159 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733866+0000", 2024-09-11T06:50:39.159 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000136666 2024-09-11T06:50:39.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.159 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:39.159 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.734036+0000", 2024-09-11T06:50:39.159 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000170018 2024-09-11T06:50:39.159 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.159 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:39.159 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.160 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.160 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:77 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch reconnect cookie 139735284585312 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:39.160 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.735944+0000", 2024-09-11T06:50:39.160 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.3742219720000008, 2024-09-11T06:50:39.160 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002511091, 2024-09-11T06:50:39.160 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:39.160 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:39.160 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:39.160 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:39.160 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:39.160 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:39.160 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 77 2024-09-11T06:50:39.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.161 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:39.161 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.161 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:39.161 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.735944+0000", 2024-09-11T06:50:39.161 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.161 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.161 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:39.161 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.735944+0000", 2024-09-11T06:50:39.161 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.161 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.162 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:39.162 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.735946+0000", 2024-09-11T06:50:39.162 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0959999999999999e-06 2024-09-11T06:50:39.162 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.162 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.162 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:39.162 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.735963+0000", 2024-09-11T06:50:39.162 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6494000000000002e-05 2024-09-11T06:50:39.162 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.162 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.162 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:39.162 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.735996+0000", 2024-09-11T06:50:39.162 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2728999999999998e-05 2024-09-11T06:50:39.163 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.163 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.163 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.163 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.736011+0000", 2024-09-11T06:50:39.163 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5631999999999999e-05 2024-09-11T06:50:39.163 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.163 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.163 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:39.163 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.736161+0000", 2024-09-11T06:50:39.163 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015007599999999999 2024-09-11T06:50:39.163 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.163 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.163 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:39.163 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.736339+0000", 2024-09-11T06:50:39.164 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017737499999999999 2024-09-11T06:50:39.164 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.164 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.164 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:39.164 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.738455+0000", 2024-09-11T06:50:39.164 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0021166890000000002 2024-09-11T06:50:39.164 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.164 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:39.164 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.164 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.164 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.164 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch watch cookie 139740787512064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:39.164 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.861993+0000", 2024-09-11T06:50:39.165 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.2481733219999995, 2024-09-11T06:50:39.165 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027042189999999999, 2024-09-11T06:50:39.165 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:39.165 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:39.165 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.165 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:39.165 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:39.165 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:39.165 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:50:39.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.165 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:39.165 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.165 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:39.166 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.861993+0000", 2024-09-11T06:50:39.166 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.166 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.166 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.166 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:39.166 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.861993+0000", 2024-09-11T06:50:39.166 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.166 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.166 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.166 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:39.166 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.861995+0000", 2024-09-11T06:50:39.166 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2529999999999999e-06 2024-09-11T06:50:39.166 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.166 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.167 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:39.167 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862006+0000", 2024-09-11T06:50:39.167 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0509999999999999e-05 2024-09-11T06:50:39.167 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.167 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.167 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:39.167 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862037+0000", 2024-09-11T06:50:39.167 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1396000000000002e-05 2024-09-11T06:50:39.167 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.167 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.167 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.167 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862056+0000", 2024-09-11T06:50:39.167 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.859e-05 2024-09-11T06:50:39.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.168 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.168 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:39.168 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862168+0000", 2024-09-11T06:50:39.168 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000111996 2024-09-11T06:50:39.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.168 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.168 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:39.168 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862304+0000", 2024-09-11T06:50:39.168 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000136716 2024-09-11T06:50:39.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.168 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.168 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:39.168 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862544+0000", 2024-09-11T06:50:39.169 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024005000000000001 2024-09-11T06:50:39.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.169 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.169 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:39.169 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.863704+0000", 2024-09-11T06:50:39.169 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011596499999999999 2024-09-11T06:50:39.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.169 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.169 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.169 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.864508+0000", 2024-09-11T06:50:39.169 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00080411999999999997 2024-09-11T06:50:39.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.169 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.169 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:39.170 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.864597+0000", 2024-09-11T06:50:39.170 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8660999999999997e-05 2024-09-11T06:50:39.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.170 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.170 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:39.170 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.864697+0000", 2024-09-11T06:50:39.170 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000100277 2024-09-11T06:50:39.170 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.170 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:39.170 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.170 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.170 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:14 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:39.171 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.865981+0000", 2024-09-11T06:50:39.171 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.2441853629999997, 2024-09-11T06:50:39.171 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0025323239999999999, 2024-09-11T06:50:39.171 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:39.171 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:39.171 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.171 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:39.171 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:39.171 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:39.171 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 14 2024-09-11T06:50:39.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.171 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:39.171 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.171 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:39.171 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.865981+0000", 2024-09-11T06:50:39.172 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.172 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.172 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.172 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:39.172 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.865981+0000", 2024-09-11T06:50:39.172 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.172 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.172 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.172 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:39.172 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.865983+0000", 2024-09-11T06:50:39.172 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8870000000000001e-06 2024-09-11T06:50:39.172 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.172 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.173 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:39.173 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.865993+0000", 2024-09-11T06:50:39.173 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0525e-05 2024-09-11T06:50:39.173 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.173 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.173 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:39.173 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.866016+0000", 2024-09-11T06:50:39.173 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2764000000000001e-05 2024-09-11T06:50:39.173 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.173 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.173 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.173 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.866035+0000", 2024-09-11T06:50:39.173 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9025000000000002e-05 2024-09-11T06:50:39.173 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.174 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.174 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:39.174 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.866136+0000", 2024-09-11T06:50:39.174 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010080400000000001 2024-09-11T06:50:39.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.174 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.174 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:39.174 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.866277+0000", 2024-09-11T06:50:39.174 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000140839 2024-09-11T06:50:39.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.174 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.174 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:39.174 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.866561+0000", 2024-09-11T06:50:39.175 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00028413599999999999 2024-09-11T06:50:39.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.175 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.175 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:39.175 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.867767+0000", 2024-09-11T06:50:39.175 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012061789999999999 2024-09-11T06:50:39.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.175 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.175 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.175 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868321+0000", 2024-09-11T06:50:39.175 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00055425400000000001 2024-09-11T06:50:39.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.175 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.176 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:39.176 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868407+0000", 2024-09-11T06:50:39.176 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6044000000000005e-05 2024-09-11T06:50:39.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.176 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.176 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:39.176 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868513+0000", 2024-09-11T06:50:39.176 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010586699999999999 2024-09-11T06:50:39.176 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.176 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:39.176 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.176 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.177 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:18 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:39.177 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:50:39.177 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.2360582989999997, 2024-09-11T06:50:39.177 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0031646579999999999, 2024-09-11T06:50:39.177 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:39.177 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:39.177 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.177 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:39.177 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:39.177 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:39.177 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18 2024-09-11T06:50:39.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.177 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:39.177 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.178 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:39.178 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:50:39.178 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.178 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:39.178 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:50:39.178 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.178 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:39.178 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874109+0000", 2024-09-11T06:50:39.178 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.426e-06 2024-09-11T06:50:39.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.179 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.179 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:39.179 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874118+0000", 2024-09-11T06:50:39.179 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6929999999999996e-06 2024-09-11T06:50:39.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.179 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.179 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:39.179 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874134+0000", 2024-09-11T06:50:39.179 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5503e-05 2024-09-11T06:50:39.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.179 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.179 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.179 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874144+0000", 2024-09-11T06:50:39.179 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0207000000000001e-05 2024-09-11T06:50:39.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.180 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.180 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:39.180 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874545+0000", 2024-09-11T06:50:39.180 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040120800000000001 2024-09-11T06:50:39.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.180 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.180 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:39.180 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874686+0000", 2024-09-11T06:50:39.180 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014128599999999999 2024-09-11T06:50:39.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.180 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.180 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:39.181 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875023+0000", 2024-09-11T06:50:39.181 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033706600000000001 2024-09-11T06:50:39.181 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.181 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.181 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:39.181 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.876217+0000", 2024-09-11T06:50:39.181 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011938999999999999 2024-09-11T06:50:39.181 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.181 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.181 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.181 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877042+0000", 2024-09-11T06:50:39.181 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00082499899999999998 2024-09-11T06:50:39.181 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.181 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.182 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:39.182 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877128+0000", 2024-09-11T06:50:39.182 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5903999999999996e-05 2024-09-11T06:50:39.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.182 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:39.182 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877273+0000", 2024-09-11T06:50:39.182 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144466 2024-09-11T06:50:39.182 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.182 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:39.182 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.183 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:126 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:39.183 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.875489+0000", 2024-09-11T06:50:39.183 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.2346770130000007, 2024-09-11T06:50:39.183 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002819996, 2024-09-11T06:50:39.183 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:39.183 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:39.183 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:39.183 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:39.183 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:39.183 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:39.183 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 126 2024-09-11T06:50:39.183 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.183 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:39.183 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.184 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:39.184 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875489+0000", 2024-09-11T06:50:39.184 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.184 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.184 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:39.184 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875489+0000", 2024-09-11T06:50:39.184 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.184 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.184 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:39.184 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875492+0000", 2024-09-11T06:50:39.184 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2600000000000001e-06 2024-09-11T06:50:39.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.185 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.185 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:39.185 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875507+0000", 2024-09-11T06:50:39.185 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4619e-05 2024-09-11T06:50:39.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.185 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.185 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:39.185 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875537+0000", 2024-09-11T06:50:39.185 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.023e-05 2024-09-11T06:50:39.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.185 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.185 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.185 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875587+0000", 2024-09-11T06:50:39.186 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9694999999999998e-05 2024-09-11T06:50:39.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.186 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:39.186 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875834+0000", 2024-09-11T06:50:39.186 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024709999999999999 2024-09-11T06:50:39.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.186 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:39.186 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.876037+0000", 2024-09-11T06:50:39.186 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00020236000000000001 2024-09-11T06:50:39.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.187 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:39.187 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877776+0000", 2024-09-11T06:50:39.187 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00173942 2024-09-11T06:50:39.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.187 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.187 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:39.187 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877998+0000", 2024-09-11T06:50:39.187 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022171100000000001 2024-09-11T06:50:39.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.187 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.187 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:39.187 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.878309+0000", 2024-09-11T06:50:39.188 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000311601 2024-09-11T06:50:39.188 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.188 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:39.188 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.188 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.188 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.188 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:24 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:39.188 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:50:39.188 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.2215525930000002, 2024-09-11T06:50:39.188 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002980271, 2024-09-11T06:50:39.188 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:39.188 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:39.188 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.188 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:39.189 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:39.189 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:39.189 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 24 2024-09-11T06:50:39.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.189 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:39.189 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.189 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:39.189 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:50:39.189 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.189 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.189 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:39.189 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:50:39.190 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.190 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.190 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:39.190 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888616+0000", 2024-09-11T06:50:39.190 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0030000000000001e-06 2024-09-11T06:50:39.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.190 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.190 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:39.190 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888626+0000", 2024-09-11T06:50:39.190 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0502999999999999e-05 2024-09-11T06:50:39.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.190 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.191 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:39.191 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888655+0000", 2024-09-11T06:50:39.191 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8419999999999999e-05 2024-09-11T06:50:39.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.191 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.191 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888688+0000", 2024-09-11T06:50:39.191 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3212e-05 2024-09-11T06:50:39.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.191 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:39.191 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888829+0000", 2024-09-11T06:50:39.191 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014114299999999999 2024-09-11T06:50:39.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.192 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:39.192 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.889082+0000", 2024-09-11T06:50:39.192 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025279499999999999 2024-09-11T06:50:39.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.192 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:39.192 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.889577+0000", 2024-09-11T06:50:39.192 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00049536299999999997 2024-09-11T06:50:39.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.192 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:39.192 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.890811+0000", 2024-09-11T06:50:39.193 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001233858 2024-09-11T06:50:39.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.193 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.193 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.193 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891368+0000", 2024-09-11T06:50:39.193 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00055677400000000001 2024-09-11T06:50:39.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.193 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.193 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:39.193 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891456+0000", 2024-09-11T06:50:39.193 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8246999999999997e-05 2024-09-11T06:50:39.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.193 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.193 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:39.194 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891594+0000", 2024-09-11T06:50:39.194 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013795300000000001 2024-09-11T06:50:39.194 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.194 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:39.194 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.194 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.194 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:26 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 139740787512064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:39.194 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:50:39.194 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.2178499489999997, 2024-09-11T06:50:39.194 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003613733, 2024-09-11T06:50:39.194 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:39.194 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:39.194 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.195 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:39.195 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:39.195 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:39.195 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:50:39.195 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.195 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:39.195 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.195 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:39.195 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:50:39.195 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.195 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.195 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.195 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:39.195 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:50:39.196 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.196 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.196 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:39.196 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892318+0000", 2024-09-11T06:50:39.196 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.623e-06 2024-09-11T06:50:39.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.196 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.196 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:39.196 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892325+0000", 2024-09-11T06:50:39.196 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:50:39.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.196 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.196 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:39.197 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892346+0000", 2024-09-11T06:50:39.197 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1246e-05 2024-09-11T06:50:39.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.197 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.197 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892362+0000", 2024-09-11T06:50:39.197 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5409000000000001e-05 2024-09-11T06:50:39.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.197 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:39.197 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892883+0000", 2024-09-11T06:50:39.197 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00052087599999999998 2024-09-11T06:50:39.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:39.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893123+0000", 2024-09-11T06:50:39.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000240769 2024-09-11T06:50:39.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:39.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893526+0000", 2024-09-11T06:50:39.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040300200000000002 2024-09-11T06:50:39.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:39.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894820+0000", 2024-09-11T06:50:39.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012934159999999999 2024-09-11T06:50:39.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.199 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.199 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895608+0000", 2024-09-11T06:50:39.199 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00078788100000000004 2024-09-11T06:50:39.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.199 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:39.199 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895724+0000", 2024-09-11T06:50:39.199 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011595999999999999 2024-09-11T06:50:39.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.199 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:39.199 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895930+0000", 2024-09-11T06:50:39.200 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000206391 2024-09-11T06:50:39.200 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.200 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:39.200 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.200 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.200 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:133 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:39.200 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:50:39.200 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.2170519320000004, 2024-09-11T06:50:39.200 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035017360000000001, 2024-09-11T06:50:39.200 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:39.200 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:39.200 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:39.200 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:39.201 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:39.201 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:39.201 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 133 2024-09-11T06:50:39.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.201 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:39.201 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.201 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:39.201 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:50:39.201 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.201 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.201 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:39.201 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:50:39.202 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.202 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.202 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:39.202 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893116+0000", 2024-09-11T06:50:39.202 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1629999999999999e-06 2024-09-11T06:50:39.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.202 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.202 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:39.202 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893128+0000", 2024-09-11T06:50:39.202 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.11e-05 2024-09-11T06:50:39.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.202 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.202 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:39.203 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893156+0000", 2024-09-11T06:50:39.203 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8266e-05 2024-09-11T06:50:39.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.203 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.203 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893171+0000", 2024-09-11T06:50:39.203 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5426000000000001e-05 2024-09-11T06:50:39.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.203 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:39.203 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894561+0000", 2024-09-11T06:50:39.203 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013899959999999999 2024-09-11T06:50:39.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.203 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:39.204 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894781+0000", 2024-09-11T06:50:39.204 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021987100000000001 2024-09-11T06:50:39.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.204 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.204 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:39.204 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896349+0000", 2024-09-11T06:50:39.204 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015681779999999999 2024-09-11T06:50:39.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.204 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.204 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:39.204 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896458+0000", 2024-09-11T06:50:39.204 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000108187 2024-09-11T06:50:39.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.205 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:39.205 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896616+0000", 2024-09-11T06:50:39.205 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015854899999999999 2024-09-11T06:50:39.205 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.205 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:39.205 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.205 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:27 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:39.205 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:50:39.205 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.2170405520000003, 2024-09-11T06:50:39.205 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003151989, 2024-09-11T06:50:39.205 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:39.205 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:39.206 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:39.206 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:39.206 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:39.206 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:39.206 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27 2024-09-11T06:50:39.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.206 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:39.206 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.206 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:39.206 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:50:39.206 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.206 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.206 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:39.207 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:50:39.207 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.207 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:39.207 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893128+0000", 2024-09-11T06:50:39.207 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.013e-06 2024-09-11T06:50:39.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.207 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:39.207 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893136+0000", 2024-09-11T06:50:39.207 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4190000000000002e-06 2024-09-11T06:50:39.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.208 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:39.208 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893158+0000", 2024-09-11T06:50:39.208 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1506999999999999e-05 2024-09-11T06:50:39.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.208 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.208 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.208 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893170+0000", 2024-09-11T06:50:39.208 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2741999999999999e-05 2024-09-11T06:50:39.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.208 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.208 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:39.208 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894255+0000", 2024-09-11T06:50:39.208 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001084453 2024-09-11T06:50:39.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:39.209 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894461+0000", 2024-09-11T06:50:39.209 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00020586700000000001 2024-09-11T06:50:39.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:39.209 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896033+0000", 2024-09-11T06:50:39.209 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001572092 2024-09-11T06:50:39.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:39.209 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896149+0000", 2024-09-11T06:50:39.209 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116636 2024-09-11T06:50:39.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.210 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.210 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:39.210 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896278+0000", 2024-09-11T06:50:39.210 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012826 2024-09-11T06:50:39.210 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.210 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:39.210 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.210 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.210 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:176 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch reconnect cookie 139735284585312 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:39.210 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:50:39.210 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.1856737329999998, 2024-09-11T06:50:39.210 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033620960000000002, 2024-09-11T06:50:39.211 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:39.211 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:39.211 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:39.211 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:39.211 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:39.211 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:39.211 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 176 2024-09-11T06:50:39.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.211 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:39.211 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.211 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:39.211 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:50:39.211 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.212 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:39.212 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:50:39.212 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.212 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:39.212 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924494+0000", 2024-09-11T06:50:39.212 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.967e-06 2024-09-11T06:50:39.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.212 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:39.212 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924501+0000", 2024-09-11T06:50:39.212 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2659999999999998e-06 2024-09-11T06:50:39.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.213 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:39.213 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924520+0000", 2024-09-11T06:50:39.213 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9038999999999998e-05 2024-09-11T06:50:39.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.213 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.213 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924535+0000", 2024-09-11T06:50:39.213 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5359999999999999e-05 2024-09-11T06:50:39.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.213 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:39.213 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.925544+0000", 2024-09-11T06:50:39.214 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010085109999999999 2024-09-11T06:50:39.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.214 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.214 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:39.214 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.925671+0000", 2024-09-11T06:50:39.214 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012756599999999999 2024-09-11T06:50:39.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.214 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.214 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:39.214 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.927855+0000", 2024-09-11T06:50:39.214 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0021833870000000002 2024-09-11T06:50:39.214 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.214 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:39.215 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.215 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.215 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.215 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:186 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 139735284585312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:39.215 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.981668+0000", 2024-09-11T06:50:39.215 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.1284981520000006, 2024-09-11T06:50:39.215 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0026256180000000001, 2024-09-11T06:50:39.215 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:39.215 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:39.215 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.215 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:39.215 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:39.215 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:39.216 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 186 2024-09-11T06:50:39.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.216 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:39.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.216 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:39.216 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.981668+0000", 2024-09-11T06:50:39.216 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.216 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:39.216 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.981668+0000", 2024-09-11T06:50:39.216 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.217 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.217 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:39.217 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.981670+0000", 2024-09-11T06:50:39.217 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3640000000000002e-06 2024-09-11T06:50:39.217 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.217 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.217 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:39.217 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.981682+0000", 2024-09-11T06:50:39.217 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1036000000000001e-05 2024-09-11T06:50:39.217 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.217 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.217 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:39.217 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.981713+0000", 2024-09-11T06:50:39.217 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1569000000000002e-05 2024-09-11T06:50:39.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.218 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.218 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.981728+0000", 2024-09-11T06:50:39.218 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5302999999999999e-05 2024-09-11T06:50:39.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.218 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:39.218 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.981858+0000", 2024-09-11T06:50:39.218 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012991599999999999 2024-09-11T06:50:39.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.218 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:39.219 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.981992+0000", 2024-09-11T06:50:39.219 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000133996 2024-09-11T06:50:39.219 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.219 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.219 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:39.219 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.982247+0000", 2024-09-11T06:50:39.219 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025445300000000003 2024-09-11T06:50:39.219 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.219 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.219 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:39.219 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.983436+0000", 2024-09-11T06:50:39.219 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011896089999999999 2024-09-11T06:50:39.219 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.220 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.220 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.220 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.984108+0000", 2024-09-11T06:50:39.220 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00067153099999999995 2024-09-11T06:50:39.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.220 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.220 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:39.220 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.984195+0000", 2024-09-11T06:50:39.220 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6804000000000004e-05 2024-09-11T06:50:39.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.220 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.220 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:39.220 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.984294+0000", 2024-09-11T06:50:39.220 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9036999999999994e-05 2024-09-11T06:50:39.221 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.221 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:39.221 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.221 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.221 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.221 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4828.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch watch cookie 140587969811264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-11T06:50:39.221 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:50:39.221 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 2.8639707749999999, 2024-09-11T06:50:39.221 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032778859999999998, 2024-09-11T06:50:39.221 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:39.221 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:39.221 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.221 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:39.222 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4828", 2024-09-11T06:50:39.222 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2410288650", 2024-09-11T06:50:39.222 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:50:39.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.222 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:39.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.222 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:39.222 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:50:39.222 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.222 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:39.222 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:50:39.222 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:39.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.223 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.223 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:39.223 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246198+0000", 2024-09-11T06:50:39.223 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.667e-06 2024-09-11T06:50:39.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.223 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.223 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:39.223 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246209+0000", 2024-09-11T06:50:39.223 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0409e-05 2024-09-11T06:50:39.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.223 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.223 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:39.223 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246241+0000", 2024-09-11T06:50:39.223 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2592999999999998e-05 2024-09-11T06:50:39.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.224 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.224 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.224 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246260+0000", 2024-09-11T06:50:39.224 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9343000000000001e-05 2024-09-11T06:50:39.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.224 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.224 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:39.224 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246466+0000", 2024-09-11T06:50:39.224 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000205701 2024-09-11T06:50:39.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.224 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.224 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:39.224 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246713+0000", 2024-09-11T06:50:39.225 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024646999999999999 2024-09-11T06:50:39.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.225 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:39.225 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.247133+0000", 2024-09-11T06:50:39.225 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000420536 2024-09-11T06:50:39.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.225 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:39.225 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.248541+0000", 2024-09-11T06:50:39.225 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001407801 2024-09-11T06:50:39.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.225 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.225 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249241+0000", 2024-09-11T06:50:39.226 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070043500000000001 2024-09-11T06:50:39.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.226 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:39.226 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249326+0000", 2024-09-11T06:50:39.226 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5037999999999995e-05 2024-09-11T06:50:39.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:39.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:39.226 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:39.226 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249473+0000", 2024-09-11T06:50:39.226 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014689299999999999 2024-09-11T06:50:39.226 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.226 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:39.226 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.227 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:39.227 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:39.227 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:39.228 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:50:39.228 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:50:39.229 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:50:39.229 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:50:39.230 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:50:39.230 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:39.223+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:39.245 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 1, 3] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T06:50:39.245 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-11T06:50:39.266 INFO:tasks.workunit.client.0.smithi176.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2024-09-11T06:50:39.283 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo rbd device --device-type nbd map rbd/ns/testrbdnbd21607@snap 2024-09-11T06:50:39.283 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:50:39.283 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:50:39.285 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:50:39.286 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which rbd 2024-09-11T06:50:39.288 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/rbd 2024-09-11T06:50:39.288 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:50:39.288 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map rbd/ns/testrbdnbd21607@snap 2024-09-11T06:50:39.329 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:39.323+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:39.366 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:39.366 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:39.366 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:50:39.366 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:39.380 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:50:39.429 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:39.427+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:39.431 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV=/dev/nbd0 2024-09-11T06:50:39.431 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd ns 2024-09-11T06:50:39.431 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:50:39.431 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns=ns 2024-09-11T06:50:39.432 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:50:39.432 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\''ns'\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd0'\'']/id' 2024-09-11T06:50:39.501 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=23037 2024-09-11T06:50:39.501 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 23037 2024-09-11T06:50:39.501 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 23037 -C rbd-nbd 2024-09-11T06:50:39.506 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:39.506 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:39.506 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:50:39.506 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:50:39.506 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:39.507 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:50:39.507 INFO:tasks.workunit.client.0.smithi176.stdout: 23037 ? 00:00:00 rbd-nbd 2024-09-11T06:50:39.507 INFO:tasks.workunit.client.0.smithi176.stderr:+ unmap_device rbd/ns/testrbdnbd21607@snap 23037 2024-09-11T06:50:39.507 INFO:tasks.workunit.client.0.smithi176.stderr:+ local args=rbd/ns/testrbdnbd21607@snap 2024-09-11T06:50:39.508 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pid=23037 2024-09-11T06:50:39.508 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device --device-type nbd unmap rbd/ns/testrbdnbd21607@snap 2024-09-11T06:50:39.508 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:39.508 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:39.508 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:39.508 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:50:39.510 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:50:39.510 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:39.510 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap rbd/ns/testrbdnbd21607@snap 2024-09-11T06:50:39.520 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:50:39.568 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:39.567+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:39.591 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:50:39.591 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^23037\b' 2024-09-11T06:50:39.591 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^23037\b' 2024-09-11T06:50:39.631 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:39.627+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:39.640 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:39.641 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -C rbd-nbd 2024-09-11T06:50:39.641 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^ *23037\b' 2024-09-11T06:50:39.641 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^ *23037\b' 2024-09-11T06:50:39.647 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:39.647 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:39.648 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:39.648 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:50:39.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.648 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:39.648 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:50:39.648 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 93.474497646000003, 2024-09-11T06:50:39.648 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011420669, 2024-09-11T06:50:39.648 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:39.648 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:39.648 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.648 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:39.648 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:39.648 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:39.649 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 37 2024-09-11T06:50:39.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.649 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:39.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.649 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:39.649 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:50:39.649 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.649 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:39.649 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170508+0000", 2024-09-11T06:50:39.649 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.649 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:39.650 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170509+0000", 2024-09-11T06:50:39.650 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.13e-06 2024-09-11T06:50:39.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.650 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:39.650 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170521+0000", 2024-09-11T06:50:39.650 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1776999999999999e-05 2024-09-11T06:50:39.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.650 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:39.650 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170538+0000", 2024-09-11T06:50:39.650 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7413000000000001e-05 2024-09-11T06:50:39.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.651 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.651 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.170570+0000", 2024-09-11T06:50:39.651 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2313e-05 2024-09-11T06:50:39.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.651 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:39.651 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179768+0000", 2024-09-11T06:50:39.651 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0091978010000000002 2024-09-11T06:50:39.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.651 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:39.651 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.179951+0000", 2024-09-11T06:50:39.651 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018227900000000001 2024-09-11T06:50:39.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.652 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:39.652 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.180175+0000", 2024-09-11T06:50:39.652 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022432800000000001 2024-09-11T06:50:39.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.652 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:39.652 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181112+0000", 2024-09-11T06:50:39.652 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00093712500000000002 2024-09-11T06:50:39.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.652 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.652 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181789+0000", 2024-09-11T06:50:39.652 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00067705100000000002 2024-09-11T06:50:39.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.653 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:39.653 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181871+0000", 2024-09-11T06:50:39.653 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1816000000000004e-05 2024-09-11T06:50:39.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.653 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:39.653 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.181929+0000", 2024-09-11T06:50:39.653 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7636000000000002e-05 2024-09-11T06:50:39.653 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.653 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:39.653 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.654 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:39.654 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:39.654 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 93.373238685000004, 2024-09-11T06:50:39.654 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283336, 2024-09-11T06:50:39.654 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:39.654 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:39.654 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.654 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:39.654 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:39.654 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:39.654 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 87 2024-09-11T06:50:39.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.654 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:39.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.655 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:39.655 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:39.655 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.655 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:39.655 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:39.655 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.655 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:39.655 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271768+0000", 2024-09-11T06:50:39.655 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7599999999999996e-07 2024-09-11T06:50:39.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.656 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:39.656 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271860+0000", 2024-09-11T06:50:39.656 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2639999999999994e-05 2024-09-11T06:50:39.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.656 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:39.656 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271873+0000", 2024-09-11T06:50:39.656 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3139000000000001e-05 2024-09-11T06:50:39.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.656 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.657 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271882+0000", 2024-09-11T06:50:39.657 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2369999999999992e-06 2024-09-11T06:50:39.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.657 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:39.657 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277150+0000", 2024-09-11T06:50:39.657 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052684419999999999 2024-09-11T06:50:39.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.657 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:39.657 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277273+0000", 2024-09-11T06:50:39.657 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012278899999999999 2024-09-11T06:50:39.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.658 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:39.658 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277466+0000", 2024-09-11T06:50:39.658 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193578 2024-09-11T06:50:39.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.658 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.658 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.280808+0000", 2024-09-11T06:50:39.658 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003341971 2024-09-11T06:50:39.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.658 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:39.658 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284419+0000", 2024-09-11T06:50:39.658 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003610982 2024-09-11T06:50:39.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.659 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:39.659 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284531+0000", 2024-09-11T06:50:39.659 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011121299999999999 2024-09-11T06:50:39.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.659 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:39.659 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284600+0000", 2024-09-11T06:50:39.659 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9592999999999996e-05 2024-09-11T06:50:39.659 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.659 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:39.659 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.660 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:39.660 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:39.660 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 93.373108432999999, 2024-09-11T06:50:39.660 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283314, 2024-09-11T06:50:39.660 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:39.660 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:39.660 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.660 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:39.660 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:39.660 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:39.660 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 88 2024-09-11T06:50:39.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.660 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:39.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.661 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:39.661 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:39.661 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.661 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.661 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:39.661 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:39.661 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.661 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.661 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:39.661 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271898+0000", 2024-09-11T06:50:39.661 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2999999999999999e-07 2024-09-11T06:50:39.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.662 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.662 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:39.662 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271904+0000", 2024-09-11T06:50:39.662 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.767e-06 2024-09-11T06:50:39.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.662 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.662 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:39.662 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271913+0000", 2024-09-11T06:50:39.662 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7200000000000001e-06 2024-09-11T06:50:39.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.662 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.662 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.662 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271920+0000", 2024-09-11T06:50:39.663 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7100000000000001e-06 2024-09-11T06:50:39.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.663 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:39.663 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278684+0000", 2024-09-11T06:50:39.663 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0067642650000000002 2024-09-11T06:50:39.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.663 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:39.663 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278838+0000", 2024-09-11T06:50:39.663 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015371499999999999 2024-09-11T06:50:39.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.663 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:39.663 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.279040+0000", 2024-09-11T06:50:39.664 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020207499999999999 2024-09-11T06:50:39.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.664 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.664 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.664 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.281448+0000", 2024-09-11T06:50:39.664 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002407686 2024-09-11T06:50:39.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.664 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.664 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:39.664 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284615+0000", 2024-09-11T06:50:39.664 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031675499999999999 2024-09-11T06:50:39.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.664 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.664 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:39.665 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284682+0000", 2024-09-11T06:50:39.665 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7032000000000005e-05 2024-09-11T06:50:39.665 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.665 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.665 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:39.665 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284730+0000", 2024-09-11T06:50:39.665 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7790000000000002e-05 2024-09-11T06:50:39.665 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.665 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:39.665 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.665 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.665 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.665 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:39.665 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:50:39.665 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 93.204616236000007, 2024-09-11T06:50:39.666 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011236556, 2024-09-11T06:50:39.666 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:39.666 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:39.666 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.666 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:39.666 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:39.666 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:39.666 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 109 2024-09-11T06:50:39.666 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.666 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:39.666 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.666 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:39.666 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:50:39.666 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.667 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.667 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:39.667 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440389+0000", 2024-09-11T06:50:39.667 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.667 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.667 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:39.667 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440391+0000", 2024-09-11T06:50:39.667 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8360000000000001e-06 2024-09-11T06:50:39.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.667 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.667 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:39.667 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440560+0000", 2024-09-11T06:50:39.668 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000169085 2024-09-11T06:50:39.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.668 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:39.668 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440585+0000", 2024-09-11T06:50:39.668 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4530000000000001e-05 2024-09-11T06:50:39.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.668 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.668 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440606+0000", 2024-09-11T06:50:39.668 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1277e-05 2024-09-11T06:50:39.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.668 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:39.669 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446479+0000", 2024-09-11T06:50:39.669 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0058734989999999999 2024-09-11T06:50:39.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.669 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.669 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:39.669 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446649+0000", 2024-09-11T06:50:39.669 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016948199999999999 2024-09-11T06:50:39.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.669 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.669 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:39.669 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.446854+0000", 2024-09-11T06:50:39.669 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020491500000000001 2024-09-11T06:50:39.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.669 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.670 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:39.670 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451380+0000", 2024-09-11T06:50:39.670 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0045264110000000001 2024-09-11T06:50:39.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.670 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.670 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451479+0000", 2024-09-11T06:50:39.670 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.8975999999999996e-05 2024-09-11T06:50:39.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.670 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:39.670 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451562+0000", 2024-09-11T06:50:39.670 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3213000000000004e-05 2024-09-11T06:50:39.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.671 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:39.671 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451626+0000", 2024-09-11T06:50:39.671 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3331999999999996e-05 2024-09-11T06:50:39.671 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.671 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:39.671 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.671 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.671 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:39.671 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:50:39.671 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 93.204367108, 2024-09-11T06:50:39.671 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011252866, 2024-09-11T06:50:39.671 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:39.671 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:39.672 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.672 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:39.672 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:39.672 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:39.672 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 110 2024-09-11T06:50:39.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.672 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:39.672 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.672 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:39.672 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:50:39.672 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.672 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.672 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:39.672 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440638+0000", 2024-09-11T06:50:39.673 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.673 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:39.673 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440640+0000", 2024-09-11T06:50:39.673 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5430000000000001e-06 2024-09-11T06:50:39.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.673 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:39.673 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440651+0000", 2024-09-11T06:50:39.673 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0672999999999999e-05 2024-09-11T06:50:39.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.673 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:39.674 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440670+0000", 2024-09-11T06:50:39.674 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9726999999999999e-05 2024-09-11T06:50:39.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.674 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.674 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.674 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440691+0000", 2024-09-11T06:50:39.674 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0526e-05 2024-09-11T06:50:39.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.674 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.674 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:39.674 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448191+0000", 2024-09-11T06:50:39.674 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0075005990000000002 2024-09-11T06:50:39.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.674 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.675 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:39.675 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448390+0000", 2024-09-11T06:50:39.675 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000198815 2024-09-11T06:50:39.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.675 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.675 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:39.675 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.448735+0000", 2024-09-11T06:50:39.675 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034424999999999999 2024-09-11T06:50:39.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.675 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.675 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.675 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451742+0000", 2024-09-11T06:50:39.676 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030077440000000001 2024-09-11T06:50:39.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.676 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:39.676 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451761+0000", 2024-09-11T06:50:39.676 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9196999999999999e-05 2024-09-11T06:50:39.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.676 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:39.676 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451834+0000", 2024-09-11T06:50:39.676 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2071999999999997e-05 2024-09-11T06:50:39.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.677 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:39.677 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451891+0000", 2024-09-11T06:50:39.677 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7720000000000003e-05 2024-09-11T06:50:39.677 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.677 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:39.677 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.677 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:39.677 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:39.677 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 93.204259233000002, 2024-09-11T06:50:39.677 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014944941, 2024-09-11T06:50:39.677 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:39.677 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:39.678 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.678 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:39.678 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:39.678 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:39.678 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 111 2024-09-11T06:50:39.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.678 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:39.678 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.678 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:39.678 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:39.678 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.678 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.679 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:39.679 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:39.679 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.679 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:39.679 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440748+0000", 2024-09-11T06:50:39.679 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:50:39.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.679 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:39.679 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440760+0000", 2024-09-11T06:50:39.679 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1897e-05 2024-09-11T06:50:39.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.680 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:39.680 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440782+0000", 2024-09-11T06:50:39.680 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2552999999999999e-05 2024-09-11T06:50:39.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.680 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.680 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440793+0000", 2024-09-11T06:50:39.680 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0643e-05 2024-09-11T06:50:39.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.680 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:39.680 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449755+0000", 2024-09-11T06:50:39.681 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089622620000000004 2024-09-11T06:50:39.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.681 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:39.681 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449918+0000", 2024-09-11T06:50:39.681 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163086 2024-09-11T06:50:39.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.681 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:39.681 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450150+0000", 2024-09-11T06:50:39.681 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023204500000000001 2024-09-11T06:50:39.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.681 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:39.682 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454533+0000", 2024-09-11T06:50:39.682 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0043831349999999998 2024-09-11T06:50:39.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.682 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.682 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455511+0000", 2024-09-11T06:50:39.682 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0009776050000000001 2024-09-11T06:50:39.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.682 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:39.682 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455597+0000", 2024-09-11T06:50:39.682 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6446000000000005e-05 2024-09-11T06:50:39.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.683 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:39.683 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455691+0000", 2024-09-11T06:50:39.683 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3832000000000006e-05 2024-09-11T06:50:39.683 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.683 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:39.683 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.683 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.683 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:39.683 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:39.683 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 93.204189470000003, 2024-09-11T06:50:39.683 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018885411000000001, 2024-09-11T06:50:39.683 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:39.683 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:39.684 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.684 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:39.684 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:39.684 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:39.684 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 112 2024-09-11T06:50:39.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.684 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:39.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.684 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:39.684 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:39.684 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.684 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:39.685 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:39.685 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.685 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:39.685 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440817+0000", 2024-09-11T06:50:39.685 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:50:39.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.685 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:39.685 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441003+0000", 2024-09-11T06:50:39.685 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018661200000000001 2024-09-11T06:50:39.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.686 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:39.686 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441027+0000", 2024-09-11T06:50:39.686 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3920000000000001e-05 2024-09-11T06:50:39.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.686 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.686 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441062+0000", 2024-09-11T06:50:39.686 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4663000000000003e-05 2024-09-11T06:50:39.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.686 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:39.686 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451981+0000", 2024-09-11T06:50:39.686 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010918926000000001 2024-09-11T06:50:39.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.687 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:39.687 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452111+0000", 2024-09-11T06:50:39.687 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129529 2024-09-11T06:50:39.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.687 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:39.687 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452311+0000", 2024-09-11T06:50:39.687 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020013799999999999 2024-09-11T06:50:39.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.687 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.688 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455809+0000", 2024-09-11T06:50:39.688 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003498443 2024-09-11T06:50:39.688 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.688 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:39.688 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459566+0000", 2024-09-11T06:50:39.688 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003757271 2024-09-11T06:50:39.688 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.688 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:39.688 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459640+0000", 2024-09-11T06:50:39.688 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3949000000000006e-05 2024-09-11T06:50:39.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.689 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:39.689 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459701+0000", 2024-09-11T06:50:39.689 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1117000000000005e-05 2024-09-11T06:50:39.689 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.689 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:39.689 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.689 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:39.689 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:39.689 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 93.203910618999998, 2024-09-11T06:50:39.690 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018760915, 2024-09-11T06:50:39.690 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:39.690 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:39.690 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.690 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:39.690 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:39.690 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:39.690 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 113 2024-09-11T06:50:39.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.690 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:39.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.690 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:39.690 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:39.691 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.691 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:39.691 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:39.691 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.691 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:39.691 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441096+0000", 2024-09-11T06:50:39.691 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4130000000000001e-06 2024-09-11T06:50:39.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.691 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:39.691 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441110+0000", 2024-09-11T06:50:39.692 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3900000000000001e-05 2024-09-11T06:50:39.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.692 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:39.692 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441131+0000", 2024-09-11T06:50:39.692 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0636999999999999e-05 2024-09-11T06:50:39.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.692 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.692 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441147+0000", 2024-09-11T06:50:39.692 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6512999999999999e-05 2024-09-11T06:50:39.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.692 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:39.693 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453274+0000", 2024-09-11T06:50:39.693 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012126764999999999 2024-09-11T06:50:39.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.693 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:39.693 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453498+0000", 2024-09-11T06:50:39.693 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000223568 2024-09-11T06:50:39.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.693 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:39.693 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453872+0000", 2024-09-11T06:50:39.693 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037467099999999998 2024-09-11T06:50:39.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.694 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.694 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458709+0000", 2024-09-11T06:50:39.694 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0048365680000000003 2024-09-11T06:50:39.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.694 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:39.694 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459716+0000", 2024-09-11T06:50:39.694 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010075240000000001 2024-09-11T06:50:39.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.694 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:39.694 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459779+0000", 2024-09-11T06:50:39.694 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2336999999999996e-05 2024-09-11T06:50:39.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.695 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:39.695 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459856+0000", 2024-09-11T06:50:39.695 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7019000000000002e-05 2024-09-11T06:50:39.695 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.695 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:39.695 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.695 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.695 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.695 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:39.695 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:39.695 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 93.203833880000005, 2024-09-11T06:50:39.695 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018963114, 2024-09-11T06:50:39.695 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:39.695 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:39.696 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.696 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:39.696 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:39.696 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:39.696 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 114 2024-09-11T06:50:39.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.696 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:39.696 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.696 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:39.696 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:39.696 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.696 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.696 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:39.697 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:39.697 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.697 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.697 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:39.697 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:39.697 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9299999999999996e-07 2024-09-11T06:50:39.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.697 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.697 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:39.697 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441180+0000", 2024-09-11T06:50:39.697 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7200000000000006e-06 2024-09-11T06:50:39.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.697 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.698 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:39.698 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441265+0000", 2024-09-11T06:50:39.698 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4753000000000007e-05 2024-09-11T06:50:39.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.698 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.698 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.698 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441288+0000", 2024-09-11T06:50:39.698 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2860000000000001e-05 2024-09-11T06:50:39.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.698 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.698 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:39.698 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454585+0000", 2024-09-11T06:50:39.698 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013297286 2024-09-11T06:50:39.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.699 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:39.699 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454714+0000", 2024-09-11T06:50:39.699 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000128465 2024-09-11T06:50:39.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.699 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:39.699 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454909+0000", 2024-09-11T06:50:39.699 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019522499999999999 2024-09-11T06:50:39.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.699 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:39.699 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459868+0000", 2024-09-11T06:50:39.699 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0049591239999999997 2024-09-11T06:50:39.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.700 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.700 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.700 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459988+0000", 2024-09-11T06:50:39.700 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000120532 2024-09-11T06:50:39.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.700 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.700 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:39.700 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460071+0000", 2024-09-11T06:50:39.700 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2256000000000007e-05 2024-09-11T06:50:39.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.700 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.700 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:39.701 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460135+0000", 2024-09-11T06:50:39.701 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3999999999999997e-05 2024-09-11T06:50:39.701 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.701 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:39.701 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.701 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.701 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.701 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:39.701 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:39.701 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 93.203676251000005, 2024-09-11T06:50:39.701 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.019036849000000002, 2024-09-11T06:50:39.701 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:39.701 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:39.701 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.701 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:39.702 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:39.702 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:39.702 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 115 2024-09-11T06:50:39.702 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.702 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:39.702 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.702 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:39.702 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:39.702 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.702 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.702 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.702 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:39.702 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:39.702 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.702 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.703 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.703 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:39.703 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441330+0000", 2024-09-11T06:50:39.703 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1930000000000001e-06 2024-09-11T06:50:39.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.703 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.703 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:39.703 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441544+0000", 2024-09-11T06:50:39.703 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021405799999999999 2024-09-11T06:50:39.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.703 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.703 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:39.704 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441572+0000", 2024-09-11T06:50:39.704 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7317e-05 2024-09-11T06:50:39.704 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.704 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.704 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.704 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441593+0000", 2024-09-11T06:50:39.704 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1702999999999999e-05 2024-09-11T06:50:39.704 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.704 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.704 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:39.704 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455891+0000", 2024-09-11T06:50:39.704 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01429708 2024-09-11T06:50:39.704 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.705 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.705 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:39.705 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456046+0000", 2024-09-11T06:50:39.705 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000155682 2024-09-11T06:50:39.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.705 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.705 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:39.705 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456242+0000", 2024-09-11T06:50:39.705 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000195692 2024-09-11T06:50:39.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.705 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.705 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:39.705 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459876+0000", 2024-09-11T06:50:39.705 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036336480000000002 2024-09-11T06:50:39.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.706 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.706 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.706 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460245+0000", 2024-09-11T06:50:39.706 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036901700000000002 2024-09-11T06:50:39.706 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.706 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.706 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:39.706 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460316+0000", 2024-09-11T06:50:39.706 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1846000000000003e-05 2024-09-11T06:50:39.706 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.706 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.706 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:39.706 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460366+0000", 2024-09-11T06:50:39.706 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9613000000000001e-05 2024-09-11T06:50:39.706 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.707 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:39.707 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.707 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.707 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.707 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:39.707 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:39.707 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 93.203354171000001, 2024-09-11T06:50:39.707 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01904432, 2024-09-11T06:50:39.707 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:39.707 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:39.707 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.707 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:39.707 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:39.707 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:39.708 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 116 2024-09-11T06:50:39.708 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.708 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:39.708 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.708 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:39.708 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:39.708 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.708 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.708 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.708 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:39.708 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:39.708 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.708 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.708 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.708 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:39.709 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441653+0000", 2024-09-11T06:50:39.709 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:50:39.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.709 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.709 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:39.709 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441665+0000", 2024-09-11T06:50:39.709 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1653e-05 2024-09-11T06:50:39.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.709 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.709 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:39.709 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441684+0000", 2024-09-11T06:50:39.709 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9490000000000001e-05 2024-09-11T06:50:39.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.709 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.710 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.710 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441699+0000", 2024-09-11T06:50:39.710 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5044e-05 2024-09-11T06:50:39.710 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.710 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.710 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:39.710 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457297+0000", 2024-09-11T06:50:39.710 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015598241000000001 2024-09-11T06:50:39.710 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.710 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.710 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:39.710 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457577+0000", 2024-09-11T06:50:39.711 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027930799999999998 2024-09-11T06:50:39.711 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.711 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.711 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:39.711 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457957+0000", 2024-09-11T06:50:39.711 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00038088 2024-09-11T06:50:39.711 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.711 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.711 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.711 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460432+0000", 2024-09-11T06:50:39.711 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024743349999999998 2024-09-11T06:50:39.711 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.711 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.711 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:39.711 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460584+0000", 2024-09-11T06:50:39.712 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015232900000000001 2024-09-11T06:50:39.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.712 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.712 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:39.712 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460646+0000", 2024-09-11T06:50:39.712 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1883000000000004e-05 2024-09-11T06:50:39.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.712 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.712 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:39.712 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460696+0000", 2024-09-11T06:50:39.712 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9589999999999998e-05 2024-09-11T06:50:39.712 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.712 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:39.713 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.713 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.713 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.713 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:39.713 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:39.713 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 93.203282105, 2024-09-11T06:50:39.713 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.021834473, 2024-09-11T06:50:39.713 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:39.713 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:39.713 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.713 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:39.713 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:39.713 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:39.714 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 117 2024-09-11T06:50:39.714 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.714 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:39.714 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.714 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:39.714 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:39.714 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.714 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.714 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.714 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:39.714 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:39.714 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.714 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.714 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.714 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:39.715 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441725+0000", 2024-09-11T06:50:39.715 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5069999999999999e-06 2024-09-11T06:50:39.715 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.715 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.715 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:39.715 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441738+0000", 2024-09-11T06:50:39.715 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2813000000000001e-05 2024-09-11T06:50:39.715 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.715 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.715 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:39.715 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441759+0000", 2024-09-11T06:50:39.715 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1246999999999998e-05 2024-09-11T06:50:39.715 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.715 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.715 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.716 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441779+0000", 2024-09-11T06:50:39.716 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0392999999999999e-05 2024-09-11T06:50:39.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.716 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.716 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:39.716 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458804+0000", 2024-09-11T06:50:39.716 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.017024182999999998 2024-09-11T06:50:39.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.716 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.716 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:39.716 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458977+0000", 2024-09-11T06:50:39.716 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000173926 2024-09-11T06:50:39.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.716 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.717 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:39.717 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459170+0000", 2024-09-11T06:50:39.717 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000192184 2024-09-11T06:50:39.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.717 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.717 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.717 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.462300+0000", 2024-09-11T06:50:39.717 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031304969999999999 2024-09-11T06:50:39.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.717 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.717 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:39.717 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463408+0000", 2024-09-11T06:50:39.717 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011077839999999999 2024-09-11T06:50:39.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.718 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.718 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:39.718 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463497+0000", 2024-09-11T06:50:39.718 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8828999999999999e-05 2024-09-11T06:50:39.718 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.718 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.718 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:39.718 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463558+0000", 2024-09-11T06:50:39.718 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1110000000000003e-05 2024-09-11T06:50:39.718 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.718 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:39.718 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.718 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.718 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.719 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:39.719 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:50:39.719 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 92.826774416999996, 2024-09-11T06:50:39.719 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012216898, 2024-09-11T06:50:39.719 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:39.719 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:39.719 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.719 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:39.719 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:39.719 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:39.719 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 137 2024-09-11T06:50:39.719 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.719 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:39.719 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.719 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:39.720 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:50:39.720 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.720 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.720 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.720 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:39.720 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818231+0000", 2024-09-11T06:50:39.720 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.720 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.720 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.720 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:39.720 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818232+0000", 2024-09-11T06:50:39.720 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2839999999999999e-06 2024-09-11T06:50:39.720 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.721 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.721 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:39.721 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818326+0000", 2024-09-11T06:50:39.721 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3531999999999998e-05 2024-09-11T06:50:39.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.721 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.721 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:39.721 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818337+0000", 2024-09-11T06:50:39.721 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0655999999999999e-05 2024-09-11T06:50:39.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.721 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.721 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.721 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818342+0000", 2024-09-11T06:50:39.721 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4600000000000002e-06 2024-09-11T06:50:39.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.722 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.722 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:39.722 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823890+0000", 2024-09-11T06:50:39.722 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0055480700000000004 2024-09-11T06:50:39.722 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.722 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.722 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:39.722 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.823953+0000", 2024-09-11T06:50:39.722 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2848999999999994e-05 2024-09-11T06:50:39.722 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.722 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.722 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:39.722 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824074+0000", 2024-09-11T06:50:39.722 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121309 2024-09-11T06:50:39.722 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.723 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.723 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.723 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828043+0000", 2024-09-11T06:50:39.723 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0039685240000000002 2024-09-11T06:50:39.723 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.723 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.723 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:39.723 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830293+0000", 2024-09-11T06:50:39.723 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0022503179999999998 2024-09-11T06:50:39.723 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.723 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.723 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:39.723 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830366+0000", 2024-09-11T06:50:39.723 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2539999999999993e-05 2024-09-11T06:50:39.723 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.724 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.724 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:39.724 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830448+0000", 2024-09-11T06:50:39.724 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2355999999999996e-05 2024-09-11T06:50:39.724 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.724 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:39.724 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.724 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.724 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.724 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:39.724 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:39.724 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 92.826640603000001, 2024-09-11T06:50:39.724 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013136297, 2024-09-11T06:50:39.724 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:39.725 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:39.725 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.725 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:39.725 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:39.725 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:39.725 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 138 2024-09-11T06:50:39.725 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.725 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:39.725 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.725 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:39.725 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:39.725 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.725 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.726 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.726 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:39.726 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:39.726 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.726 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.726 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.726 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:39.726 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:39.726 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.89e-07 2024-09-11T06:50:39.726 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.726 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.726 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:39.726 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818369+0000", 2024-09-11T06:50:39.726 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4170000000000001e-06 2024-09-11T06:50:39.726 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.726 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.727 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:39.727 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818376+0000", 2024-09-11T06:50:39.727 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4270000000000002e-06 2024-09-11T06:50:39.727 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.727 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.727 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.727 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818383+0000", 2024-09-11T06:50:39.727 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3060000000000003e-06 2024-09-11T06:50:39.727 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.727 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.727 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:39.727 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824783+0000", 2024-09-11T06:50:39.727 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064003020000000001 2024-09-11T06:50:39.727 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.728 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.728 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:39.728 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825013+0000", 2024-09-11T06:50:39.728 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023039499999999999 2024-09-11T06:50:39.728 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.728 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.728 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:39.728 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825484+0000", 2024-09-11T06:50:39.728 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000470562 2024-09-11T06:50:39.728 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.728 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.728 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.728 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828633+0000", 2024-09-11T06:50:39.728 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031496689999999999 2024-09-11T06:50:39.729 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.729 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.729 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:39.729 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831413+0000", 2024-09-11T06:50:39.729 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027793219999999999 2024-09-11T06:50:39.729 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.729 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.729 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:39.729 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831450+0000", 2024-09-11T06:50:39.729 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7370999999999998e-05 2024-09-11T06:50:39.729 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.729 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.729 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:39.729 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831501+0000", 2024-09-11T06:50:39.730 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1036999999999999e-05 2024-09-11T06:50:39.730 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.730 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:39.730 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.730 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.730 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.730 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:39.730 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:39.730 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 92.826616122999994, 2024-09-11T06:50:39.730 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013223011, 2024-09-11T06:50:39.730 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:39.730 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:39.730 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.730 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:39.731 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:39.731 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:39.731 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 139 2024-09-11T06:50:39.731 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.731 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:39.731 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.731 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:39.731 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:39.731 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.731 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.731 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.731 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:39.731 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:39.731 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.732 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.732 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.732 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:39.732 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818390+0000", 2024-09-11T06:50:39.732 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5600000000000001e-07 2024-09-11T06:50:39.732 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.732 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.732 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:39.732 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818394+0000", 2024-09-11T06:50:39.732 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5929999999999999e-06 2024-09-11T06:50:39.732 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.732 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.732 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:39.732 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818399+0000", 2024-09-11T06:50:39.733 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7740000000000004e-06 2024-09-11T06:50:39.733 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.733 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.733 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.733 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818412+0000", 2024-09-11T06:50:39.733 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2625e-05 2024-09-11T06:50:39.733 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.733 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.733 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:39.733 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826205+0000", 2024-09-11T06:50:39.733 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0077930730000000002 2024-09-11T06:50:39.733 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.733 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.733 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:39.734 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826281+0000", 2024-09-11T06:50:39.734 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5876000000000003e-05 2024-09-11T06:50:39.734 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.734 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.734 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:39.734 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826384+0000", 2024-09-11T06:50:39.734 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103838 2024-09-11T06:50:39.734 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.734 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.734 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.734 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830642+0000", 2024-09-11T06:50:39.734 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004257083 2024-09-11T06:50:39.734 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.734 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.735 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:39.735 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831509+0000", 2024-09-11T06:50:39.735 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086730900000000003 2024-09-11T06:50:39.735 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.735 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.735 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:39.735 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831547+0000", 2024-09-11T06:50:39.735 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7793000000000002e-05 2024-09-11T06:50:39.735 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.735 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.735 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:39.735 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831612+0000", 2024-09-11T06:50:39.735 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5691000000000005e-05 2024-09-11T06:50:39.735 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.735 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:39.736 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.736 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.736 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.736 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:39.736 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:39.736 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 92.826587429, 2024-09-11T06:50:39.736 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016098024999999998, 2024-09-11T06:50:39.736 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:39.736 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:39.736 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.736 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:39.736 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:39.736 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:39.737 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 140 2024-09-11T06:50:39.737 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.737 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:39.737 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.737 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:39.737 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:39.737 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.737 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.737 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.737 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:39.737 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:39.737 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.737 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.737 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.738 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:39.738 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:39.738 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5199999999999998e-07 2024-09-11T06:50:39.738 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.738 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.738 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:39.738 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818479+0000", 2024-09-11T06:50:39.738 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0368000000000002e-05 2024-09-11T06:50:39.738 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.738 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.738 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:39.738 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818487+0000", 2024-09-11T06:50:39.738 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2339999999999994e-06 2024-09-11T06:50:39.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.739 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.739 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.739 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818491+0000", 2024-09-11T06:50:39.739 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4329999999999997e-06 2024-09-11T06:50:39.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.739 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.739 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:39.739 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828080+0000", 2024-09-11T06:50:39.739 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0095890449999999992 2024-09-11T06:50:39.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.739 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.739 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:39.739 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828130+0000", 2024-09-11T06:50:39.739 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9239000000000001e-05 2024-09-11T06:50:39.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.740 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.740 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:39.740 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828208+0000", 2024-09-11T06:50:39.740 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.852e-05 2024-09-11T06:50:39.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.740 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.740 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:39.740 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831625+0000", 2024-09-11T06:50:39.740 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034169410000000002 2024-09-11T06:50:39.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.740 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.740 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.740 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834411+0000", 2024-09-11T06:50:39.741 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002786111 2024-09-11T06:50:39.741 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.741 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.741 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:39.741 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834457+0000", 2024-09-11T06:50:39.741 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6041000000000002e-05 2024-09-11T06:50:39.741 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.741 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.741 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:39.741 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834516+0000", 2024-09-11T06:50:39.741 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8740999999999999e-05 2024-09-11T06:50:39.741 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.741 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:39.742 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.742 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.742 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.742 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:39.742 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:39.742 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 92.826506362000003, 2024-09-11T06:50:39.742 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016588772000000002, 2024-09-11T06:50:39.742 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:39.742 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:39.742 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.742 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:39.742 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:39.742 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:39.742 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 141 2024-09-11T06:50:39.743 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.743 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:39.743 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.743 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:39.743 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:39.743 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.743 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.743 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.743 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:39.743 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:39.743 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.743 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.743 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.743 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:39.743 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818500+0000", 2024-09-11T06:50:39.744 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0800000000000005e-07 2024-09-11T06:50:39.744 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.744 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.744 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:39.744 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818505+0000", 2024-09-11T06:50:39.744 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2429999999999996e-06 2024-09-11T06:50:39.744 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.744 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.744 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:39.744 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818510+0000", 2024-09-11T06:50:39.744 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9339999999999997e-06 2024-09-11T06:50:39.744 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.744 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.744 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.745 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818514+0000", 2024-09-11T06:50:39.745 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7270000000000001e-06 2024-09-11T06:50:39.745 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.745 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.745 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:39.745 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828711+0000", 2024-09-11T06:50:39.745 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010197031 2024-09-11T06:50:39.745 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.745 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.745 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:39.745 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828853+0000", 2024-09-11T06:50:39.745 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014201200000000001 2024-09-11T06:50:39.745 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.745 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.746 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:39.746 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.829115+0000", 2024-09-11T06:50:39.746 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026257500000000003 2024-09-11T06:50:39.746 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.746 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.746 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:39.746 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831629+0000", 2024-09-11T06:50:39.746 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025138169999999998 2024-09-11T06:50:39.746 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.746 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.746 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.746 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834991+0000", 2024-09-11T06:50:39.746 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033624459999999998 2024-09-11T06:50:39.746 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.746 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.747 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:39.747 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835032+0000", 2024-09-11T06:50:39.747 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1001000000000003e-05 2024-09-11T06:50:39.747 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.747 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.747 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:39.747 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835088+0000", 2024-09-11T06:50:39.747 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5478e-05 2024-09-11T06:50:39.747 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.747 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:39.747 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.747 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.747 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.747 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T06:50:39.748 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:50:39.748 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 92.202685357000007, 2024-09-11T06:50:39.748 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011561900999999999, 2024-09-11T06:50:39.748 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:39.748 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:39.748 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.748 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:39.748 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:39.748 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:39.748 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 159 2024-09-11T06:50:39.748 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.748 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:39.748 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.748 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:39.749 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:50:39.749 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.749 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.749 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.749 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:39.749 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442320+0000", 2024-09-11T06:50:39.749 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.749 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.749 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.749 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:39.749 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442321+0000", 2024-09-11T06:50:39.749 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8400000000000003e-07 2024-09-11T06:50:39.749 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.749 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.750 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:39.750 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442328+0000", 2024-09-11T06:50:39.750 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5130000000000002e-06 2024-09-11T06:50:39.750 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.750 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.750 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:39.750 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442341+0000", 2024-09-11T06:50:39.750 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.363e-05 2024-09-11T06:50:39.750 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.750 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.750 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.750 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.442349+0000", 2024-09-11T06:50:39.750 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4760000000000001e-06 2024-09-11T06:50:39.750 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.751 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.751 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:39.751 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451257+0000", 2024-09-11T06:50:39.751 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089088569999999992 2024-09-11T06:50:39.751 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.751 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.751 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:39.751 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451389+0000", 2024-09-11T06:50:39.751 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013110899999999999 2024-09-11T06:50:39.751 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.751 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.751 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:39.751 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.451608+0000", 2024-09-11T06:50:39.751 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000219588 2024-09-11T06:50:39.751 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.752 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.752 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:39.752 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453517+0000", 2024-09-11T06:50:39.752 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0019088779999999999 2024-09-11T06:50:39.752 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.752 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.752 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.752 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453727+0000", 2024-09-11T06:50:39.752 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020955100000000001 2024-09-11T06:50:39.752 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.752 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.752 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:39.752 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453815+0000", 2024-09-11T06:50:39.752 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8858999999999997e-05 2024-09-11T06:50:39.752 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.753 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.753 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:39.753 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:07.453882+0000", 2024-09-11T06:50:39.753 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.6556000000000004e-05 2024-09-11T06:50:39.753 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.753 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:39.753 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.753 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.753 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.753 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5235.0:1 2.5 2:b2cc3336:::rbd_id.testrbdnbd21607:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-11T06:50:39.753 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:50:39.753 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 5.0857043400000004, 2024-09-11T06:50:39.753 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.68353462499999995, 2024-09-11T06:50:39.753 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:39.754 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:39.754 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:39.754 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:39.754 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5235", 2024-09-11T06:50:39.754 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2008977164", 2024-09-11T06:50:39.754 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:50:39.754 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.754 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:39.754 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.754 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:39.754 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:50:39.754 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.754 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.755 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.755 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:39.755 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:50:39.755 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.755 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.755 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.755 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:39.755 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559303+0000", 2024-09-11T06:50:39.755 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1830000000000001e-06 2024-09-11T06:50:39.755 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.755 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.755 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:39.755 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559314+0000", 2024-09-11T06:50:39.755 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0396e-05 2024-09-11T06:50:39.756 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.756 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.756 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:39.756 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559335+0000", 2024-09-11T06:50:39.756 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1183e-05 2024-09-11T06:50:39.756 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.756 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.756 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.756 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559354+0000", 2024-09-11T06:50:39.756 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9225999999999999e-05 2024-09-11T06:50:39.756 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.756 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.756 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:39.756 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559522+0000", 2024-09-11T06:50:39.757 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000168021 2024-09-11T06:50:39.757 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.757 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.757 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:50:39.757 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559543+0000", 2024-09-11T06:50:39.757 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0616999999999999e-05 2024-09-11T06:50:39.757 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.757 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.757 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:39.757 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.238584+0000", 2024-09-11T06:50:39.757 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.67904107499999999 2024-09-11T06:50:39.757 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.757 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.758 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for missing object", 2024-09-11T06:50:39.758 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.238933+0000", 2024-09-11T06:50:39.758 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034879600000000001 2024-09-11T06:50:39.758 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.758 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.758 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:39.758 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.241968+0000", 2024-09-11T06:50:39.758 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030350830000000001 2024-09-11T06:50:39.758 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.758 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.758 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:39.758 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.242281+0000", 2024-09-11T06:50:39.758 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00031299499999999999 2024-09-11T06:50:39.758 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.759 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.759 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:39.759 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.242836+0000", 2024-09-11T06:50:39.759 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00055504999999999997 2024-09-11T06:50:39.759 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.759 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:39.759 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.759 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.759 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.759 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5235.0:18 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 0~1024 [fadvise_sequential]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T06:50:39.759 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:50:39.759 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.3832119010000001, 2024-09-11T06:50:39.759 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.048993760999999997, 2024-09-11T06:50:39.760 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:39.760 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:39.760 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:39.760 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:39.760 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5235", 2024-09-11T06:50:39.760 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2008977164", 2024-09-11T06:50:39.760 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 18 2024-09-11T06:50:39.760 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.760 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:39.760 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.760 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:39.760 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:50:39.760 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.760 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.761 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.761 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:39.761 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:50:39.761 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:39.761 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.761 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.761 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:39.761 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261797+0000", 2024-09-11T06:50:39.761 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2059999999999999e-06 2024-09-11T06:50:39.761 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.761 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.761 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:39.761 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261812+0000", 2024-09-11T06:50:39.761 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5159e-05 2024-09-11T06:50:39.762 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.762 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.762 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:39.762 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261852+0000", 2024-09-11T06:50:39.762 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.9906e-05 2024-09-11T06:50:39.762 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.762 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.762 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:39.762 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261865+0000", 2024-09-11T06:50:39.762 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.312e-05 2024-09-11T06:50:39.762 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.762 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.762 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:39.762 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.263166+0000", 2024-09-11T06:50:39.763 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001301134 2024-09-11T06:50:39.763 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.763 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.763 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for missing object", 2024-09-11T06:50:39.763 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.263529+0000", 2024-09-11T06:50:39.763 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036279900000000002 2024-09-11T06:50:39.763 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.763 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.763 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:39.763 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.300567+0000", 2024-09-11T06:50:39.763 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.037038326000000003 2024-09-11T06:50:39.763 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.763 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.763 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:39.763 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.300800+0000", 2024-09-11T06:50:39.764 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023323300000000001 2024-09-11T06:50:39.764 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.764 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.764 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:39.764 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.306709+0000", 2024-09-11T06:50:39.764 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0059087690000000003 2024-09-11T06:50:39.764 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.764 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.764 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:39.764 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.306850+0000", 2024-09-11T06:50:39.764 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014106200000000001 2024-09-11T06:50:39.764 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.764 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.764 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:39.765 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.307121+0000", 2024-09-11T06:50:39.765 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027083599999999999 2024-09-11T06:50:39.765 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.765 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.765 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:39.765 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.308809+0000", 2024-09-11T06:50:39.765 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0016880759999999999 2024-09-11T06:50:39.765 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.765 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.765 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:39.765 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310382+0000", 2024-09-11T06:50:39.765 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0015732840000000001 2024-09-11T06:50:39.765 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.765 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.766 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:39.766 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310501+0000", 2024-09-11T06:50:39.766 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000118276 2024-09-11T06:50:39.766 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:39.766 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:39.766 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:39.766 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310787+0000", 2024-09-11T06:50:39.766 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00028657500000000001 2024-09-11T06:50:39.766 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.766 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:39.766 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.766 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:39.766 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:39.766 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:39.767 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:39.768 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 0.5 2024-09-11T06:50:39.768 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:50:39.769 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:39.763+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:39.869 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:39.863+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:39.937 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:39.937 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:39.937 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:50:39.937 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:39.951 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:50:39.955 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T06:50:40.001 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:39.999+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:40.070 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:40.067+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:40.076 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:40.076 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:40.076 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:50:40.076 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:50:40.076 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:40.090 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:50:40.148 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV= 2024-09-11T06:50:40.149 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd snap ls rbd/ns/testrbdnbd21607 2024-09-11T06:50:40.150 INFO:tasks.workunit.client.0.smithi176.stderr:++ awk '$2 == "snap" {print $1}' 2024-09-11T06:50:40.171 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:40.167+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:40.209 INFO:tasks.workunit.client.0.smithi176.stderr:+ SNAPID=4 2024-09-11T06:50:40.210 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo rbd device --device-type nbd map --snap-id 4 rbd/ns/testrbdnbd21607 2024-09-11T06:50:40.210 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:50:40.210 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:50:40.211 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:50:40.211 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which rbd 2024-09-11T06:50:40.213 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/rbd 2024-09-11T06:50:40.213 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:50:40.213 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map --snap-id 4 rbd/ns/testrbdnbd21607 2024-09-11T06:50:40.217 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:40.218 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:40.218 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:40.218 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:50:40.218 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.218 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4558.0:1 3.3 3:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-11T06:50:40.218 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:50:40.218 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 23.125337346999999, 2024-09-11T06:50:40.218 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0085520600000000002, 2024-09-11T06:50:40.218 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.218 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.218 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:40.218 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:40.218 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4558", 2024-09-11T06:50:40.219 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1478388898", 2024-09-11T06:50:40.219 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:50:40.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.219 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.219 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.219 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.219 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:50:40.219 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.219 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.219 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.219 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:50:40.219 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.220 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.220 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.220 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.220 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090533+0000", 2024-09-11T06:50:40.220 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2629999999999999e-06 2024-09-11T06:50:40.220 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.220 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.220 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.220 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090546+0000", 2024-09-11T06:50:40.220 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2918999999999999e-05 2024-09-11T06:50:40.220 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.220 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.220 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.220 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090572+0000", 2024-09-11T06:50:40.221 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6412000000000001e-05 2024-09-11T06:50:40.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.221 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.221 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.221 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090594+0000", 2024-09-11T06:50:40.221 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2476e-05 2024-09-11T06:50:40.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.221 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.221 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.221 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090938+0000", 2024-09-11T06:50:40.221 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034325099999999998 2024-09-11T06:50:40.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.222 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.222 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:40.222 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.091149+0000", 2024-09-11T06:50:40.222 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021151300000000001 2024-09-11T06:50:40.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.222 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.222 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:40.222 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.091377+0000", 2024-09-11T06:50:40.222 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022785399999999999 2024-09-11T06:50:40.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.222 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.222 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:40.222 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.092912+0000", 2024-09-11T06:50:40.222 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0015346089999999999 2024-09-11T06:50:40.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.223 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:40.223 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098919+0000", 2024-09-11T06:50:40.223 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0060076799999999996 2024-09-11T06:50:40.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.223 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:40.223 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.099009+0000", 2024-09-11T06:50:40.223 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0056000000000006e-05 2024-09-11T06:50:40.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.223 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.223 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.099081+0000", 2024-09-11T06:50:40.223 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2026999999999993e-05 2024-09-11T06:50:40.224 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.224 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.224 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.224 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.224 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:50:40.224 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:50:40.224 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.083848267, 2024-09-11T06:50:40.224 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0071924390000000001, 2024-09-11T06:50:40.224 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.224 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.224 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:40.224 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.224 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.225 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.225 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:50:40.225 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.225 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.225 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:50:40.225 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.225 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.225 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132020+0000", 2024-09-11T06:50:40.225 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.846e-06 2024-09-11T06:50:40.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.226 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.226 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.226 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135688+0000", 2024-09-11T06:50:40.226 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036677350000000001 2024-09-11T06:50:40.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.226 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.226 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.226 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135747+0000", 2024-09-11T06:50:40.226 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.9314999999999997e-05 2024-09-11T06:50:40.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.226 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.226 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.226 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135797+0000", 2024-09-11T06:50:40.226 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9984000000000001e-05 2024-09-11T06:50:40.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.227 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.227 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.227 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136143+0000", 2024-09-11T06:50:40.227 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034594299999999998 2024-09-11T06:50:40.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.227 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.227 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:40.227 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136179+0000", 2024-09-11T06:50:40.227 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5636000000000002e-05 2024-09-11T06:50:40.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.227 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.227 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.227 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.139211+0000", 2024-09-11T06:50:40.227 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030319800000000001 2024-09-11T06:50:40.228 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.228 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.228 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.228 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:50:40.228 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:50:40.228 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.081785013000001, 2024-09-11T06:50:40.228 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064954139999999997, 2024-09-11T06:50:40.228 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.228 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.228 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:40.228 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.228 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.229 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:50:40.229 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.229 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.229 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.229 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:50:40.229 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.229 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.229 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.229 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134084+0000", 2024-09-11T06:50:40.229 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.894e-06 2024-09-11T06:50:40.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.229 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.229 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.230 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136513+0000", 2024-09-11T06:50:40.230 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024290969999999999 2024-09-11T06:50:40.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.230 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.230 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.230 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136554+0000", 2024-09-11T06:50:40.230 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.0932e-05 2024-09-11T06:50:40.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.230 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.230 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.230 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136584+0000", 2024-09-11T06:50:40.230 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0056000000000002e-05 2024-09-11T06:50:40.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.230 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.231 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.231 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136805+0000", 2024-09-11T06:50:40.232 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022086100000000001 2024-09-11T06:50:40.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.232 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.232 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:40.232 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136864+0000", 2024-09-11T06:50:40.232 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.9907000000000001e-05 2024-09-11T06:50:40.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.232 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.232 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.232 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.140577+0000", 2024-09-11T06:50:40.232 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0037126669999999998 2024-09-11T06:50:40.232 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.232 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.232 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.233 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.233 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:50:40.233 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:50:40.233 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.073494213, 2024-09-11T06:50:40.233 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035763180000000002, 2024-09-11T06:50:40.233 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.233 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.233 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:40.233 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.233 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.233 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:50:40.233 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.234 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.234 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.234 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:50:40.234 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.234 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.234 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.234 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142374+0000", 2024-09-11T06:50:40.234 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5460000000000001e-06 2024-09-11T06:50:40.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.234 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.234 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.234 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143902+0000", 2024-09-11T06:50:40.235 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00152808 2024-09-11T06:50:40.235 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.235 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.235 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.235 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143946+0000", 2024-09-11T06:50:40.235 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3874e-05 2024-09-11T06:50:40.235 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.235 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.235 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.235 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143978+0000", 2024-09-11T06:50:40.235 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2425999999999998e-05 2024-09-11T06:50:40.235 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.235 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.236 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.236 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.144177+0000", 2024-09-11T06:50:40.236 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019837500000000001 2024-09-11T06:50:40.236 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.236 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.236 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:40.236 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.144198+0000", 2024-09-11T06:50:40.236 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1639000000000001e-05 2024-09-11T06:50:40.236 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.236 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.236 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.236 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.145949+0000", 2024-09-11T06:50:40.236 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0017503779999999999 2024-09-11T06:50:40.236 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.237 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.237 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.237 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.237 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.237 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:50:40.237 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:50:40.238 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.069178511, 2024-09-11T06:50:40.238 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0057693220000000003, 2024-09-11T06:50:40.238 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.238 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.238 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:40.238 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.238 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.239 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.239 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:50:40.239 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.239 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.239 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.239 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.239 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:50:40.240 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.240 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.240 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.240 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.240 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146690+0000", 2024-09-11T06:50:40.240 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.607e-06 2024-09-11T06:50:40.240 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.240 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.241 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.241 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148693+0000", 2024-09-11T06:50:40.241 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0020029739999999998 2024-09-11T06:50:40.241 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.241 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.241 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.241 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148735+0000", 2024-09-11T06:50:40.242 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1958e-05 2024-09-11T06:50:40.242 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.242 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.242 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.242 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148764+0000", 2024-09-11T06:50:40.242 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8999000000000001e-05 2024-09-11T06:50:40.242 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.242 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.243 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.243 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149069+0000", 2024-09-11T06:50:40.243 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000305032 2024-09-11T06:50:40.243 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.243 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.243 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:40.243 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149204+0000", 2024-09-11T06:50:40.243 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013495 2024-09-11T06:50:40.244 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.244 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.244 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.244 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.152458+0000", 2024-09-11T06:50:40.244 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0032538020000000001 2024-09-11T06:50:40.244 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.244 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.245 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.245 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.245 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.245 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:50:40.245 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:50:40.245 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.068279969999999, 2024-09-11T06:50:40.245 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0042425329999999997, 2024-09-11T06:50:40.245 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.246 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.246 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:40.246 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.246 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.246 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.246 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:50:40.246 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.246 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.247 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.247 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.247 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:50:40.247 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.247 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.247 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.248 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.248 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147588+0000", 2024-09-11T06:50:40.248 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.277e-06 2024-09-11T06:50:40.248 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.248 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.248 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.248 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149051+0000", 2024-09-11T06:50:40.248 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001462669 2024-09-11T06:50:40.248 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.248 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.248 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.249 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149090+0000", 2024-09-11T06:50:40.249 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.8967999999999997e-05 2024-09-11T06:50:40.249 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.249 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.249 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.249 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149120+0000", 2024-09-11T06:50:40.249 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0766000000000003e-05 2024-09-11T06:50:40.249 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.249 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.249 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.249 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149395+0000", 2024-09-11T06:50:40.249 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027484900000000002 2024-09-11T06:50:40.250 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.250 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.250 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:40.250 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149439+0000", 2024-09-11T06:50:40.250 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3940999999999998e-05 2024-09-11T06:50:40.250 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.250 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.250 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.250 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.151829+0000", 2024-09-11T06:50:40.250 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002390063 2024-09-11T06:50:40.250 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.251 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.251 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.251 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.251 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.251 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:50:40.251 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:50:40.251 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.062139339000002, 2024-09-11T06:50:40.251 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0046568219999999997, 2024-09-11T06:50:40.251 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.251 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.251 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:40.251 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.251 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.251 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.251 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:50:40.252 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.252 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.252 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.252 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.252 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:50:40.252 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.252 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.252 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.252 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.252 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153729+0000", 2024-09-11T06:50:40.252 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5629999999999999e-06 2024-09-11T06:50:40.252 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.252 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.252 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.253 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155179+0000", 2024-09-11T06:50:40.253 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001450469 2024-09-11T06:50:40.253 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.253 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.253 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.253 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155221+0000", 2024-09-11T06:50:40.253 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1448e-05 2024-09-11T06:50:40.253 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.253 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.253 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.253 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155249+0000", 2024-09-11T06:50:40.253 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8433e-05 2024-09-11T06:50:40.253 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.253 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.254 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.254 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155420+0000", 2024-09-11T06:50:40.254 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017070599999999999 2024-09-11T06:50:40.254 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.254 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.254 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:40.254 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155445+0000", 2024-09-11T06:50:40.254 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4813000000000001e-05 2024-09-11T06:50:40.254 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.254 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.254 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.254 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.158384+0000", 2024-09-11T06:50:40.254 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0029393900000000001 2024-09-11T06:50:40.254 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.254 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.255 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.255 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.255 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.255 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:50:40.255 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:50:40.255 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.060112589999999, 2024-09-11T06:50:40.255 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0046826020000000001, 2024-09-11T06:50:40.255 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.255 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.255 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:40.255 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.255 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.255 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.255 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:50:40.256 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.256 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.256 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.256 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.256 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:50:40.256 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.256 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.256 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.256 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.256 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155756+0000", 2024-09-11T06:50:40.256 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.384e-06 2024-09-11T06:50:40.256 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.256 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.256 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.257 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157205+0000", 2024-09-11T06:50:40.257 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014496260000000001 2024-09-11T06:50:40.257 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.257 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.257 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.257 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157252+0000", 2024-09-11T06:50:40.257 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6368000000000001e-05 2024-09-11T06:50:40.257 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.257 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.257 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.257 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157286+0000", 2024-09-11T06:50:40.257 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4944999999999998e-05 2024-09-11T06:50:40.257 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.257 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.257 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.258 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157540+0000", 2024-09-11T06:50:40.258 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00025307999999999998 2024-09-11T06:50:40.258 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.258 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.258 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:40.258 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157576+0000", 2024-09-11T06:50:40.258 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.6878999999999997e-05 2024-09-11T06:50:40.258 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.258 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.258 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.258 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.160437+0000", 2024-09-11T06:50:40.258 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00286032 2024-09-11T06:50:40.258 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.259 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.259 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.259 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.259 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.259 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:50:40.259 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.280679+0000", 2024-09-11T06:50:40.259 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.935187601, 2024-09-11T06:50:40.259 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0040934309999999998, 2024-09-11T06:50:40.259 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.259 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.259 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:40.259 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.259 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.259 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.260 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280679+0000", 2024-09-11T06:50:40.260 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.260 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.260 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.260 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.260 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280679+0000", 2024-09-11T06:50:40.260 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.260 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.260 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.260 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.260 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280681+0000", 2024-09-11T06:50:40.260 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2529999999999999e-06 2024-09-11T06:50:40.260 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.261 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.261 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.261 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280713+0000", 2024-09-11T06:50:40.261 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.1936000000000001e-05 2024-09-11T06:50:40.261 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.261 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.261 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.261 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280758+0000", 2024-09-11T06:50:40.261 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5124999999999999e-05 2024-09-11T06:50:40.261 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.261 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.261 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.261 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280779+0000", 2024-09-11T06:50:40.261 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0936000000000001e-05 2024-09-11T06:50:40.262 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.262 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.262 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.262 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280932+0000", 2024-09-11T06:50:40.262 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015220899999999999 2024-09-11T06:50:40.262 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.262 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.262 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.262 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.284773+0000", 2024-09-11T06:50:40.262 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0038409719999999998 2024-09-11T06:50:40.262 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.262 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.262 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.262 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.262 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.263 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:50:40.263 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.292943+0000", 2024-09-11T06:50:40.263 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.9229234069999999, 2024-09-11T06:50:40.263 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0028686010000000001, 2024-09-11T06:50:40.263 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.263 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.263 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:40.263 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.263 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.263 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.263 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.292943+0000", 2024-09-11T06:50:40.263 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.263 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.263 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.263 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.264 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.292943+0000", 2024-09-11T06:50:40.264 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.264 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.264 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.264 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.264 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.292945+0000", 2024-09-11T06:50:40.264 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7799999999999999e-06 2024-09-11T06:50:40.264 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.264 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.264 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.264 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.292968+0000", 2024-09-11T06:50:40.264 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2586e-05 2024-09-11T06:50:40.264 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.265 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.265 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.265 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.292998+0000", 2024-09-11T06:50:40.265 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9893e-05 2024-09-11T06:50:40.265 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.265 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.265 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.265 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293012+0000", 2024-09-11T06:50:40.265 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4504999999999999e-05 2024-09-11T06:50:40.265 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.265 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.265 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.265 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293155+0000", 2024-09-11T06:50:40.266 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000142993 2024-09-11T06:50:40.266 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.266 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.266 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.266 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.295812+0000", 2024-09-11T06:50:40.266 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0026568439999999998 2024-09-11T06:50:40.266 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.266 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.266 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.266 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.266 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.266 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:50:40.266 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.293438+0000", 2024-09-11T06:50:40.267 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.9224288649999997, 2024-09-11T06:50:40.267 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0061479350000000002, 2024-09-11T06:50:40.267 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.267 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.267 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:40.267 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.267 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.267 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.267 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293438+0000", 2024-09-11T06:50:40.267 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.267 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.267 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.267 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.267 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293438+0000", 2024-09-11T06:50:40.267 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.268 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.268 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.268 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.268 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293439+0000", 2024-09-11T06:50:40.268 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:50:40.268 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.268 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.268 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.268 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293459+0000", 2024-09-11T06:50:40.268 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9922e-05 2024-09-11T06:50:40.268 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.268 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.268 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.268 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293482+0000", 2024-09-11T06:50:40.269 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2969e-05 2024-09-11T06:50:40.269 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.269 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.269 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.269 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293494+0000", 2024-09-11T06:50:40.269 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.131e-05 2024-09-11T06:50:40.269 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.269 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.269 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.269 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.295901+0000", 2024-09-11T06:50:40.269 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024077949999999999 2024-09-11T06:50:40.269 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.269 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.270 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.270 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.299586+0000", 2024-09-11T06:50:40.270 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036845020000000001 2024-09-11T06:50:40.270 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.270 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.270 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.270 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.270 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.270 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:50:40.270 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.309418+0000", 2024-09-11T06:50:40.270 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.9064488339999999, 2024-09-11T06:50:40.270 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031185879999999998, 2024-09-11T06:50:40.270 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.270 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.271 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:40.271 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.271 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.271 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.271 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.309418+0000", 2024-09-11T06:50:40.271 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.271 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.271 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.271 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.271 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.309418+0000", 2024-09-11T06:50:40.271 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.271 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.271 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.271 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.272 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.309419+0000", 2024-09-11T06:50:40.272 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4699999999999999e-06 2024-09-11T06:50:40.272 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.272 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.272 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.272 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.309441+0000", 2024-09-11T06:50:40.272 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1668999999999999e-05 2024-09-11T06:50:40.272 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.272 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.272 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.272 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.309463+0000", 2024-09-11T06:50:40.272 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2106000000000001e-05 2024-09-11T06:50:40.272 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.272 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.273 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.273 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.309477+0000", 2024-09-11T06:50:40.273 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3482999999999999e-05 2024-09-11T06:50:40.273 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.273 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.273 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.273 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310475+0000", 2024-09-11T06:50:40.273 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00099831099999999999 2024-09-11T06:50:40.273 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.273 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.273 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.273 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.312536+0000", 2024-09-11T06:50:40.273 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0020615490000000002 2024-09-11T06:50:40.273 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.274 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.274 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.274 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.274 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.274 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:50:40.274 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.315262+0000", 2024-09-11T06:50:40.274 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.9006050270000001, 2024-09-11T06:50:40.274 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0028909550000000002, 2024-09-11T06:50:40.274 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.274 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.274 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:40.274 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.274 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.274 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.275 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.315262+0000", 2024-09-11T06:50:40.275 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.275 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.275 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.275 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.275 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.315262+0000", 2024-09-11T06:50:40.275 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.275 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.275 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.275 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.275 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.315264+0000", 2024-09-11T06:50:40.275 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1629999999999999e-06 2024-09-11T06:50:40.275 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.275 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.275 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.276 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.315291+0000", 2024-09-11T06:50:40.276 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7489000000000001e-05 2024-09-11T06:50:40.276 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.276 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.276 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.276 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.315329+0000", 2024-09-11T06:50:40.276 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7175999999999998e-05 2024-09-11T06:50:40.276 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.276 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.276 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.276 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.315348+0000", 2024-09-11T06:50:40.276 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9596e-05 2024-09-11T06:50:40.276 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.276 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.277 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.277 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.315478+0000", 2024-09-11T06:50:40.277 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129975 2024-09-11T06:50:40.277 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.277 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.277 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.277 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.318153+0000", 2024-09-11T06:50:40.277 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0026745559999999998 2024-09-11T06:50:40.277 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.277 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.277 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.277 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.277 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.277 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:50:40.278 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.324823+0000", 2024-09-11T06:50:40.278 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.8910439659999998, 2024-09-11T06:50:40.278 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033143130000000002, 2024-09-11T06:50:40.278 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.278 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.278 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:40.278 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.278 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.278 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.278 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.324823+0000", 2024-09-11T06:50:40.278 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.278 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.278 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.279 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.279 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.324823+0000", 2024-09-11T06:50:40.279 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.279 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.279 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.279 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.279 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.324825+0000", 2024-09-11T06:50:40.279 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.88e-06 2024-09-11T06:50:40.279 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.279 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.279 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.279 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.324851+0000", 2024-09-11T06:50:40.279 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6126e-05 2024-09-11T06:50:40.279 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.280 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.280 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.280 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.324882+0000", 2024-09-11T06:50:40.280 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0778999999999998e-05 2024-09-11T06:50:40.280 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.280 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.280 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.280 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.324902+0000", 2024-09-11T06:50:40.280 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0046000000000001e-05 2024-09-11T06:50:40.280 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.280 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.280 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.280 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.325015+0000", 2024-09-11T06:50:40.280 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000113739 2024-09-11T06:50:40.281 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.281 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.281 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.281 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.328137+0000", 2024-09-11T06:50:40.281 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031217430000000002 2024-09-11T06:50:40.281 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.281 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.281 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.281 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.281 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.281 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:50:40.281 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.327802+0000", 2024-09-11T06:50:40.281 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.8880649510000005, 2024-09-11T06:50:40.281 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027834639999999998, 2024-09-11T06:50:40.282 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.282 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.282 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:40.282 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.282 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.282 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.282 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.327802+0000", 2024-09-11T06:50:40.282 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.282 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.282 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.282 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.282 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.327802+0000", 2024-09-11T06:50:40.282 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.282 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.282 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.282 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.283 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.327804+0000", 2024-09-11T06:50:40.283 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7370000000000001e-06 2024-09-11T06:50:40.283 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.283 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.283 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.283 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.327828+0000", 2024-09-11T06:50:40.283 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4831999999999999e-05 2024-09-11T06:50:40.283 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.283 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.283 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.283 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.327854+0000", 2024-09-11T06:50:40.283 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5525999999999999e-05 2024-09-11T06:50:40.283 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.284 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.284 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.284 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.327870+0000", 2024-09-11T06:50:40.284 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6330000000000001e-05 2024-09-11T06:50:40.284 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.284 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.284 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.284 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.328230+0000", 2024-09-11T06:50:40.284 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036026800000000001 2024-09-11T06:50:40.284 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.284 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.284 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.284 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.330585+0000", 2024-09-11T06:50:40.284 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0023547709999999999 2024-09-11T06:50:40.285 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.285 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.285 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.285 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.285 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.285 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.7 e70/69 cost 1048576)", 2024-09-11T06:50:40.285 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.343042+0000", 2024-09-11T06:50:40.285 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.872824348, 2024-09-11T06:50:40.285 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031815010000000002, 2024-09-11T06:50:40.285 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.285 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.285 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:40.285 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.285 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.286 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.286 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.343042+0000", 2024-09-11T06:50:40.286 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.286 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.286 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.286 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.286 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.343042+0000", 2024-09-11T06:50:40.286 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.286 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.286 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.286 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.286 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.343044+0000", 2024-09-11T06:50:40.286 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.716e-06 2024-09-11T06:50:40.287 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.287 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.287 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.287 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.343071+0000", 2024-09-11T06:50:40.287 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.654e-05 2024-09-11T06:50:40.287 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.287 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.287 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.287 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.343098+0000", 2024-09-11T06:50:40.287 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7529e-05 2024-09-11T06:50:40.287 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.287 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.287 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.287 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.343115+0000", 2024-09-11T06:50:40.288 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6629e-05 2024-09-11T06:50:40.288 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.288 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.288 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.288 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.343229+0000", 2024-09-11T06:50:40.288 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011443900000000001 2024-09-11T06:50:40.288 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.288 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.288 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.288 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.346224+0000", 2024-09-11T06:50:40.288 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0029946479999999999 2024-09-11T06:50:40.288 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.288 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.289 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.289 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.289 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.289 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.7 e70/69 cost 1048576)", 2024-09-11T06:50:40.289 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.352608+0000", 2024-09-11T06:50:40.289 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.8632590640000002, 2024-09-11T06:50:40.289 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0032189060000000001, 2024-09-11T06:50:40.289 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.289 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.289 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:40.289 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.289 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.289 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.289 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.352608+0000", 2024-09-11T06:50:40.290 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.290 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.290 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.290 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.290 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.352608+0000", 2024-09-11T06:50:40.290 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.290 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.290 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.290 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.290 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.352610+0000", 2024-09-11T06:50:40.290 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.627e-06 2024-09-11T06:50:40.290 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.290 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.290 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.291 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.352641+0000", 2024-09-11T06:50:40.291 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0442000000000001e-05 2024-09-11T06:50:40.291 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.291 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.291 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.291 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.352673+0000", 2024-09-11T06:50:40.291 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2493000000000002e-05 2024-09-11T06:50:40.291 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.291 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.291 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.291 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.352693+0000", 2024-09-11T06:50:40.291 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9911999999999998e-05 2024-09-11T06:50:40.291 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.292 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.292 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.292 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.352785+0000", 2024-09-11T06:50:40.292 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.1539999999999994e-05 2024-09-11T06:50:40.292 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.292 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.292 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.292 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.355827+0000", 2024-09-11T06:50:40.292 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003041892 2024-09-11T06:50:40.292 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.292 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.292 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.292 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.292 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.293 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.7 e70/69 cost 1048576)", 2024-09-11T06:50:40.293 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.379470+0000", 2024-09-11T06:50:40.293 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.8363962709999999, 2024-09-11T06:50:40.293 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0029267730000000001, 2024-09-11T06:50:40.293 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.293 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.293 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:40.293 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.293 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.293 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.293 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.379470+0000", 2024-09-11T06:50:40.293 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.293 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.294 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.294 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.294 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.379470+0000", 2024-09-11T06:50:40.294 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.294 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.294 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.294 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.294 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.379472+0000", 2024-09-11T06:50:40.294 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6729999999999999e-06 2024-09-11T06:50:40.294 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.294 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.294 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.294 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.379494+0000", 2024-09-11T06:50:40.294 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1823000000000001e-05 2024-09-11T06:50:40.295 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.295 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.295 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.295 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.379521+0000", 2024-09-11T06:50:40.295 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7161999999999998e-05 2024-09-11T06:50:40.295 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.295 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.295 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.295 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.379538+0000", 2024-09-11T06:50:40.295 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7212999999999999e-05 2024-09-11T06:50:40.295 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.295 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.295 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.296 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.379662+0000", 2024-09-11T06:50:40.296 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000123776 2024-09-11T06:50:40.296 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.296 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.296 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.296 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.382397+0000", 2024-09-11T06:50:40.296 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027351260000000001 2024-09-11T06:50:40.296 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.296 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.296 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.296 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.296 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.296 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.7 e70/69 cost 1048576)", 2024-09-11T06:50:40.296 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.466322+0000", 2024-09-11T06:50:40.296 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.7495445849999998, 2024-09-11T06:50:40.297 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002802531, 2024-09-11T06:50:40.297 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.297 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.297 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:40.297 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.297 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.297 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.297 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.466322+0000", 2024-09-11T06:50:40.297 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.297 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.297 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.297 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.297 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.466322+0000", 2024-09-11T06:50:40.298 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.298 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.298 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.298 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.298 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.466324+0000", 2024-09-11T06:50:40.298 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7770000000000001e-06 2024-09-11T06:50:40.298 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.298 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.298 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.298 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.466345+0000", 2024-09-11T06:50:40.298 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0716e-05 2024-09-11T06:50:40.298 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.298 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.298 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.298 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.466372+0000", 2024-09-11T06:50:40.299 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7583e-05 2024-09-11T06:50:40.299 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.299 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.299 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.299 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.466386+0000", 2024-09-11T06:50:40.299 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3532e-05 2024-09-11T06:50:40.299 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.299 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.299 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.299 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.466545+0000", 2024-09-11T06:50:40.299 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000159333 2024-09-11T06:50:40.299 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.299 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.299 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.299 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.469125+0000", 2024-09-11T06:50:40.300 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025795900000000001 2024-09-11T06:50:40.300 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.300 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.300 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.300 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.300 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.300 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.7 e70/69 cost 1048576)", 2024-09-11T06:50:40.300 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.497220+0000", 2024-09-11T06:50:40.300 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.7186469310000003, 2024-09-11T06:50:40.300 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027665979999999999, 2024-09-11T06:50:40.300 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.300 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.300 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:40.300 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.301 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.301 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.301 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.497220+0000", 2024-09-11T06:50:40.301 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.301 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.301 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.301 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.301 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.497220+0000", 2024-09-11T06:50:40.301 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.301 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.301 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.301 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.301 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.497222+0000", 2024-09-11T06:50:40.301 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.883e-06 2024-09-11T06:50:40.302 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.302 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.302 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.302 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.497250+0000", 2024-09-11T06:50:40.302 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8293000000000002e-05 2024-09-11T06:50:40.302 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.302 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.302 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.302 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.497286+0000", 2024-09-11T06:50:40.302 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5651999999999997e-05 2024-09-11T06:50:40.302 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.302 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.302 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.302 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.497303+0000", 2024-09-11T06:50:40.303 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7767000000000001e-05 2024-09-11T06:50:40.303 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.303 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.303 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.303 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.497424+0000", 2024-09-11T06:50:40.303 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000120432 2024-09-11T06:50:40.303 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.303 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.303 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.303 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.499986+0000", 2024-09-11T06:50:40.303 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002562571 2024-09-11T06:50:40.303 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.303 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.303 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.304 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.304 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.304 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:40.305 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.0 to 1 (10000) 2024-09-11T06:50:40.306 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:50:40.306 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:40.303+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:40.306 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-11T06:50:40.348 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV=/dev/nbd0 2024-09-11T06:50:40.348 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd ns 2024-09-11T06:50:40.348 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:50:40.348 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns=ns 2024-09-11T06:50:40.348 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:50:40.348 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\''ns'\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd0'\'']/id' 2024-09-11T06:50:40.406 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:40.403+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:40.419 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=23121 2024-09-11T06:50:40.419 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 23121 2024-09-11T06:50:40.419 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 23121 -C rbd-nbd 2024-09-11T06:50:40.425 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:50:40.426 INFO:tasks.workunit.client.0.smithi176.stdout: 23121 ? 00:00:00 rbd-nbd 2024-09-11T06:50:40.426 INFO:tasks.workunit.client.0.smithi176.stderr:+ unmap_device '--snap-id 4 rbd/ns/testrbdnbd21607' 23121 2024-09-11T06:50:40.426 INFO:tasks.workunit.client.0.smithi176.stderr:+ local 'args=--snap-id 4 rbd/ns/testrbdnbd21607' 2024-09-11T06:50:40.426 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pid=23121 2024-09-11T06:50:40.426 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device --device-type nbd unmap --snap-id 4 rbd/ns/testrbdnbd21607 2024-09-11T06:50:40.427 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:40.427 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:40.427 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:40.428 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:50:40.429 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:50:40.429 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:40.429 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap --snap-id 4 rbd/ns/testrbdnbd21607 2024-09-11T06:50:40.478 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:40.479 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:40.479 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:50:40.479 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:40.493 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:50:40.518 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:50:40.519 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^23121\b' 2024-09-11T06:50:40.519 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^23121\b' 2024-09-11T06:50:40.536 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:40.535+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:40.572 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:40.573 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -C rbd-nbd 2024-09-11T06:50:40.573 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^ *23121\b' 2024-09-11T06:50:40.573 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^ *23121\b' 2024-09-11T06:50:40.579 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:40.579 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 0.5 2024-09-11T06:50:40.608 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:40.607+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:40.660 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:40.660 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:40.660 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:50:40.660 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:50:40.660 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:40.674 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:50:40.716 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:40.715+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:40.809 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:40.807+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:40.838 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:40.838 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:40.838 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:40.838 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:50:40.838 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.838 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:50:40.839 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:50:40.839 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 89.605941493000003, 2024-09-11T06:50:40.839 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010553425999999999, 2024-09-11T06:50:40.839 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.839 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.839 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:40.839 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:40.839 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:40.839 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:40.839 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:50:40.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.839 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.839 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.840 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.840 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:50:40.840 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.840 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.840 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.840 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.840 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:50:40.840 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.840 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.840 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.840 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.840 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230274+0000", 2024-09-11T06:50:40.840 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.533e-06 2024-09-11T06:50:40.841 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.841 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.841 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.841 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230285+0000", 2024-09-11T06:50:40.841 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0540000000000001e-05 2024-09-11T06:50:40.841 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.842 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.842 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.842 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230309+0000", 2024-09-11T06:50:40.842 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4223000000000002e-05 2024-09-11T06:50:40.842 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.842 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.842 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.842 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230327+0000", 2024-09-11T06:50:40.842 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:50:40.842 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.842 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.842 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.843 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230478+0000", 2024-09-11T06:50:40.843 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000151459 2024-09-11T06:50:40.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.843 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.843 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:40.843 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230490+0000", 2024-09-11T06:50:40.843 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.147e-05 2024-09-11T06:50:40.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.843 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.843 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.843 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231803+0000", 2024-09-11T06:50:40.843 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001313158 2024-09-11T06:50:40.844 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.844 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.844 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:50:40.844 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231817+0000", 2024-09-11T06:50:40.844 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4197e-05 2024-09-11T06:50:40.844 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.844 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.844 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.844 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.235945+0000", 2024-09-11T06:50:40.844 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0041282480000000002 2024-09-11T06:50:40.844 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.844 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.844 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:40.845 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236312+0000", 2024-09-11T06:50:40.845 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036652700000000001 2024-09-11T06:50:40.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.845 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.845 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:50:40.845 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236689+0000", 2024-09-11T06:50:40.845 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000377117 2024-09-11T06:50:40.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.846 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.846 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:40.846 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238460+0000", 2024-09-11T06:50:40.846 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001770918 2024-09-11T06:50:40.846 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.846 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.846 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:40.846 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240627+0000", 2024-09-11T06:50:40.846 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021670779999999998 2024-09-11T06:50:40.846 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.846 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.846 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:40.846 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240751+0000", 2024-09-11T06:50:40.847 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012384900000000001 2024-09-11T06:50:40.847 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.847 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.847 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.847 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240821+0000", 2024-09-11T06:50:40.847 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0516e-05 2024-09-11T06:50:40.847 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.847 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.847 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.847 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.847 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.848 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-11T06:50:40.848 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:50:40.848 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 88.603770650000001, 2024-09-11T06:50:40.848 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069904770000000001, 2024-09-11T06:50:40.848 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.848 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.848 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:40.848 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:40.848 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:40.848 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:40.848 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 5 2024-09-11T06:50:40.848 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.848 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.849 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.849 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.849 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:50:40.849 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.849 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.849 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.849 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.849 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:50:40.849 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.849 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.849 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.849 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.849 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232442+0000", 2024-09-11T06:50:40.850 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.044e-06 2024-09-11T06:50:40.850 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.850 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.850 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.850 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232460+0000", 2024-09-11T06:50:40.850 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7863000000000002e-05 2024-09-11T06:50:40.850 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.850 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.850 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.850 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232492+0000", 2024-09-11T06:50:40.850 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2812999999999999e-05 2024-09-11T06:50:40.850 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.850 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.850 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.851 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232510+0000", 2024-09-11T06:50:40.851 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7552999999999999e-05 2024-09-11T06:50:40.851 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.851 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.851 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.851 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232640+0000", 2024-09-11T06:50:40.851 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013029900000000001 2024-09-11T06:50:40.851 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.851 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.851 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:40.851 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232650+0000", 2024-09-11T06:50:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0023999999999999e-05 2024-09-11T06:50:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235641+0000", 2024-09-11T06:50:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00299089 2024-09-11T06:50:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235980+0000", 2024-09-11T06:50:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000338907 2024-09-11T06:50:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:40.853 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.236351+0000", 2024-09-11T06:50:40.853 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037087699999999998 2024-09-11T06:50:40.853 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.853 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.853 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:40.853 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238074+0000", 2024-09-11T06:50:40.853 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001722859 2024-09-11T06:50:40.853 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.853 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.853 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:40.853 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239160+0000", 2024-09-11T06:50:40.853 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001086074 2024-09-11T06:50:40.853 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239299+0000", 2024-09-11T06:50:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013933500000000001 2024-09-11T06:50:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239429+0000", 2024-09-11T06:50:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129939 2024-09-11T06:50:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-11T06:50:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:50:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 87.589682698000004, 2024-09-11T06:50:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069253250000000004, 2024-09-11T06:50:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 7 2024-09-11T06:50:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.856 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:50:40.856 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.856 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.856 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.856 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.856 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:50:40.856 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.856 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.856 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.856 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.856 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246530+0000", 2024-09-11T06:50:40.856 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9469999999999999e-06 2024-09-11T06:50:40.856 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.856 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.857 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.857 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246545+0000", 2024-09-11T06:50:40.857 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5973000000000001e-05 2024-09-11T06:50:40.857 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.857 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.857 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.857 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246578+0000", 2024-09-11T06:50:40.857 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2759999999999998e-05 2024-09-11T06:50:40.857 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.857 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.857 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.857 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246595+0000", 2024-09-11T06:50:40.857 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6939000000000002e-05 2024-09-11T06:50:40.857 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.858 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.858 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.858 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246770+0000", 2024-09-11T06:50:40.858 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000174723 2024-09-11T06:50:40.858 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.858 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.858 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:40.858 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246789+0000", 2024-09-11T06:50:40.858 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9159000000000001e-05 2024-09-11T06:50:40.858 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.858 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.858 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.858 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250337+0000", 2024-09-11T06:50:40.858 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035480759999999998 2024-09-11T06:50:40.858 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.859 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.859 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:40.859 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250549+0000", 2024-09-11T06:50:40.859 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021192199999999999 2024-09-11T06:50:40.859 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.859 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.859 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:40.859 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250750+0000", 2024-09-11T06:50:40.859 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020104199999999999 2024-09-11T06:50:40.859 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.859 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.859 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:40.859 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.252455+0000", 2024-09-11T06:50:40.859 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001705249 2024-09-11T06:50:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253305+0000", 2024-09-11T06:50:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00084941200000000002 2024-09-11T06:50:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253388+0000", 2024-09-11T06:50:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3642999999999999e-05 2024-09-11T06:50:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253452+0000", 2024-09-11T06:50:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3479999999999995e-05 2024-09-11T06:50:40.861 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.861 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.861 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.861 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.861 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.861 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(client.4558.0:5 3.2 e57/55 3:5cea7035:::rbd_info:head v 57'1, mlcod=0'0)", 2024-09-11T06:50:40.861 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:50:40.861 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 22.741578336, 2024-09-11T06:50:40.861 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011016419999999999, 2024-09-11T06:50:40.861 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.861 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.861 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:40.861 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.862 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.862 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.862 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:50:40.862 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.862 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.862 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.862 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.862 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:50:40.862 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.862 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.862 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.863 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.863 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:50:40.863 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4000000000000002e-07 2024-09-11T06:50:40.863 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.863 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.863 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.863 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094640+0000", 2024-09-11T06:50:40.863 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6230000000000003e-06 2024-09-11T06:50:40.863 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.863 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.863 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.863 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094669+0000", 2024-09-11T06:50:40.863 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8462e-05 2024-09-11T06:50:40.863 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.864 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.864 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.864 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094684+0000", 2024-09-11T06:50:40.864 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5536000000000002e-05 2024-09-11T06:50:40.864 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.864 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.864 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.864 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094798+0000", 2024-09-11T06:50:40.864 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011439599999999999 2024-09-11T06:50:40.864 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.864 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.865 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:40.865 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094819+0000", 2024-09-11T06:50:40.865 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0296e-05 2024-09-11T06:50:40.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.865 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.865 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:40.865 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.095687+0000", 2024-09-11T06:50:40.865 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086851799999999998 2024-09-11T06:50:40.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.866 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.866 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.095733+0000", 2024-09-11T06:50:40.866 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5271000000000001e-05 2024-09-11T06:50:40.866 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.866 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.866 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.866 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:160 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 4096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:40.866 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:50:40.866 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 19.802463328999998, 2024-09-11T06:50:40.866 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001122215, 2024-09-11T06:50:40.866 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.867 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.867 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:40.867 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:40.867 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:40.867 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:40.867 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 160 2024-09-11T06:50:40.867 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.867 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.867 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.867 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.867 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:50:40.867 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.867 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.867 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.867 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.868 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:50:40.868 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.868 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.868 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.868 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.868 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:50:40.868 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0999999999999999e-07 2024-09-11T06:50:40.868 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.868 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.868 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.869 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033749+0000", 2024-09-11T06:50:40.869 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8930000000000001e-06 2024-09-11T06:50:40.869 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.869 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.869 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.869 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033758+0000", 2024-09-11T06:50:40.869 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.242e-06 2024-09-11T06:50:40.869 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.869 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.869 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.869 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033764+0000", 2024-09-11T06:50:40.869 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0510000000000002e-06 2024-09-11T06:50:40.869 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.870 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.870 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.870 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.034207+0000", 2024-09-11T06:50:40.870 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00044320600000000003 2024-09-11T06:50:40.870 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.870 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.870 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:40.870 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.034280+0000", 2024-09-11T06:50:40.870 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3094000000000002e-05 2024-09-11T06:50:40.870 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.870 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.870 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.870 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.034868+0000", 2024-09-11T06:50:40.871 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00058821899999999996 2024-09-11T06:50:40.871 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.871 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.871 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.871 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.871 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.871 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:162 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:40.871 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.039519+0000", 2024-09-11T06:50:40.871 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 19.796690211000001, 2024-09-11T06:50:40.871 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001298485, 2024-09-11T06:50:40.871 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.871 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.871 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:40.872 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:40.872 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:40.872 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:40.872 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 162 2024-09-11T06:50:40.872 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.872 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.872 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.872 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.872 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039519+0000", 2024-09-11T06:50:40.872 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.872 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.872 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.873 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.873 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039519+0000", 2024-09-11T06:50:40.873 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.873 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.873 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.873 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.873 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039520+0000", 2024-09-11T06:50:40.873 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9299999999999998e-07 2024-09-11T06:50:40.873 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.873 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.873 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.873 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039523+0000", 2024-09-11T06:50:40.874 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.247e-06 2024-09-11T06:50:40.874 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.874 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.874 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.874 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039529+0000", 2024-09-11T06:50:40.874 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8300000000000001e-06 2024-09-11T06:50:40.874 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.874 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.874 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.874 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039533+0000", 2024-09-11T06:50:40.874 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.2379999999999999e-06 2024-09-11T06:50:40.874 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.874 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.875 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.875 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.040236+0000", 2024-09-11T06:50:40.875 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00070326500000000001 2024-09-11T06:50:40.875 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.875 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.875 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:40.875 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.040299+0000", 2024-09-11T06:50:40.875 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3077999999999995e-05 2024-09-11T06:50:40.875 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.875 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.875 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.875 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.040818+0000", 2024-09-11T06:50:40.875 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00051833399999999996 2024-09-11T06:50:40.875 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.875 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.876 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.876 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.876 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.876 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:164 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 528384~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:40.876 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.041536+0000", 2024-09-11T06:50:40.876 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 19.794673489000001, 2024-09-11T06:50:40.876 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00119962, 2024-09-11T06:50:40.876 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.876 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.876 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:40.876 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:40.876 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:40.876 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:40.877 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 164 2024-09-11T06:50:40.877 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.877 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.877 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.877 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.877 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041536+0000", 2024-09-11T06:50:40.877 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.877 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.877 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.877 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.877 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041536+0000", 2024-09-11T06:50:40.877 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.877 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.877 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.878 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.878 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041537+0000", 2024-09-11T06:50:40.878 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3e-06 2024-09-11T06:50:40.878 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.878 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.878 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.878 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041543+0000", 2024-09-11T06:50:40.878 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2360000000000001e-06 2024-09-11T06:50:40.878 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.878 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.878 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.878 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041558+0000", 2024-09-11T06:50:40.878 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4909000000000001e-05 2024-09-11T06:50:40.878 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.879 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.879 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.879 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041569+0000", 2024-09-11T06:50:40.879 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0946e-05 2024-09-11T06:50:40.879 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.879 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.879 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.879 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.042138+0000", 2024-09-11T06:50:40.879 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000568996 2024-09-11T06:50:40.879 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.879 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.879 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:40.880 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.042190+0000", 2024-09-11T06:50:40.880 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1609e-05 2024-09-11T06:50:40.880 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.880 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.880 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.880 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.042735+0000", 2024-09-11T06:50:40.880 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00054562399999999998 2024-09-11T06:50:40.880 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.881 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.881 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.881 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.881 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.881 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:166 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 790528~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:40.881 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.043564+0000", 2024-09-11T06:50:40.881 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 19.792645092000001, 2024-09-11T06:50:40.881 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011042000000000001, 2024-09-11T06:50:40.881 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.881 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.881 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:40.881 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:40.882 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:40.882 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:40.882 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 166 2024-09-11T06:50:40.882 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.882 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.882 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.882 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.882 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043564+0000", 2024-09-11T06:50:40.882 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.882 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.882 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.882 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.882 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043564+0000", 2024-09-11T06:50:40.882 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.883 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.883 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.883 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.883 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043565+0000", 2024-09-11T06:50:40.883 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0449999999999999e-06 2024-09-11T06:50:40.883 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.883 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.883 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.883 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043571+0000", 2024-09-11T06:50:40.883 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.6849999999999998e-06 2024-09-11T06:50:40.883 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.883 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.883 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.883 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043586+0000", 2024-09-11T06:50:40.884 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4902000000000001e-05 2024-09-11T06:50:40.884 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.884 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.884 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.884 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043593+0000", 2024-09-11T06:50:40.884 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9240000000000001e-06 2024-09-11T06:50:40.884 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.884 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.884 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.884 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.044073+0000", 2024-09-11T06:50:40.884 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000479813 2024-09-11T06:50:40.884 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.884 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.885 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:40.885 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.044123+0000", 2024-09-11T06:50:40.885 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0908000000000001e-05 2024-09-11T06:50:40.885 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.885 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.885 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.885 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.044668+0000", 2024-09-11T06:50:40.885 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000544923 2024-09-11T06:50:40.885 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.885 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.885 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.885 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.885 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.886 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:178 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 2363392~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:40.886 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.061599+0000", 2024-09-11T06:50:40.886 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 19.774610376999998, 2024-09-11T06:50:40.886 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012194529999999999, 2024-09-11T06:50:40.886 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.886 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.886 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:40.886 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:40.886 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:40.886 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:40.886 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 178 2024-09-11T06:50:40.886 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.886 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.886 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.887 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.887 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061599+0000", 2024-09-11T06:50:40.887 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.887 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.887 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.887 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061599+0000", 2024-09-11T06:50:40.887 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.887 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.888 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.888 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061600+0000", 2024-09-11T06:50:40.888 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3600000000000003e-07 2024-09-11T06:50:40.888 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.888 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.888 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.888 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061603+0000", 2024-09-11T06:50:40.888 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.6550000000000002e-06 2024-09-11T06:50:40.888 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.888 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.888 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.889 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061618+0000", 2024-09-11T06:50:40.889 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4377e-05 2024-09-11T06:50:40.889 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.889 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.889 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.889 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061626+0000", 2024-09-11T06:50:40.889 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.0679999999999997e-06 2024-09-11T06:50:40.889 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.889 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.889 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.889 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.062098+0000", 2024-09-11T06:50:40.889 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00047251299999999998 2024-09-11T06:50:40.890 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.890 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.890 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:40.890 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.062253+0000", 2024-09-11T06:50:40.890 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015533100000000001 2024-09-11T06:50:40.890 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.890 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.890 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.890 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.062818+0000", 2024-09-11T06:50:40.890 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00056487300000000002 2024-09-11T06:50:40.890 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.890 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.890 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.891 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.891 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:184 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 3149824~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:40.891 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:50:40.891 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 19.770000673999998, 2024-09-11T06:50:40.891 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010685969999999999, 2024-09-11T06:50:40.891 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.891 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.891 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:40.891 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:40.891 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:40.891 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:40.891 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 184 2024-09-11T06:50:40.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.892 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.892 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.892 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.892 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:50:40.892 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.892 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.892 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.892 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.892 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:50:40.892 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.892 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.892 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.892 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.893 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:50:40.893 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.6700000000000003e-07 2024-09-11T06:50:40.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.893 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.893 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.893 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066212+0000", 2024-09-11T06:50:40.893 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.1669999999999999e-06 2024-09-11T06:50:40.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.893 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.893 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.893 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066219+0000", 2024-09-11T06:50:40.893 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5699999999999998e-06 2024-09-11T06:50:40.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.894 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.894 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.894 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066223+0000", 2024-09-11T06:50:40.894 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3540000000000002e-06 2024-09-11T06:50:40.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.894 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.894 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.894 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066713+0000", 2024-09-11T06:50:40.894 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00048985399999999996 2024-09-11T06:50:40.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.894 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.894 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:40.894 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066813+0000", 2024-09-11T06:50:40.895 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00010003 2024-09-11T06:50:40.895 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.895 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.895 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.895 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.067277+0000", 2024-09-11T06:50:40.895 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000463955 2024-09-11T06:50:40.895 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.895 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.895 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.895 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.895 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.895 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:186 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 3411968~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:40.895 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.071198+0000", 2024-09-11T06:50:40.895 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 19.76501133, 2024-09-11T06:50:40.896 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012515779999999999, 2024-09-11T06:50:40.896 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.896 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.896 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:40.896 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:40.896 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:40.896 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:40.896 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 186 2024-09-11T06:50:40.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.896 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.896 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.896 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.896 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071198+0000", 2024-09-11T06:50:40.896 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.897 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.897 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.897 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.897 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071198+0000", 2024-09-11T06:50:40.897 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.897 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.897 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.897 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.897 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071199+0000", 2024-09-11T06:50:40.897 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0299999999999999e-07 2024-09-11T06:50:40.897 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.897 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.897 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.897 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071201+0000", 2024-09-11T06:50:40.898 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8540000000000001e-06 2024-09-11T06:50:40.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.898 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.898 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.898 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071209+0000", 2024-09-11T06:50:40.898 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3629999999999996e-06 2024-09-11T06:50:40.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.898 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.898 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.898 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071214+0000", 2024-09-11T06:50:40.898 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2109999999999997e-06 2024-09-11T06:50:40.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.899 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.899 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071915+0000", 2024-09-11T06:50:40.899 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00070101799999999998 2024-09-11T06:50:40.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.899 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:40.899 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.072011+0000", 2024-09-11T06:50:40.899 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.5755999999999996e-05 2024-09-11T06:50:40.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.899 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.900 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.072450+0000", 2024-09-11T06:50:40.900 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000438773 2024-09-11T06:50:40.900 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.900 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.900 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.900 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.900 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.900 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:190 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 3936256~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:40.900 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.074875+0000", 2024-09-11T06:50:40.900 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 19.761334415, 2024-09-11T06:50:40.900 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0013119620000000001, 2024-09-11T06:50:40.900 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.901 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.901 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:40.901 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:40.901 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:40.901 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:40.901 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 190 2024-09-11T06:50:40.901 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.901 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.901 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.901 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.901 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074875+0000", 2024-09-11T06:50:40.901 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.901 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.902 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.902 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.902 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074875+0000", 2024-09-11T06:50:40.902 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.902 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.902 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.902 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.902 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074876+0000", 2024-09-11T06:50:40.902 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7400000000000003e-07 2024-09-11T06:50:40.902 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.902 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.902 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.902 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074887+0000", 2024-09-11T06:50:40.903 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1817999999999999e-05 2024-09-11T06:50:40.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.903 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.903 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.903 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074896+0000", 2024-09-11T06:50:40.903 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9120000000000004e-06 2024-09-11T06:50:40.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.903 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.903 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.903 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074900+0000", 2024-09-11T06:50:40.903 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.2220000000000004e-06 2024-09-11T06:50:40.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.904 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.904 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.904 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.075198+0000", 2024-09-11T06:50:40.904 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00029750399999999998 2024-09-11T06:50:40.904 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.904 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.904 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:40.904 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.075430+0000", 2024-09-11T06:50:40.904 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023200400000000001 2024-09-11T06:50:40.904 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.904 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.904 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.905 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.076187+0000", 2024-09-11T06:50:40.905 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00075682799999999997 2024-09-11T06:50:40.905 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.905 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.905 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.905 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.905 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:359 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 135168~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:40.905 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.298832+0000", 2024-09-11T06:50:40.905 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 19.537376780999999, 2024-09-11T06:50:40.905 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001078575, 2024-09-11T06:50:40.905 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.905 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.905 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:40.906 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:40.906 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:40.906 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:40.906 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 359 2024-09-11T06:50:40.906 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.906 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.906 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.906 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.906 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298832+0000", 2024-09-11T06:50:40.906 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.906 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.906 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.906 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.906 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298832+0000", 2024-09-11T06:50:40.907 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.907 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.907 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.907 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.907 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298834+0000", 2024-09-11T06:50:40.907 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.646e-06 2024-09-11T06:50:40.907 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.907 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.907 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.907 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298845+0000", 2024-09-11T06:50:40.907 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.081e-05 2024-09-11T06:50:40.907 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.907 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.907 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.907 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298872+0000", 2024-09-11T06:50:40.908 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7382e-05 2024-09-11T06:50:40.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.908 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.908 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.908 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298884+0000", 2024-09-11T06:50:40.908 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1956e-05 2024-09-11T06:50:40.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.908 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.908 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.908 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298959+0000", 2024-09-11T06:50:40.908 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5214000000000002e-05 2024-09-11T06:50:40.909 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.909 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.909 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:40.909 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.299090+0000", 2024-09-11T06:50:40.909 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000130247 2024-09-11T06:50:40.909 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.909 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.909 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.909 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.299911+0000", 2024-09-11T06:50:40.909 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00082131999999999995 2024-09-11T06:50:40.909 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.909 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.910 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.910 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.910 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.910 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:360 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:40.910 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.298914+0000", 2024-09-11T06:50:40.910 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 19.537295110999999, 2024-09-11T06:50:40.910 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0023571690000000001, 2024-09-11T06:50:40.910 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.910 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.910 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:40.910 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:40.910 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:40.910 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:40.910 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 360 2024-09-11T06:50:40.910 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.911 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.911 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.911 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.911 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298914+0000", 2024-09-11T06:50:40.911 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.911 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.911 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.911 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.911 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298914+0000", 2024-09-11T06:50:40.911 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.911 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.911 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.911 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.911 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298915+0000", 2024-09-11T06:50:40.912 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9300000000000006e-07 2024-09-11T06:50:40.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.912 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.912 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.912 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298921+0000", 2024-09-11T06:50:40.912 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.6030000000000004e-06 2024-09-11T06:50:40.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.912 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.912 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.912 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298932+0000", 2024-09-11T06:50:40.912 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.082e-05 2024-09-11T06:50:40.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.912 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.912 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.912 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298939+0000", 2024-09-11T06:50:40.913 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7919999999999993e-06 2024-09-11T06:50:40.913 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.913 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.913 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.913 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.300011+0000", 2024-09-11T06:50:40.913 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001071179 2024-09-11T06:50:40.913 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.913 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.913 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:40.913 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.300245+0000", 2024-09-11T06:50:40.913 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000234434 2024-09-11T06:50:40.913 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.914 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.914 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.914 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.301271+0000", 2024-09-11T06:50:40.914 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010263480000000001 2024-09-11T06:50:40.914 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.914 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.914 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.914 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.914 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.914 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:362 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 528384~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:40.914 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.302376+0000", 2024-09-11T06:50:40.914 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 19.533832788000002, 2024-09-11T06:50:40.914 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014132909999999999, 2024-09-11T06:50:40.914 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.914 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.915 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:40.915 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:40.915 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:40.915 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:40.915 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 362 2024-09-11T06:50:40.915 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.915 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.915 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.915 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.915 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302376+0000", 2024-09-11T06:50:40.915 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.915 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.915 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.915 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.915 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302376+0000", 2024-09-11T06:50:40.916 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.916 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.916 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.916 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.916 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302378+0000", 2024-09-11T06:50:40.916 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1629999999999999e-06 2024-09-11T06:50:40.916 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.916 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.916 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.916 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302383+0000", 2024-09-11T06:50:40.916 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.463e-06 2024-09-11T06:50:40.916 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.916 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.916 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.916 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302401+0000", 2024-09-11T06:50:40.917 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8219e-05 2024-09-11T06:50:40.917 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.917 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.917 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.917 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302411+0000", 2024-09-11T06:50:40.917 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9000000000000001e-06 2024-09-11T06:50:40.917 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.917 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.917 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.917 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302524+0000", 2024-09-11T06:50:40.917 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000113199 2024-09-11T06:50:40.917 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.917 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.917 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:40.918 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302745+0000", 2024-09-11T06:50:40.918 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022057399999999999 2024-09-11T06:50:40.918 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.918 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.918 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.918 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.303790+0000", 2024-09-11T06:50:40.918 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010447729999999999 2024-09-11T06:50:40.918 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.918 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.918 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.918 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.918 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.918 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:363 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 659456~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:40.919 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.302938+0000", 2024-09-11T06:50:40.919 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 19.533271195000001, 2024-09-11T06:50:40.919 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00185012, 2024-09-11T06:50:40.919 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.919 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.919 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:40.919 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:40.919 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:40.919 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:40.919 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 363 2024-09-11T06:50:40.919 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.919 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.919 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.919 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.920 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302938+0000", 2024-09-11T06:50:40.920 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.920 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.920 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.920 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.920 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302938+0000", 2024-09-11T06:50:40.920 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.920 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.920 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.920 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.920 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302939+0000", 2024-09-11T06:50:40.921 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2300000000000001e-06 2024-09-11T06:50:40.921 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.921 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.921 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.921 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302944+0000", 2024-09-11T06:50:40.921 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.493e-06 2024-09-11T06:50:40.921 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.921 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.921 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.921 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302960+0000", 2024-09-11T06:50:40.921 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.632e-05 2024-09-11T06:50:40.921 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.921 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.922 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.922 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302972+0000", 2024-09-11T06:50:40.922 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1885000000000001e-05 2024-09-11T06:50:40.922 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.922 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.922 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.922 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.303835+0000", 2024-09-11T06:50:40.922 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000863141 2024-09-11T06:50:40.922 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.922 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.922 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:40.923 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.303965+0000", 2024-09-11T06:50:40.923 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013032199999999999 2024-09-11T06:50:40.923 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.923 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.923 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.923 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.304788+0000", 2024-09-11T06:50:40.923 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00082272900000000002 2024-09-11T06:50:40.923 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.923 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.923 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.923 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.923 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.923 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:366 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 1052672~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:40.923 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.306960+0000", 2024-09-11T06:50:40.923 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 19.529249325999999, 2024-09-11T06:50:40.924 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010286749999999999, 2024-09-11T06:50:40.924 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.924 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.924 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:40.924 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:40.924 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:40.924 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:40.924 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 366 2024-09-11T06:50:40.924 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.924 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.924 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.924 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.924 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.306960+0000", 2024-09-11T06:50:40.924 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.925 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.925 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.925 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.925 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.306960+0000", 2024-09-11T06:50:40.925 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.925 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.925 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.925 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.925 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.306961+0000", 2024-09-11T06:50:40.925 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3939999999999999e-06 2024-09-11T06:50:40.925 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.925 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.925 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.926 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.306970+0000", 2024-09-11T06:50:40.926 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6859999999999995e-06 2024-09-11T06:50:40.926 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.926 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.926 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.926 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.306991+0000", 2024-09-11T06:50:40.926 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0772e-05 2024-09-11T06:50:40.926 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.926 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.926 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.926 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.307002+0000", 2024-09-11T06:50:40.926 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0757000000000001e-05 2024-09-11T06:50:40.926 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.926 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.927 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.927 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.307077+0000", 2024-09-11T06:50:40.927 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5337000000000001e-05 2024-09-11T06:50:40.927 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.927 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.927 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:40.927 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.307185+0000", 2024-09-11T06:50:40.927 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000108335 2024-09-11T06:50:40.927 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.927 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.927 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.927 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.307989+0000", 2024-09-11T06:50:40.927 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00080339400000000003 2024-09-11T06:50:40.927 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.927 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.928 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.928 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.928 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.928 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:367 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 1183744~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:40.928 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.312392+0000", 2024-09-11T06:50:40.928 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 19.523817555000001, 2024-09-11T06:50:40.928 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011343460000000001, 2024-09-11T06:50:40.928 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.928 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.928 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:40.928 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:40.928 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:40.928 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:40.929 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 367 2024-09-11T06:50:40.929 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.929 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.929 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.929 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.929 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312392+0000", 2024-09-11T06:50:40.929 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.929 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.929 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.929 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.929 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312392+0000", 2024-09-11T06:50:40.929 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.929 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.929 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.930 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.930 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312393+0000", 2024-09-11T06:50:40.930 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6199999999999999e-06 2024-09-11T06:50:40.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.930 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.930 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.930 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312400+0000", 2024-09-11T06:50:40.930 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0700000000000001e-06 2024-09-11T06:50:40.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.930 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.930 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.930 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312420+0000", 2024-09-11T06:50:40.930 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9375999999999999e-05 2024-09-11T06:50:40.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.931 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.931 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.931 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312430+0000", 2024-09-11T06:50:40.931 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0209e-05 2024-09-11T06:50:40.931 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.931 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.931 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.931 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312549+0000", 2024-09-11T06:50:40.931 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000118779 2024-09-11T06:50:40.931 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.931 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.931 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:40.931 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312767+0000", 2024-09-11T06:50:40.931 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021786400000000001 2024-09-11T06:50:40.932 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.932 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.932 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.932 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313526+0000", 2024-09-11T06:50:40.932 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00075942800000000003 2024-09-11T06:50:40.932 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.932 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.932 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.932 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.932 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.932 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:368 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 1314816~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:40.932 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.312454+0000", 2024-09-11T06:50:40.932 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 19.523755031, 2024-09-11T06:50:40.932 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014704550000000001, 2024-09-11T06:50:40.933 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.933 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.933 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:40.933 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:40.933 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:40.933 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:40.933 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 368 2024-09-11T06:50:40.933 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.933 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.933 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.933 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.933 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312454+0000", 2024-09-11T06:50:40.933 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.934 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.934 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.934 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.934 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312454+0000", 2024-09-11T06:50:40.934 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.934 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.934 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.934 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.934 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312455+0000", 2024-09-11T06:50:40.934 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4399999999999999e-07 2024-09-11T06:50:40.934 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.935 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.935 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.935 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312474+0000", 2024-09-11T06:50:40.935 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8899e-05 2024-09-11T06:50:40.935 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.935 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.935 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.935 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312485+0000", 2024-09-11T06:50:40.935 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0793e-05 2024-09-11T06:50:40.935 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.935 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.935 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.935 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312492+0000", 2024-09-11T06:50:40.936 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.593e-06 2024-09-11T06:50:40.936 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.936 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.936 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.936 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313576+0000", 2024-09-11T06:50:40.936 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010837970000000001 2024-09-11T06:50:40.936 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.936 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.936 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:40.936 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313620+0000", 2024-09-11T06:50:40.936 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3640000000000002e-05 2024-09-11T06:50:40.936 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.936 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.936 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.937 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313925+0000", 2024-09-11T06:50:40.937 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00030498899999999998 2024-09-11T06:50:40.937 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.937 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.937 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.937 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.937 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.937 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:370 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 1576960~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:40.937 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.314817+0000", 2024-09-11T06:50:40.937 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 19.521392322000001, 2024-09-11T06:50:40.937 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011606190000000001, 2024-09-11T06:50:40.937 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:40.937 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:40.937 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:40.937 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:40.938 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:40.938 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:40.938 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 370 2024-09-11T06:50:40.938 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.938 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:40.938 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.938 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:40.938 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314817+0000", 2024-09-11T06:50:40.938 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.938 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.938 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.938 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:40.938 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314817+0000", 2024-09-11T06:50:40.938 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:40.939 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.939 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.939 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:40.939 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314818+0000", 2024-09-11T06:50:40.939 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.463e-06 2024-09-11T06:50:40.939 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.939 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.939 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:40.939 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314826+0000", 2024-09-11T06:50:40.939 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3860000000000001e-06 2024-09-11T06:50:40.939 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.939 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.939 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:40.939 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314842+0000", 2024-09-11T06:50:40.940 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6220000000000001e-05 2024-09-11T06:50:40.940 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.940 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.940 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:40.940 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314854+0000", 2024-09-11T06:50:40.940 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1562e-05 2024-09-11T06:50:40.940 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.940 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.940 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:40.940 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.315212+0000", 2024-09-11T06:50:40.940 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000358012 2024-09-11T06:50:40.940 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.940 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.940 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:40.941 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.315433+0000", 2024-09-11T06:50:40.941 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022103800000000001 2024-09-11T06:50:40.941 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:40.941 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:40.941 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:40.941 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.315978+0000", 2024-09-11T06:50:40.941 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00054493800000000002 2024-09-11T06:50:40.941 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.941 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.941 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.941 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:40.941 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:40.941 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:40.943 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:50:40.943 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:40.939+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:41.044 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:41.039+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:41.072 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:41.072 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:41.072 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:50:41.072 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:41.080 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV= 2024-09-11T06:50:41.081 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo rbd device --device-type nbd map --pool rbd --namespace ns --image testrbdnbd21607 2024-09-11T06:50:41.081 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:50:41.081 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:50:41.082 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:50:41.083 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which rbd 2024-09-11T06:50:41.085 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/rbd 2024-09-11T06:50:41.085 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:50:41.085 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map --pool rbd --namespace ns --image testrbdnbd21607 2024-09-11T06:50:41.086 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:50:41.181 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:41.179+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:41.231 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:50:41.232 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV=/dev/nbd0 2024-09-11T06:50:41.233 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd ns 2024-09-11T06:50:41.233 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:50:41.233 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns=ns 2024-09-11T06:50:41.234 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:50:41.234 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\''ns'\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd0'\'']/id' 2024-09-11T06:50:41.245 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:41.243+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:41.255 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:41.256 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:41.256 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:50:41.256 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:50:41.256 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:41.262 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.2 to 1 (10000) 2024-09-11T06:50:41.266 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is unset 2024-09-11T06:50:41.278 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:50:41.291 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-11T06:50:41.303 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=23183 2024-09-11T06:50:41.303 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 23183 2024-09-11T06:50:41.303 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 23183 -C rbd-nbd 2024-09-11T06:50:41.311 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:50:41.311 INFO:tasks.workunit.client.0.smithi176.stdout: 23183 ? 00:00:00 rbd-nbd 2024-09-11T06:50:41.312 INFO:tasks.workunit.client.0.smithi176.stderr:+ unmap_device '--pool rbd --namespace ns --image testrbdnbd21607' 23183 2024-09-11T06:50:41.312 INFO:tasks.workunit.client.0.smithi176.stderr:+ local 'args=--pool rbd --namespace ns --image testrbdnbd21607' 2024-09-11T06:50:41.312 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pid=23183 2024-09-11T06:50:41.312 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device --device-type nbd unmap --pool rbd --namespace ns --image testrbdnbd21607 2024-09-11T06:50:41.312 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:41.312 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:41.312 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:41.313 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:50:41.314 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:50:41.314 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:41.314 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap --pool rbd --namespace ns --image testrbdnbd21607 2024-09-11T06:50:41.377 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:41.375+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:41.378 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:50:41.379 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:50:41.395 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:50:41.396 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:50:41.401 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:50:41.402 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^23183\b' 2024-09-11T06:50:41.402 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^23183\b' 2024-09-11T06:50:41.408 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:41.408 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:41.412 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:50:41.413 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:50:41.423 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:50:41.433 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:50:41.434 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:50:41.447 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:41.443+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:41.449 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:41.449 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -C rbd-nbd 2024-09-11T06:50:41.449 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^ *23183\b' 2024-09-11T06:50:41.450 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^ *23183\b' 2024-09-11T06:50:41.455 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:50:41.455 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:50:41.456 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:41.456 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 0.5 2024-09-11T06:50:41.485 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:50:41.485 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:50:41.547 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:41.547+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:41.548 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:41.548 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:41.562 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:50:41.649 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:41.647+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:41.729 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:41.729 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:41.743 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:50:41.750 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:41.747+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:41.851 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:41.847+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:41.900 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:41.900 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:41.913 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:50:41.957 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:41.955+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:41.958 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo rbd device --device-type nbd map --pool rbd --namespace ns --image testrbdnbd21607 --snap snap 2024-09-11T06:50:41.958 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:50:41.958 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:50:41.960 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:50:41.960 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which rbd 2024-09-11T06:50:41.962 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/rbd 2024-09-11T06:50:41.963 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:50:41.963 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map --pool rbd --namespace ns --image testrbdnbd21607 --snap snap 2024-09-11T06:50:42.020 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:42.020 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:42.032 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:50:42.053 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:42.051+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:42.113 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV=/dev/nbd0 2024-09-11T06:50:42.113 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd ns 2024-09-11T06:50:42.113 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:50:42.113 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns=ns 2024-09-11T06:50:42.113 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:50:42.113 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\''ns'\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd0'\'']/id' 2024-09-11T06:50:42.154 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:42.151+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:42.179 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=23245 2024-09-11T06:50:42.179 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 23245 2024-09-11T06:50:42.179 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 23245 -C rbd-nbd 2024-09-11T06:50:42.186 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:50:42.186 INFO:tasks.workunit.client.0.smithi176.stdout: 23245 ? 00:00:00 rbd-nbd 2024-09-11T06:50:42.187 INFO:tasks.workunit.client.0.smithi176.stderr:+ unmap_device '--pool rbd --namespace ns --image testrbdnbd21607 --snap snap' 23245 2024-09-11T06:50:42.187 INFO:tasks.workunit.client.0.smithi176.stderr:+ local 'args=--pool rbd --namespace ns --image testrbdnbd21607 --snap snap' 2024-09-11T06:50:42.187 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pid=23245 2024-09-11T06:50:42.187 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device --device-type nbd unmap --pool rbd --namespace ns --image testrbdnbd21607 --snap snap 2024-09-11T06:50:42.187 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:42.187 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:42.188 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:42.188 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:50:42.189 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:50:42.189 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:42.190 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap --pool rbd --namespace ns --image testrbdnbd21607 --snap snap 2024-09-11T06:50:42.194 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:42.194 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:42.208 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:50:42.271 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.4 to 1 (10000) 2024-09-11T06:50:42.271 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:42.267+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:42.284 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:50:42.284 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^23245\b' 2024-09-11T06:50:42.285 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^23245\b' 2024-09-11T06:50:42.291 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-11T06:50:42.333 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:42.334 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -C rbd-nbd 2024-09-11T06:50:42.334 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^ *23245\b' 2024-09-11T06:50:42.334 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^ *23245\b' 2024-09-11T06:50:42.340 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:42.340 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 0.5 2024-09-11T06:50:42.356 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:42.355+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:42.375 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:42.375 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:42.390 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:50:42.456 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:42.451+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:42.522 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:42.522 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:42.536 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:50:42.579 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:42.579+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:42.654 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:42.654 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:42.657 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:42.655+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:42.668 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:50:42.759 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:42.755+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:42.835 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:42.836 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:42.841 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV= 2024-09-11T06:50:42.842 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo rbd device --device-type nbd map rbd/testrbdnbd21607 2024-09-11T06:50:42.842 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:50:42.843 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:50:42.844 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:50:42.844 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which rbd 2024-09-11T06:50:42.846 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/rbd 2024-09-11T06:50:42.847 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:50:42.847 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map rbd/testrbdnbd21607 2024-09-11T06:50:42.849 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:50:42.892 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:42.891+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:42.960 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:42.955+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:42.986 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV=/dev/nbd0 2024-09-11T06:50:42.986 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd 2024-09-11T06:50:42.986 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:50:42.987 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:50:42.987 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:50:42.987 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd0'\'']/id' 2024-09-11T06:50:43.013 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:43.013 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:43.027 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:50:43.053 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=23307 2024-09-11T06:50:43.053 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 23307 2024-09-11T06:50:43.053 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 23307 -C rbd-nbd 2024-09-11T06:50:43.058 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:50:43.059 INFO:tasks.workunit.client.0.smithi176.stdout: 23307 ? 00:00:00 rbd-nbd 2024-09-11T06:50:43.059 INFO:tasks.workunit.client.0.smithi176.stderr:+ pid=23307 2024-09-11T06:50:43.060 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo rbd device --device-type nbd map rbd/ns/testrbdnbd21607 2024-09-11T06:50:43.060 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:50:43.060 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:50:43.060 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:50:43.061 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which rbd 2024-09-11T06:50:43.062 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/rbd 2024-09-11T06:50:43.062 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:50:43.062 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map rbd/ns/testrbdnbd21607 2024-09-11T06:50:43.072 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:43.071+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:43.152 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:43.152 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:43.162 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:43.159+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:43.168 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:50:43.201 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV=/dev/nbd1 2024-09-11T06:50:43.201 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd ns 2024-09-11T06:50:43.201 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:50:43.201 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns=ns 2024-09-11T06:50:43.201 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:50:43.201 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\''ns'\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd1'\'']/id' 2024-09-11T06:50:43.262 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:43.259+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:43.269 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.1 to 1 (10000) 2024-09-11T06:50:43.275 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=23353 2024-09-11T06:50:43.275 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 23353 2024-09-11T06:50:43.275 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 23353 -C rbd-nbd 2024-09-11T06:50:43.283 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:50:43.283 INFO:tasks.workunit.client.0.smithi176.stdout: 23307 ? 00:00:00 rbd-nbd 2024-09-11T06:50:43.283 INFO:tasks.workunit.client.0.smithi176.stdout: 23353 ? 00:00:00 rbd-nbd 2024-09-11T06:50:43.284 INFO:tasks.workunit.client.0.smithi176.stderr:+ unmap_device rbd/ns/testrbdnbd21607 23353 2024-09-11T06:50:43.284 INFO:tasks.workunit.client.0.smithi176.stderr:+ local args=rbd/ns/testrbdnbd21607 2024-09-11T06:50:43.284 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pid=23353 2024-09-11T06:50:43.284 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device --device-type nbd unmap rbd/ns/testrbdnbd21607 2024-09-11T06:50:43.284 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:43.285 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:43.286 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:43.286 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:50:43.288 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:50:43.288 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:43.288 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap rbd/ns/testrbdnbd21607 2024-09-11T06:50:43.290 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T06:50:43.296 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-11T06:50:43.333 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:43.333 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:43.347 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:50:43.384 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:50:43.384 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^23353\b' 2024-09-11T06:50:43.384 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^23353\b' 2024-09-11T06:50:43.388 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:43.387+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:43.451 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:43.451 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -C rbd-nbd 2024-09-11T06:50:43.452 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^ *23353\b' 2024-09-11T06:50:43.452 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^ *23353\b' 2024-09-11T06:50:43.460 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:43.460 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 0.5 2024-09-11T06:50:43.464 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:43.459+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:43.509 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:50:43.513 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:43.513 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:43.528 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:50:43.576 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:43.575+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:43.644 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:50:43.644 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:50:43.650 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:43.650 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:43.660 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:50:43.660 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:50:43.665 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:50:43.676 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:50:43.677 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:50:43.693 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:50:43.694 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:50:43.708 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:43.707+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:43.712 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:50:43.712 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:50:43.732 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:50:43.732 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:50:43.767 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:43.763+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:43.833 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:43.833 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:43.833 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:43.833 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:50:43.833 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.833 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:50:43.833 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:50:43.833 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 92.602033096, 2024-09-11T06:50:43.834 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010553425999999999, 2024-09-11T06:50:43.834 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:43.834 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:43.834 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:43.834 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:43.834 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:43.834 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:43.834 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:50:43.834 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.834 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:43.834 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.834 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:43.834 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:50:43.834 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.835 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.835 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.835 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:43.835 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:50:43.835 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.835 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.835 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.835 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:43.835 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230274+0000", 2024-09-11T06:50:43.835 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.533e-06 2024-09-11T06:50:43.835 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.835 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.835 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:43.836 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230285+0000", 2024-09-11T06:50:43.836 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0540000000000001e-05 2024-09-11T06:50:43.836 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.836 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.836 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:43.836 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230309+0000", 2024-09-11T06:50:43.836 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4223000000000002e-05 2024-09-11T06:50:43.836 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.836 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.836 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:43.836 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230327+0000", 2024-09-11T06:50:43.836 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:50:43.836 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.836 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.837 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:43.837 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230478+0000", 2024-09-11T06:50:43.837 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000151459 2024-09-11T06:50:43.837 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.837 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.837 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:43.837 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230490+0000", 2024-09-11T06:50:43.837 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.147e-05 2024-09-11T06:50:43.837 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.837 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.837 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:43.837 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231803+0000", 2024-09-11T06:50:43.837 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001313158 2024-09-11T06:50:43.838 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.838 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.838 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:50:43.838 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231817+0000", 2024-09-11T06:50:43.838 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4197e-05 2024-09-11T06:50:43.838 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.838 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.838 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:43.838 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.235945+0000", 2024-09-11T06:50:43.838 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0041282480000000002 2024-09-11T06:50:43.838 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.838 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.838 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:43.838 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236312+0000", 2024-09-11T06:50:43.839 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036652700000000001 2024-09-11T06:50:43.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.839 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.839 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:50:43.839 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236689+0000", 2024-09-11T06:50:43.839 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000377117 2024-09-11T06:50:43.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.839 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.839 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:43.839 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238460+0000", 2024-09-11T06:50:43.839 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001770918 2024-09-11T06:50:43.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.839 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.840 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:43.840 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240627+0000", 2024-09-11T06:50:43.840 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021670779999999998 2024-09-11T06:50:43.840 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.840 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.840 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:43.840 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240751+0000", 2024-09-11T06:50:43.840 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012384900000000001 2024-09-11T06:50:43.840 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.840 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.840 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:43.840 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240821+0000", 2024-09-11T06:50:43.840 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0516e-05 2024-09-11T06:50:43.840 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.841 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:43.841 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.841 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.841 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.841 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-11T06:50:43.841 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:50:43.841 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 91.599862252999998, 2024-09-11T06:50:43.841 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069904770000000001, 2024-09-11T06:50:43.841 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:43.841 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:43.841 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:43.841 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:43.841 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:43.842 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:43.842 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 5 2024-09-11T06:50:43.842 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.842 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:43.842 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.842 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:43.842 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:50:43.842 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.842 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.842 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.842 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:43.842 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:50:43.842 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.843 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.843 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:43.843 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232442+0000", 2024-09-11T06:50:43.843 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.044e-06 2024-09-11T06:50:43.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.843 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.843 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:43.843 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232460+0000", 2024-09-11T06:50:43.843 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7863000000000002e-05 2024-09-11T06:50:43.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.843 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.843 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:43.843 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232492+0000", 2024-09-11T06:50:43.844 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2812999999999999e-05 2024-09-11T06:50:43.844 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.844 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.844 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:43.844 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232510+0000", 2024-09-11T06:50:43.844 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7552999999999999e-05 2024-09-11T06:50:43.844 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.844 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.844 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:43.844 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232640+0000", 2024-09-11T06:50:43.844 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013029900000000001 2024-09-11T06:50:43.844 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.844 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.845 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:43.845 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232650+0000", 2024-09-11T06:50:43.845 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0023999999999999e-05 2024-09-11T06:50:43.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.845 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.845 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:43.845 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235641+0000", 2024-09-11T06:50:43.845 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00299089 2024-09-11T06:50:43.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.845 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.845 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:43.845 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235980+0000", 2024-09-11T06:50:43.845 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000338907 2024-09-11T06:50:43.846 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.846 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.846 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:43.846 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.236351+0000", 2024-09-11T06:50:43.846 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037087699999999998 2024-09-11T06:50:43.846 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.846 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.846 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:43.846 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238074+0000", 2024-09-11T06:50:43.846 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001722859 2024-09-11T06:50:43.846 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.846 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.847 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:43.847 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239160+0000", 2024-09-11T06:50:43.847 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001086074 2024-09-11T06:50:43.847 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.847 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.847 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:43.847 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239299+0000", 2024-09-11T06:50:43.847 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013933500000000001 2024-09-11T06:50:43.847 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.847 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.847 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:43.847 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239429+0000", 2024-09-11T06:50:43.847 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129939 2024-09-11T06:50:43.847 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.848 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:43.848 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.848 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.848 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.848 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-11T06:50:43.848 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:50:43.848 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 90.585774301000001, 2024-09-11T06:50:43.848 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069253250000000004, 2024-09-11T06:50:43.848 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:43.848 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:43.848 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:43.848 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:43.848 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:43.849 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:43.849 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 7 2024-09-11T06:50:43.849 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.849 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:43.849 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.849 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:43.849 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:50:43.849 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.849 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.849 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.849 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:43.849 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:50:43.849 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.850 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.850 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.850 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:43.850 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246530+0000", 2024-09-11T06:50:43.850 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9469999999999999e-06 2024-09-11T06:50:43.850 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.850 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.850 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:43.850 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246545+0000", 2024-09-11T06:50:43.850 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5973000000000001e-05 2024-09-11T06:50:43.850 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.850 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.850 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:43.851 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246578+0000", 2024-09-11T06:50:43.851 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2759999999999998e-05 2024-09-11T06:50:43.851 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.851 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.851 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:43.851 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246595+0000", 2024-09-11T06:50:43.851 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6939000000000002e-05 2024-09-11T06:50:43.851 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.851 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.851 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:43.851 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246770+0000", 2024-09-11T06:50:43.851 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000174723 2024-09-11T06:50:43.851 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.851 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.852 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:43.852 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246789+0000", 2024-09-11T06:50:43.852 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9159000000000001e-05 2024-09-11T06:50:43.852 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.852 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.852 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:43.852 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250337+0000", 2024-09-11T06:50:43.852 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035480759999999998 2024-09-11T06:50:43.852 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.852 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.852 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:43.852 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250549+0000", 2024-09-11T06:50:43.852 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021192199999999999 2024-09-11T06:50:43.853 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.853 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.853 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:43.853 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250750+0000", 2024-09-11T06:50:43.853 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020104199999999999 2024-09-11T06:50:43.853 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.853 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.853 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:43.853 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.252455+0000", 2024-09-11T06:50:43.853 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001705249 2024-09-11T06:50:43.853 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.853 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.853 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:43.854 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253305+0000", 2024-09-11T06:50:43.854 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00084941200000000002 2024-09-11T06:50:43.854 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.854 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.854 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:43.854 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253388+0000", 2024-09-11T06:50:43.854 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3642999999999999e-05 2024-09-11T06:50:43.854 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.854 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.854 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:43.854 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253452+0000", 2024-09-11T06:50:43.854 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3479999999999995e-05 2024-09-11T06:50:43.854 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.855 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:43.855 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.855 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.855 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.855 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(client.4558.0:5 3.2 e57/55 3:5cea7035:::rbd_info:head v 57'1, mlcod=0'0)", 2024-09-11T06:50:43.855 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:50:43.855 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 25.737669939, 2024-09-11T06:50:43.855 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011016419999999999, 2024-09-11T06:50:43.855 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:43.855 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:43.855 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:43.855 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:43.855 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.855 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:43.856 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:50:43.856 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.856 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.856 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.856 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:43.856 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:50:43.856 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.856 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.856 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.856 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:43.856 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:50:43.856 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4000000000000002e-07 2024-09-11T06:50:43.856 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.857 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.857 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:43.857 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094640+0000", 2024-09-11T06:50:43.857 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6230000000000003e-06 2024-09-11T06:50:43.857 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.857 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.857 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:43.857 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094669+0000", 2024-09-11T06:50:43.857 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8462e-05 2024-09-11T06:50:43.857 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.857 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.857 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:43.857 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094684+0000", 2024-09-11T06:50:43.857 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5536000000000002e-05 2024-09-11T06:50:43.858 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.858 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.858 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:43.858 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094798+0000", 2024-09-11T06:50:43.858 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011439599999999999 2024-09-11T06:50:43.858 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.858 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.858 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:43.858 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094819+0000", 2024-09-11T06:50:43.858 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0296e-05 2024-09-11T06:50:43.858 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.858 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.858 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:43.858 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.095687+0000", 2024-09-11T06:50:43.859 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086851799999999998 2024-09-11T06:50:43.859 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.859 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.859 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:43.859 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.095733+0000", 2024-09-11T06:50:43.859 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5271000000000001e-05 2024-09-11T06:50:43.859 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.859 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:43.859 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.859 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.859 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.859 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:160 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 4096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:43.859 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:50:43.859 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 22.798554932000002, 2024-09-11T06:50:43.860 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001122215, 2024-09-11T06:50:43.860 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:43.860 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:43.860 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:43.860 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:43.860 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:43.860 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:43.860 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 160 2024-09-11T06:50:43.860 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.860 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:43.860 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.860 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:43.860 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:50:43.861 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.861 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.861 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.861 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:43.861 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:50:43.861 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.861 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.861 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.861 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:43.861 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:50:43.861 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0999999999999999e-07 2024-09-11T06:50:43.861 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.861 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.862 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:43.862 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033749+0000", 2024-09-11T06:50:43.862 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8930000000000001e-06 2024-09-11T06:50:43.862 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.862 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.862 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:43.862 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033758+0000", 2024-09-11T06:50:43.862 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.242e-06 2024-09-11T06:50:43.862 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.862 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.862 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:43.862 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033764+0000", 2024-09-11T06:50:43.862 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0510000000000002e-06 2024-09-11T06:50:43.862 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.863 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.863 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:43.863 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.034207+0000", 2024-09-11T06:50:43.863 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00044320600000000003 2024-09-11T06:50:43.863 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.863 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.863 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:43.863 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.034280+0000", 2024-09-11T06:50:43.863 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3094000000000002e-05 2024-09-11T06:50:43.863 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.863 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.863 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:43.863 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.034868+0000", 2024-09-11T06:50:43.863 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00058821899999999996 2024-09-11T06:50:43.864 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.864 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:43.864 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.864 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.864 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.864 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:162 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:43.864 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.039519+0000", 2024-09-11T06:50:43.864 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 22.792781814000001, 2024-09-11T06:50:43.864 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001298485, 2024-09-11T06:50:43.864 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:43.864 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:43.864 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:43.864 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:43.865 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:43.865 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:43.865 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 162 2024-09-11T06:50:43.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.865 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:43.865 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.865 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:43.865 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039519+0000", 2024-09-11T06:50:43.865 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.865 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.865 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:43.865 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039519+0000", 2024-09-11T06:50:43.866 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.866 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:43.866 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039520+0000", 2024-09-11T06:50:43.866 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9299999999999998e-07 2024-09-11T06:50:43.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.866 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:43.866 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039523+0000", 2024-09-11T06:50:43.866 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.247e-06 2024-09-11T06:50:43.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.866 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:43.867 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039529+0000", 2024-09-11T06:50:43.867 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8300000000000001e-06 2024-09-11T06:50:43.867 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.867 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.867 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:43.867 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039533+0000", 2024-09-11T06:50:43.867 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.2379999999999999e-06 2024-09-11T06:50:43.867 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.867 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.867 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:43.867 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.040236+0000", 2024-09-11T06:50:43.867 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00070326500000000001 2024-09-11T06:50:43.867 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.867 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.868 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:43.868 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.040299+0000", 2024-09-11T06:50:43.868 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3077999999999995e-05 2024-09-11T06:50:43.868 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.868 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.868 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:43.868 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.040818+0000", 2024-09-11T06:50:43.868 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00051833399999999996 2024-09-11T06:50:43.868 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.868 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:43.868 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.868 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.868 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.869 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:164 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 528384~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:43.869 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.041536+0000", 2024-09-11T06:50:43.869 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 22.790765092000001, 2024-09-11T06:50:43.869 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00119962, 2024-09-11T06:50:43.869 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:43.869 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:43.869 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:43.869 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:43.869 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:43.869 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:43.869 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 164 2024-09-11T06:50:43.869 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.869 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:43.869 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.870 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:43.870 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041536+0000", 2024-09-11T06:50:43.870 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.870 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.870 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.870 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:43.870 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041536+0000", 2024-09-11T06:50:43.870 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.870 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.870 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.870 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:43.870 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041537+0000", 2024-09-11T06:50:43.870 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3e-06 2024-09-11T06:50:43.870 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.871 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.871 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:43.871 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041543+0000", 2024-09-11T06:50:43.871 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2360000000000001e-06 2024-09-11T06:50:43.871 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.871 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.871 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:43.871 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041558+0000", 2024-09-11T06:50:43.871 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4909000000000001e-05 2024-09-11T06:50:43.871 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.871 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.871 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:43.871 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041569+0000", 2024-09-11T06:50:43.871 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0946e-05 2024-09-11T06:50:43.872 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.872 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.872 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:43.872 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.042138+0000", 2024-09-11T06:50:43.872 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000568996 2024-09-11T06:50:43.872 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.872 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.872 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:43.872 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.042190+0000", 2024-09-11T06:50:43.873 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1609e-05 2024-09-11T06:50:43.873 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.873 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.873 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:43.873 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.042735+0000", 2024-09-11T06:50:43.873 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00054562399999999998 2024-09-11T06:50:43.873 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.873 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:43.873 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.873 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.873 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.873 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:166 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 790528~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:43.873 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.043564+0000", 2024-09-11T06:50:43.873 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 22.788736695000001, 2024-09-11T06:50:43.874 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011042000000000001, 2024-09-11T06:50:43.874 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:43.874 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:43.874 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:43.874 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:43.874 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:43.874 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:43.874 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 166 2024-09-11T06:50:43.874 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.874 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:43.874 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.874 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:43.874 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043564+0000", 2024-09-11T06:50:43.875 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.875 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.875 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.875 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:43.875 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043564+0000", 2024-09-11T06:50:43.875 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.875 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.875 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.875 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:43.875 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043565+0000", 2024-09-11T06:50:43.875 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0449999999999999e-06 2024-09-11T06:50:43.875 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.875 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.875 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:43.876 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043571+0000", 2024-09-11T06:50:43.876 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.6849999999999998e-06 2024-09-11T06:50:43.876 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.876 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.876 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:43.876 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043586+0000", 2024-09-11T06:50:43.876 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4902000000000001e-05 2024-09-11T06:50:43.876 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.876 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.876 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:43.876 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043593+0000", 2024-09-11T06:50:43.876 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9240000000000001e-06 2024-09-11T06:50:43.876 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.876 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.877 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:43.877 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.044073+0000", 2024-09-11T06:50:43.877 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000479813 2024-09-11T06:50:43.877 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.877 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.877 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:43.877 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.044123+0000", 2024-09-11T06:50:43.877 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0908000000000001e-05 2024-09-11T06:50:43.877 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.877 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.877 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:43.877 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.044668+0000", 2024-09-11T06:50:43.878 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000544923 2024-09-11T06:50:43.878 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.878 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:43.878 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.878 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.878 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.878 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:178 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 2363392~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:43.878 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.061599+0000", 2024-09-11T06:50:43.878 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 22.770701979999998, 2024-09-11T06:50:43.878 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012194529999999999, 2024-09-11T06:50:43.878 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:43.878 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:43.878 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:43.879 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:43.879 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:43.879 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:43.879 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 178 2024-09-11T06:50:43.879 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.879 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:43.879 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.879 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:43.879 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061599+0000", 2024-09-11T06:50:43.879 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.879 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.879 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.879 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:43.879 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061599+0000", 2024-09-11T06:50:43.880 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.880 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.880 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.880 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:43.880 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061600+0000", 2024-09-11T06:50:43.880 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3600000000000003e-07 2024-09-11T06:50:43.880 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.880 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.880 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:43.880 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061603+0000", 2024-09-11T06:50:43.880 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.6550000000000002e-06 2024-09-11T06:50:43.880 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.880 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.881 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:43.881 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061618+0000", 2024-09-11T06:50:43.881 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4377e-05 2024-09-11T06:50:43.881 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.881 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.881 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:43.881 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061626+0000", 2024-09-11T06:50:43.881 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.0679999999999997e-06 2024-09-11T06:50:43.881 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.881 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.881 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:43.881 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.062098+0000", 2024-09-11T06:50:43.881 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00047251299999999998 2024-09-11T06:50:43.882 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.882 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.882 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:43.882 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.062253+0000", 2024-09-11T06:50:43.882 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015533100000000001 2024-09-11T06:50:43.882 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.882 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.882 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:43.882 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.062818+0000", 2024-09-11T06:50:43.882 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00056487300000000002 2024-09-11T06:50:43.882 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.882 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:43.882 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.882 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.883 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.883 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:184 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 3149824~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:43.883 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:50:43.883 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 22.766092276999998, 2024-09-11T06:50:43.883 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010685969999999999, 2024-09-11T06:50:43.883 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:43.883 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:43.883 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:43.883 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:43.883 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:43.883 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:43.883 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 184 2024-09-11T06:50:43.883 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.884 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:43.884 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.884 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:43.884 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:50:43.884 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.884 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.884 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.884 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:43.884 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:50:43.884 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.884 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.884 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.884 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:43.884 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:50:43.885 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.6700000000000003e-07 2024-09-11T06:50:43.885 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.885 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.885 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:43.885 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066212+0000", 2024-09-11T06:50:43.885 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.1669999999999999e-06 2024-09-11T06:50:43.885 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.885 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.885 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:43.885 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066219+0000", 2024-09-11T06:50:43.885 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5699999999999998e-06 2024-09-11T06:50:43.885 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.885 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.886 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:43.886 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066223+0000", 2024-09-11T06:50:43.886 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3540000000000002e-06 2024-09-11T06:50:43.886 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.886 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.886 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:43.886 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066713+0000", 2024-09-11T06:50:43.886 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00048985399999999996 2024-09-11T06:50:43.886 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.886 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.886 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:43.886 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066813+0000", 2024-09-11T06:50:43.886 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00010003 2024-09-11T06:50:43.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.887 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.887 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:43.887 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.067277+0000", 2024-09-11T06:50:43.887 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000463955 2024-09-11T06:50:43.887 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.887 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:43.887 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.887 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.887 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:186 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 3411968~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:43.887 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.071198+0000", 2024-09-11T06:50:43.887 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 22.761102933, 2024-09-11T06:50:43.887 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012515779999999999, 2024-09-11T06:50:43.887 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:43.888 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:43.888 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:43.888 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:43.888 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:43.888 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:43.888 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 186 2024-09-11T06:50:43.888 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.888 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:43.888 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.888 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:43.888 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071198+0000", 2024-09-11T06:50:43.888 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.888 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.889 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.889 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:43.889 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071198+0000", 2024-09-11T06:50:43.889 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.889 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.889 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.889 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:43.889 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071199+0000", 2024-09-11T06:50:43.889 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0299999999999999e-07 2024-09-11T06:50:43.889 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.889 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.889 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:43.889 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071201+0000", 2024-09-11T06:50:43.890 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8540000000000001e-06 2024-09-11T06:50:43.890 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.890 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.890 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:43.890 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071209+0000", 2024-09-11T06:50:43.890 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3629999999999996e-06 2024-09-11T06:50:43.890 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.890 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.890 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:43.890 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071214+0000", 2024-09-11T06:50:43.890 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2109999999999997e-06 2024-09-11T06:50:43.890 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.890 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.890 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:43.891 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071915+0000", 2024-09-11T06:50:43.891 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00070101799999999998 2024-09-11T06:50:43.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.891 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.891 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:43.891 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.072011+0000", 2024-09-11T06:50:43.891 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.5755999999999996e-05 2024-09-11T06:50:43.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.891 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.891 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:43.891 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.072450+0000", 2024-09-11T06:50:43.891 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000438773 2024-09-11T06:50:43.892 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.892 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:43.892 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.892 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.892 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.892 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:190 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 3936256~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:43.892 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.074875+0000", 2024-09-11T06:50:43.892 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 22.757426018, 2024-09-11T06:50:43.892 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0013119620000000001, 2024-09-11T06:50:43.892 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:43.892 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:43.892 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:43.892 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:43.893 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:43.893 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:43.893 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 190 2024-09-11T06:50:43.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.893 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:43.893 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.893 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:43.893 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074875+0000", 2024-09-11T06:50:43.893 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.893 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.893 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:43.893 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074875+0000", 2024-09-11T06:50:43.893 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.894 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.894 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:43.894 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074876+0000", 2024-09-11T06:50:43.894 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7400000000000003e-07 2024-09-11T06:50:43.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.894 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.894 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:43.894 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074887+0000", 2024-09-11T06:50:43.894 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1817999999999999e-05 2024-09-11T06:50:43.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.894 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.894 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:43.894 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074896+0000", 2024-09-11T06:50:43.895 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9120000000000004e-06 2024-09-11T06:50:43.895 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.895 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.895 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:43.895 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074900+0000", 2024-09-11T06:50:43.895 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.2220000000000004e-06 2024-09-11T06:50:43.895 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.895 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.895 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:43.895 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.075198+0000", 2024-09-11T06:50:43.895 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00029750399999999998 2024-09-11T06:50:43.895 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.895 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.896 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:43.896 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.075430+0000", 2024-09-11T06:50:43.896 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023200400000000001 2024-09-11T06:50:43.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.896 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.896 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:43.896 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.076187+0000", 2024-09-11T06:50:43.896 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00075682799999999997 2024-09-11T06:50:43.896 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.896 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:43.896 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.896 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.897 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:359 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 135168~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:43.897 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.298832+0000", 2024-09-11T06:50:43.897 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 22.533468383999999, 2024-09-11T06:50:43.897 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001078575, 2024-09-11T06:50:43.897 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:43.897 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:43.897 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:43.897 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:43.897 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:43.897 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:43.897 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 359 2024-09-11T06:50:43.897 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.897 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:43.898 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.898 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:43.898 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298832+0000", 2024-09-11T06:50:43.898 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.898 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.898 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:43.898 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298832+0000", 2024-09-11T06:50:43.898 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.898 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.898 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:43.898 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298834+0000", 2024-09-11T06:50:43.899 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.646e-06 2024-09-11T06:50:43.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.899 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:43.899 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298845+0000", 2024-09-11T06:50:43.899 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.081e-05 2024-09-11T06:50:43.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.899 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:43.899 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298872+0000", 2024-09-11T06:50:43.899 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7382e-05 2024-09-11T06:50:43.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.899 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:43.900 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298884+0000", 2024-09-11T06:50:43.900 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1956e-05 2024-09-11T06:50:43.900 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.900 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.900 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:43.900 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298959+0000", 2024-09-11T06:50:43.900 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5214000000000002e-05 2024-09-11T06:50:43.900 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.900 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.900 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:43.900 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.299090+0000", 2024-09-11T06:50:43.900 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000130247 2024-09-11T06:50:43.901 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.901 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.901 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:43.901 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.299911+0000", 2024-09-11T06:50:43.901 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00082131999999999995 2024-09-11T06:50:43.901 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.901 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:43.901 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.901 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.901 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.901 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:360 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:43.901 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.298914+0000", 2024-09-11T06:50:43.901 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 22.533386713999999, 2024-09-11T06:50:43.902 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0023571690000000001, 2024-09-11T06:50:43.902 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:43.902 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:43.902 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:43.902 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:43.902 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:43.902 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:43.902 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 360 2024-09-11T06:50:43.902 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.902 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:43.902 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.902 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:43.902 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298914+0000", 2024-09-11T06:50:43.903 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.903 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.903 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:43.903 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298914+0000", 2024-09-11T06:50:43.903 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.903 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.903 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:43.903 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298915+0000", 2024-09-11T06:50:43.903 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9300000000000006e-07 2024-09-11T06:50:43.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.903 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.903 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:43.904 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298921+0000", 2024-09-11T06:50:43.904 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.6030000000000004e-06 2024-09-11T06:50:43.904 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.904 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.904 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:43.904 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298932+0000", 2024-09-11T06:50:43.904 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.082e-05 2024-09-11T06:50:43.904 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.904 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.904 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:43.904 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298939+0000", 2024-09-11T06:50:43.904 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7919999999999993e-06 2024-09-11T06:50:43.904 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.905 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.905 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:43.905 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.300011+0000", 2024-09-11T06:50:43.905 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001071179 2024-09-11T06:50:43.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.905 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.905 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:43.905 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.300245+0000", 2024-09-11T06:50:43.905 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000234434 2024-09-11T06:50:43.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.905 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.905 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:43.905 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.301271+0000", 2024-09-11T06:50:43.906 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010263480000000001 2024-09-11T06:50:43.906 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.906 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:43.906 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.906 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.906 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.906 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:362 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 528384~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:43.906 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.302376+0000", 2024-09-11T06:50:43.906 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 22.529924391000002, 2024-09-11T06:50:43.906 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014132909999999999, 2024-09-11T06:50:43.906 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:43.906 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:43.906 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:43.906 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:43.907 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:43.907 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:43.907 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 362 2024-09-11T06:50:43.907 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.907 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:43.907 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.907 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:43.907 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302376+0000", 2024-09-11T06:50:43.907 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.907 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.907 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.907 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:43.908 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302376+0000", 2024-09-11T06:50:43.908 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.908 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.908 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:43.908 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302378+0000", 2024-09-11T06:50:43.908 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1629999999999999e-06 2024-09-11T06:50:43.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.908 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.908 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:43.908 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302383+0000", 2024-09-11T06:50:43.908 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.463e-06 2024-09-11T06:50:43.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.908 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.909 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:43.909 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302401+0000", 2024-09-11T06:50:43.909 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8219e-05 2024-09-11T06:50:43.909 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.909 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.909 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:43.909 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302411+0000", 2024-09-11T06:50:43.909 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9000000000000001e-06 2024-09-11T06:50:43.909 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.909 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.909 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:43.909 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302524+0000", 2024-09-11T06:50:43.909 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000113199 2024-09-11T06:50:43.910 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.910 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.910 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:43.910 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302745+0000", 2024-09-11T06:50:43.910 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022057399999999999 2024-09-11T06:50:43.910 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.910 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.910 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:43.910 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.303790+0000", 2024-09-11T06:50:43.910 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010447729999999999 2024-09-11T06:50:43.910 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.910 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:43.910 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.910 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.911 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.911 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:363 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 659456~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:43.911 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.302938+0000", 2024-09-11T06:50:43.911 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 22.529362798000001, 2024-09-11T06:50:43.911 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00185012, 2024-09-11T06:50:43.911 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:43.911 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:43.911 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:43.911 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:43.911 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:43.911 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:43.911 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 363 2024-09-11T06:50:43.911 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.912 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:43.912 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.912 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:43.912 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302938+0000", 2024-09-11T06:50:43.912 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.912 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.912 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:43.912 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302938+0000", 2024-09-11T06:50:43.912 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.912 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.912 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:43.913 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302939+0000", 2024-09-11T06:50:43.913 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2300000000000001e-06 2024-09-11T06:50:43.913 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.913 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.913 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:43.913 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302944+0000", 2024-09-11T06:50:43.913 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.493e-06 2024-09-11T06:50:43.913 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.913 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.913 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:43.913 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302960+0000", 2024-09-11T06:50:43.913 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.632e-05 2024-09-11T06:50:43.913 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.913 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.914 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:43.914 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302972+0000", 2024-09-11T06:50:43.914 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1885000000000001e-05 2024-09-11T06:50:43.914 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.914 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.914 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:43.914 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.303835+0000", 2024-09-11T06:50:43.914 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000863141 2024-09-11T06:50:43.914 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.914 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.914 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:43.914 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.303965+0000", 2024-09-11T06:50:43.914 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013032199999999999 2024-09-11T06:50:43.914 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.915 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.915 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:43.915 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.304788+0000", 2024-09-11T06:50:43.915 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00082272900000000002 2024-09-11T06:50:43.915 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.915 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:43.915 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.915 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.915 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.915 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:366 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 1052672~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:43.915 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.306960+0000", 2024-09-11T06:50:43.915 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 22.525340928999999, 2024-09-11T06:50:43.915 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010286749999999999, 2024-09-11T06:50:43.915 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:43.916 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:43.916 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:43.916 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:43.916 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:43.916 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:43.916 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 366 2024-09-11T06:50:43.916 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.916 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:43.916 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.916 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:43.916 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.306960+0000", 2024-09-11T06:50:43.916 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.916 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.916 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.917 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:43.917 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.306960+0000", 2024-09-11T06:50:43.917 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.917 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.917 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.917 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:43.917 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.306961+0000", 2024-09-11T06:50:43.917 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3939999999999999e-06 2024-09-11T06:50:43.917 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.917 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.917 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:43.917 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.306970+0000", 2024-09-11T06:50:43.917 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6859999999999995e-06 2024-09-11T06:50:43.917 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.918 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.918 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:43.918 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.306991+0000", 2024-09-11T06:50:43.918 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0772e-05 2024-09-11T06:50:43.918 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.918 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.918 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:43.918 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.307002+0000", 2024-09-11T06:50:43.918 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0757000000000001e-05 2024-09-11T06:50:43.918 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.918 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.918 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:43.918 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.307077+0000", 2024-09-11T06:50:43.918 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5337000000000001e-05 2024-09-11T06:50:43.919 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.919 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.919 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:43.919 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.307185+0000", 2024-09-11T06:50:43.919 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000108335 2024-09-11T06:50:43.919 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.919 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.919 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:43.919 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.307989+0000", 2024-09-11T06:50:43.919 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00080339400000000003 2024-09-11T06:50:43.919 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.919 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:43.919 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.919 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.920 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.920 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:367 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 1183744~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:43.920 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.312392+0000", 2024-09-11T06:50:43.920 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 22.519909158000001, 2024-09-11T06:50:43.920 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011343460000000001, 2024-09-11T06:50:43.920 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:43.920 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:43.920 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:43.920 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:43.920 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:43.920 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:43.920 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 367 2024-09-11T06:50:43.920 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.921 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:43.921 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.921 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:43.921 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312392+0000", 2024-09-11T06:50:43.921 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.921 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.921 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.921 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:43.921 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312392+0000", 2024-09-11T06:50:43.921 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.921 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.921 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.921 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:43.921 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312393+0000", 2024-09-11T06:50:43.922 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6199999999999999e-06 2024-09-11T06:50:43.922 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.922 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.922 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:43.922 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312400+0000", 2024-09-11T06:50:43.922 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0700000000000001e-06 2024-09-11T06:50:43.922 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.922 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.922 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:43.922 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312420+0000", 2024-09-11T06:50:43.922 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9375999999999999e-05 2024-09-11T06:50:43.922 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.922 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.922 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:43.923 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312430+0000", 2024-09-11T06:50:43.923 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0209e-05 2024-09-11T06:50:43.923 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.923 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.923 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:43.923 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312549+0000", 2024-09-11T06:50:43.923 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000118779 2024-09-11T06:50:43.923 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.923 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.923 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:43.923 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312767+0000", 2024-09-11T06:50:43.923 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021786400000000001 2024-09-11T06:50:43.923 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.924 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.924 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:43.924 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313526+0000", 2024-09-11T06:50:43.924 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00075942800000000003 2024-09-11T06:50:43.924 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.924 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:43.924 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.924 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.924 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.924 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:368 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 1314816~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:43.924 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.312454+0000", 2024-09-11T06:50:43.924 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 22.519846634, 2024-09-11T06:50:43.924 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014704550000000001, 2024-09-11T06:50:43.924 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:43.925 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:43.925 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:43.925 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:43.925 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:43.925 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:43.925 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 368 2024-09-11T06:50:43.925 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.925 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:43.925 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.925 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:43.925 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312454+0000", 2024-09-11T06:50:43.925 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.925 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.926 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.926 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:43.926 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312454+0000", 2024-09-11T06:50:43.926 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.926 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.926 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.926 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:43.926 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312455+0000", 2024-09-11T06:50:43.926 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4399999999999999e-07 2024-09-11T06:50:43.926 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.926 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.926 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:43.926 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312474+0000", 2024-09-11T06:50:43.926 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8899e-05 2024-09-11T06:50:43.927 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.927 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.927 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:43.927 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312485+0000", 2024-09-11T06:50:43.927 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0793e-05 2024-09-11T06:50:43.927 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.927 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.927 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:43.927 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312492+0000", 2024-09-11T06:50:43.927 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.593e-06 2024-09-11T06:50:43.927 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.927 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.927 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:43.927 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313576+0000", 2024-09-11T06:50:43.928 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010837970000000001 2024-09-11T06:50:43.928 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.928 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.928 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:43.928 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313620+0000", 2024-09-11T06:50:43.928 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3640000000000002e-05 2024-09-11T06:50:43.928 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.928 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.928 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:43.928 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313925+0000", 2024-09-11T06:50:43.928 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00030498899999999998 2024-09-11T06:50:43.928 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.928 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:43.928 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.929 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.929 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.929 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:370 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 1576960~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:43.929 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.314817+0000", 2024-09-11T06:50:43.929 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 22.517483925000001, 2024-09-11T06:50:43.929 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011606190000000001, 2024-09-11T06:50:43.929 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:43.929 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:43.929 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:43.929 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:43.929 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:43.929 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:43.929 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 370 2024-09-11T06:50:43.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.930 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:43.930 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.930 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:43.930 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314817+0000", 2024-09-11T06:50:43.930 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.930 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.930 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:43.930 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314817+0000", 2024-09-11T06:50:43.930 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:43.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.930 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.930 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:43.931 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314818+0000", 2024-09-11T06:50:43.931 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.463e-06 2024-09-11T06:50:43.931 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.931 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.931 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:43.931 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314826+0000", 2024-09-11T06:50:43.931 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3860000000000001e-06 2024-09-11T06:50:43.931 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.931 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.931 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:43.931 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314842+0000", 2024-09-11T06:50:43.931 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6220000000000001e-05 2024-09-11T06:50:43.931 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.932 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.932 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:43.932 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314854+0000", 2024-09-11T06:50:43.932 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1562e-05 2024-09-11T06:50:43.932 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.932 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.932 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:43.932 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.315212+0000", 2024-09-11T06:50:43.932 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000358012 2024-09-11T06:50:43.932 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.932 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.932 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:43.932 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.315433+0000", 2024-09-11T06:50:43.932 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022103800000000001 2024-09-11T06:50:43.933 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:43.933 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:43.933 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:43.933 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.315978+0000", 2024-09-11T06:50:43.933 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00054493800000000002 2024-09-11T06:50:43.933 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.933 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:43.933 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.933 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:43.933 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:43.933 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:43.935 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:43.931+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:43.935 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:50:43.962 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV= 2024-09-11T06:50:43.962 INFO:tasks.workunit.client.0.smithi176.stderr:+ unmap_device rbd/testrbdnbd21607 23307 2024-09-11T06:50:43.962 INFO:tasks.workunit.client.0.smithi176.stderr:+ local args=rbd/testrbdnbd21607 2024-09-11T06:50:43.962 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pid=23307 2024-09-11T06:50:43.962 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device --device-type nbd unmap rbd/testrbdnbd21607 2024-09-11T06:50:43.962 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:43.962 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:43.963 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:43.964 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:50:43.966 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:50:43.966 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:43.966 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap rbd/testrbdnbd21607 2024-09-11T06:50:44.036 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:44.035+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:44.059 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:50:44.059 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^23307\b' 2024-09-11T06:50:44.059 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^23307\b' 2024-09-11T06:50:44.062 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:44.063 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:44.063 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:50:44.063 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:44.077 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:50:44.116 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:44.117 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -C rbd-nbd 2024-09-11T06:50:44.117 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^ *23307\b' 2024-09-11T06:50:44.117 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^ *23307\b' 2024-09-11T06:50:44.123 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:44.123 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 0.5 2024-09-11T06:50:44.137 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:44.135+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:44.196 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:44.196 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:44.196 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:50:44.196 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:50:44.196 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:44.210 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:50:44.257 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:44.255+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:44.290 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is set 2024-09-11T06:50:44.290 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.3 to 1 (10000) 2024-09-11T06:50:44.311 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-11T06:50:44.311 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:50:44.335 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:44.335 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:50:44.336 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:50:44.336 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:50:44.336 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.336 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:54 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'32, mlcod=13'29)", 2024-09-11T06:50:44.336 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:50:44.336 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 98.123347942999999, 2024-09-11T06:50:44.336 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034271670000000001, 2024-09-11T06:50:44.336 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.336 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.336 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.336 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.336 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.336 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.337 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:50:44.337 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.337 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.337 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.337 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:50:44.337 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.337 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.337 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.337 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210361+0000", 2024-09-11T06:50:44.337 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1260000000000001e-06 2024-09-11T06:50:44.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.338 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.338 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210370+0000", 2024-09-11T06:50:44.338 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8200000000000003e-06 2024-09-11T06:50:44.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.338 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.338 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210390+0000", 2024-09-11T06:50:44.338 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0126e-05 2024-09-11T06:50:44.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.338 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.338 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210404+0000", 2024-09-11T06:50:44.338 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4589999999999999e-05 2024-09-11T06:50:44.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.339 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.339 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210538+0000", 2024-09-11T06:50:44.339 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000133719 2024-09-11T06:50:44.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.339 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.339 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210593+0000", 2024-09-11T06:50:44.339 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5016999999999999e-05 2024-09-11T06:50:44.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.339 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.339 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213695+0000", 2024-09-11T06:50:44.339 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003101866 2024-09-11T06:50:44.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.340 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.340 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.340 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213787+0000", 2024-09-11T06:50:44.340 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1903000000000004e-05 2024-09-11T06:50:44.340 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.340 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.340 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.340 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.340 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.340 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:55 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'33, mlcod=13'29)", 2024-09-11T06:50:44.340 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:50:44.340 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 98.122328705000001, 2024-09-11T06:50:44.340 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.005086091, 2024-09-11T06:50:44.341 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.341 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.341 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.341 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.341 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.341 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.341 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:50:44.341 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.341 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.341 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.341 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:50:44.341 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.342 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.342 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.342 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.342 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211380+0000", 2024-09-11T06:50:44.342 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.097e-06 2024-09-11T06:50:44.342 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.342 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.342 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.342 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211537+0000", 2024-09-11T06:50:44.342 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000157369 2024-09-11T06:50:44.342 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.342 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.342 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.342 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211559+0000", 2024-09-11T06:50:44.343 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000001e-05 2024-09-11T06:50:44.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.343 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.343 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.343 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211573+0000", 2024-09-11T06:50:44.343 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4946000000000001e-05 2024-09-11T06:50:44.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.343 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.343 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.343 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211676+0000", 2024-09-11T06:50:44.343 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0001026 2024-09-11T06:50:44.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.343 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.343 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.344 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211709+0000", 2024-09-11T06:50:44.344 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3071999999999998e-05 2024-09-11T06:50:44.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.344 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.344 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.344 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216379+0000", 2024-09-11T06:50:44.344 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0046693780000000001 2024-09-11T06:50:44.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.344 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.344 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.344 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216465+0000", 2024-09-11T06:50:44.344 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6469000000000002e-05 2024-09-11T06:50:44.344 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.345 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.345 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.345 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.345 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:56 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'34, mlcod=13'29)", 2024-09-11T06:50:44.345 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:50:44.345 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 98.120894845999999, 2024-09-11T06:50:44.345 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0060310629999999997, 2024-09-11T06:50:44.345 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.345 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.345 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.345 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.345 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.346 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:50:44.346 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.346 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.346 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.346 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:50:44.346 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.346 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.346 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.346 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212814+0000", 2024-09-11T06:50:44.346 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0869999999999999e-06 2024-09-11T06:50:44.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.346 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.347 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.347 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212822+0000", 2024-09-11T06:50:44.347 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0870000000000007e-06 2024-09-11T06:50:44.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.347 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.347 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.347 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212840+0000", 2024-09-11T06:50:44.347 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:50:44.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.347 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.347 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.347 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212854+0000", 2024-09-11T06:50:44.348 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4003e-05 2024-09-11T06:50:44.348 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.348 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.348 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.348 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213026+0000", 2024-09-11T06:50:44.348 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017207499999999999 2024-09-11T06:50:44.348 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.348 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.348 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.348 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213063+0000", 2024-09-11T06:50:44.349 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7187000000000002e-05 2024-09-11T06:50:44.349 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.349 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.349 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218762+0000", 2024-09-11T06:50:44.349 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0056993290000000004 2024-09-11T06:50:44.349 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.349 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.349 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218844+0000", 2024-09-11T06:50:44.349 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1701999999999998e-05 2024-09-11T06:50:44.349 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.349 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.349 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.350 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:60 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'38, mlcod=13'36)", 2024-09-11T06:50:44.350 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:50:44.351 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 98.106953497000006, 2024-09-11T06:50:44.352 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039254270000000004, 2024-09-11T06:50:44.352 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.352 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.352 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.352 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.352 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.352 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.352 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:50:44.352 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.352 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.352 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.352 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.353 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:50:44.353 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.353 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.353 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.354 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226756+0000", 2024-09-11T06:50:44.354 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-11T06:50:44.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.354 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.354 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.354 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226768+0000", 2024-09-11T06:50:44.354 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1919999999999999e-05 2024-09-11T06:50:44.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.354 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.354 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.354 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226797+0000", 2024-09-11T06:50:44.354 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9573e-05 2024-09-11T06:50:44.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.355 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.355 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226818+0000", 2024-09-11T06:50:44.355 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0432999999999999e-05 2024-09-11T06:50:44.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.355 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.355 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226909+0000", 2024-09-11T06:50:44.355 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0920000000000004e-05 2024-09-11T06:50:44.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.356 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.356 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226936+0000", 2024-09-11T06:50:44.356 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7472999999999999e-05 2024-09-11T06:50:44.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.356 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.356 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230596+0000", 2024-09-11T06:50:44.356 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0036602520000000001 2024-09-11T06:50:44.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.357 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.357 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230680+0000", 2024-09-11T06:50:44.357 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3106e-05 2024-09-11T06:50:44.357 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.357 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.357 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.357 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:107 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'69, mlcod=13'64)", 2024-09-11T06:50:44.357 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:50:44.358 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 97.890506040000005, 2024-09-11T06:50:44.358 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040728830000000002, 2024-09-11T06:50:44.358 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.358 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.358 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.358 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.358 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.358 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:50:44.358 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.358 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:50:44.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.359 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443203+0000", 2024-09-11T06:50:44.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1480000000000001e-06 2024-09-11T06:50:44.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.359 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443222+0000", 2024-09-11T06:50:44.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9040000000000001e-05 2024-09-11T06:50:44.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.360 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.360 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443246+0000", 2024-09-11T06:50:44.360 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4606000000000001e-05 2024-09-11T06:50:44.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.360 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.360 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443277+0000", 2024-09-11T06:50:44.360 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0207999999999999e-05 2024-09-11T06:50:44.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.360 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.360 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443340+0000", 2024-09-11T06:50:44.360 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3602000000000005e-05 2024-09-11T06:50:44.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.361 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.361 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443363+0000", 2024-09-11T06:50:44.361 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2331e-05 2024-09-11T06:50:44.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.361 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.361 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447216+0000", 2024-09-11T06:50:44.361 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0038539690000000001 2024-09-11T06:50:44.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.361 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.361 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447274+0000", 2024-09-11T06:50:44.361 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7979000000000002e-05 2024-09-11T06:50:44.362 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.362 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.362 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.362 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.362 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.362 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:111 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'73, mlcod=13'67)", 2024-09-11T06:50:44.362 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:50:44.362 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 97.88399441, 2024-09-11T06:50:44.362 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034372360000000002, 2024-09-11T06:50:44.362 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.362 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.363 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.363 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.363 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.363 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.363 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:50:44.363 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.363 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.363 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.363 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:50:44.363 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.364 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.364 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449714+0000", 2024-09-11T06:50:44.364 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5000000000000001e-07 2024-09-11T06:50:44.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.364 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.364 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449721+0000", 2024-09-11T06:50:44.364 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8530000000000003e-06 2024-09-11T06:50:44.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.364 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.364 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449736+0000", 2024-09-11T06:50:44.364 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5114e-05 2024-09-11T06:50:44.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.365 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.365 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449743+0000", 2024-09-11T06:50:44.365 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0210000000000002e-06 2024-09-11T06:50:44.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.365 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.365 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449804+0000", 2024-09-11T06:50:44.365 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0479999999999997e-05 2024-09-11T06:50:44.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.366 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.366 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449826+0000", 2024-09-11T06:50:44.366 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1922000000000001e-05 2024-09-11T06:50:44.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.366 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.366 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453104+0000", 2024-09-11T06:50:44.366 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003278652 2024-09-11T06:50:44.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.366 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.367 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453150+0000", 2024-09-11T06:50:44.367 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6244e-05 2024-09-11T06:50:44.367 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.367 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.367 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.367 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.367 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:114 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'76, mlcod=13'71)", 2024-09-11T06:50:44.367 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:50:44.367 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 97.879316338999999, 2024-09-11T06:50:44.367 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0042576660000000002, 2024-09-11T06:50:44.367 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.367 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.368 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.368 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.368 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.368 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.368 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:50:44.368 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.368 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.368 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.368 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:50:44.368 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.369 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.369 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454392+0000", 2024-09-11T06:50:44.369 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5999999999999991e-07 2024-09-11T06:50:44.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.369 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.369 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454399+0000", 2024-09-11T06:50:44.369 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.878e-06 2024-09-11T06:50:44.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.369 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.370 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454415+0000", 2024-09-11T06:50:44.370 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5659999999999999e-05 2024-09-11T06:50:44.370 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.370 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.370 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454422+0000", 2024-09-11T06:50:44.370 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:50:44.370 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.370 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.370 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454474+0000", 2024-09-11T06:50:44.370 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1860999999999997e-05 2024-09-11T06:50:44.370 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.371 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.371 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454497+0000", 2024-09-11T06:50:44.371 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3686e-05 2024-09-11T06:50:44.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.371 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.371 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458552+0000", 2024-09-11T06:50:44.371 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040544550000000002 2024-09-11T06:50:44.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.371 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.371 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458649+0000", 2024-09-11T06:50:44.372 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7005999999999999e-05 2024-09-11T06:50:44.372 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.372 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.372 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.372 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:115 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'77, mlcod=13'72)", 2024-09-11T06:50:44.372 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:44.372 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 97.877975727000006, 2024-09-11T06:50:44.372 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003355584, 2024-09-11T06:50:44.372 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.372 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.372 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.373 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:44.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:44.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:44.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5999999999999999e-07 2024-09-11T06:50:44.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.374 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455803+0000", 2024-09-11T06:50:44.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0310999999999998e-05 2024-09-11T06:50:44.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.374 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455815+0000", 2024-09-11T06:50:44.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2787e-05 2024-09-11T06:50:44.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.375 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455823+0000", 2024-09-11T06:50:44.375 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5959999999999998e-06 2024-09-11T06:50:44.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.375 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.375 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455875+0000", 2024-09-11T06:50:44.375 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2054e-05 2024-09-11T06:50:44.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.375 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.375 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455896+0000", 2024-09-11T06:50:44.375 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0554999999999999e-05 2024-09-11T06:50:44.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.376 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.376 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459000+0000", 2024-09-11T06:50:44.376 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003104842 2024-09-11T06:50:44.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.376 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.376 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459087+0000", 2024-09-11T06:50:44.376 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6978999999999995e-05 2024-09-11T06:50:44.376 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.376 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.376 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.377 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:120 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'82, mlcod=13'78)", 2024-09-11T06:50:44.377 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:50:44.377 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 97.867298442000006, 2024-09-11T06:50:44.377 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0064612549999999999, 2024-09-11T06:50:44.377 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.377 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.377 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.377 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.377 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.377 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.377 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:50:44.377 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.377 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.378 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:50:44.378 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.378 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466410+0000", 2024-09-11T06:50:44.378 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0389999999999999e-06 2024-09-11T06:50:44.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.378 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466496+0000", 2024-09-11T06:50:44.378 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5519000000000006e-05 2024-09-11T06:50:44.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.379 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.379 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466510+0000", 2024-09-11T06:50:44.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3842999999999999e-05 2024-09-11T06:50:44.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.379 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.379 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466517+0000", 2024-09-11T06:50:44.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5229999999999998e-06 2024-09-11T06:50:44.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.379 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.379 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466595+0000", 2024-09-11T06:50:44.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8019e-05 2024-09-11T06:50:44.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.380 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466612+0000", 2024-09-11T06:50:44.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7013000000000001e-05 2024-09-11T06:50:44.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.380 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472787+0000", 2024-09-11T06:50:44.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0061750269999999996 2024-09-11T06:50:44.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.381 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472870+0000", 2024-09-11T06:50:44.381 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3271999999999998e-05 2024-09-11T06:50:44.381 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.381 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.381 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.381 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:121 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'83, mlcod=13'78)", 2024-09-11T06:50:44.381 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:50:44.382 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 97.866073646000004, 2024-09-11T06:50:44.382 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055613069999999997, 2024-09-11T06:50:44.382 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.382 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.382 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.382 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.382 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.382 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:50:44.382 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.382 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.382 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:50:44.383 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.383 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.383 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467635+0000", 2024-09-11T06:50:44.383 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6899999999999996e-07 2024-09-11T06:50:44.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.383 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.383 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467727+0000", 2024-09-11T06:50:44.384 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1800999999999997e-05 2024-09-11T06:50:44.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.384 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.384 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467740+0000", 2024-09-11T06:50:44.384 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3747999999999999e-05 2024-09-11T06:50:44.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.384 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.384 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467753+0000", 2024-09-11T06:50:44.384 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2807e-05 2024-09-11T06:50:44.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.385 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.385 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467812+0000", 2024-09-11T06:50:44.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8381999999999998e-05 2024-09-11T06:50:44.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.385 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.385 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467828+0000", 2024-09-11T06:50:44.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6728e-05 2024-09-11T06:50:44.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.385 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.385 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473112+0000", 2024-09-11T06:50:44.386 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052833619999999998 2024-09-11T06:50:44.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.386 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.386 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473195+0000", 2024-09-11T06:50:44.386 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3609999999999994e-05 2024-09-11T06:50:44.386 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.386 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.386 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.387 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:131 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'89, mlcod=13'85)", 2024-09-11T06:50:44.387 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:50:44.387 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 97.514667931000005, 2024-09-11T06:50:44.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052492260000000001, 2024-09-11T06:50:44.387 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.387 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.387 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.387 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.387 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.387 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:50:44.388 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.388 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.388 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:50:44.388 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.388 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.388 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819041+0000", 2024-09-11T06:50:44.389 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4360000000000001e-06 2024-09-11T06:50:44.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.389 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.389 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819100+0000", 2024-09-11T06:50:44.389 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9225999999999998e-05 2024-09-11T06:50:44.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.389 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.389 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819128+0000", 2024-09-11T06:50:44.390 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.779e-05 2024-09-11T06:50:44.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.390 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.390 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819142+0000", 2024-09-11T06:50:44.390 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3573e-05 2024-09-11T06:50:44.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.390 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.390 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819263+0000", 2024-09-11T06:50:44.390 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000121086 2024-09-11T06:50:44.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.391 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.391 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819292+0000", 2024-09-11T06:50:44.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9363e-05 2024-09-11T06:50:44.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.391 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.391 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824185+0000", 2024-09-11T06:50:44.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0048924620000000002 2024-09-11T06:50:44.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.391 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.391 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824289+0000", 2024-09-11T06:50:44.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010429000000000001 2024-09-11T06:50:44.392 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.392 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.392 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.392 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:132 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'90, mlcod=13'85)", 2024-09-11T06:50:44.392 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:50:44.392 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 97.513893241000005, 2024-09-11T06:50:44.392 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.004532342, 2024-09-11T06:50:44.392 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.392 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.392 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.392 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.393 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.393 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:50:44.393 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.393 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.393 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:50:44.393 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.393 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.393 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819816+0000", 2024-09-11T06:50:44.393 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.61e-06 2024-09-11T06:50:44.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.394 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.394 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819827+0000", 2024-09-11T06:50:44.394 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1e-05 2024-09-11T06:50:44.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.394 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.394 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819851+0000", 2024-09-11T06:50:44.394 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4267e-05 2024-09-11T06:50:44.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.394 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.394 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819864+0000", 2024-09-11T06:50:44.394 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2767e-05 2024-09-11T06:50:44.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820071+0000", 2024-09-11T06:50:44.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000207061 2024-09-11T06:50:44.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820103+0000", 2024-09-11T06:50:44.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2326999999999997e-05 2024-09-11T06:50:44.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824302+0000", 2024-09-11T06:50:44.396 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0041986000000000002 2024-09-11T06:50:44.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.396 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.396 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824347+0000", 2024-09-11T06:50:44.396 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4709999999999997e-05 2024-09-11T06:50:44.396 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.396 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.396 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.396 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:133 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'91, mlcod=13'86)", 2024-09-11T06:50:44.396 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:50:44.397 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 97.513261581999998, 2024-09-11T06:50:44.397 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039421459999999997, 2024-09-11T06:50:44.397 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.397 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.397 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.397 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.397 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.397 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.397 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:50:44.397 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.398 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.398 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.398 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:50:44.398 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.398 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.398 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.398 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.398 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820448+0000", 2024-09-11T06:50:44.398 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.84e-06 2024-09-11T06:50:44.398 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.398 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.398 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.398 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820463+0000", 2024-09-11T06:50:44.398 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5356e-05 2024-09-11T06:50:44.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.399 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.399 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820499+0000", 2024-09-11T06:50:44.399 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.625e-05 2024-09-11T06:50:44.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.399 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.399 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820546+0000", 2024-09-11T06:50:44.399 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7017000000000001e-05 2024-09-11T06:50:44.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.400 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.400 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820882+0000", 2024-09-11T06:50:44.400 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033512 2024-09-11T06:50:44.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.400 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.400 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.400 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820941+0000", 2024-09-11T06:50:44.400 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8922999999999998e-05 2024-09-11T06:50:44.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.400 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.400 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.400 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824351+0000", 2024-09-11T06:50:44.401 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034101940000000001 2024-09-11T06:50:44.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.401 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.401 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824388+0000", 2024-09-11T06:50:44.401 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7446e-05 2024-09-11T06:50:44.401 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.401 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.401 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.401 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:134 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'92, mlcod=13'86)", 2024-09-11T06:50:44.402 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:50:44.402 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 97.512602463999997, 2024-09-11T06:50:44.402 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033248179999999998, 2024-09-11T06:50:44.402 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.402 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.402 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.402 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.402 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.402 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.402 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:50:44.402 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.403 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.403 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.403 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821105+0000", 2024-09-11T06:50:44.403 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.403 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.403 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.403 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821107+0000", 2024-09-11T06:50:44.403 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7430000000000001e-06 2024-09-11T06:50:44.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.403 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.403 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.403 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821205+0000", 2024-09-11T06:50:44.403 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7666999999999998e-05 2024-09-11T06:50:44.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.404 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.404 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.404 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821234+0000", 2024-09-11T06:50:44.404 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9756000000000001e-05 2024-09-11T06:50:44.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.404 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.404 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.404 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821254+0000", 2024-09-11T06:50:44.404 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9446e-05 2024-09-11T06:50:44.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.405 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.405 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821846+0000", 2024-09-11T06:50:44.405 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00059187599999999997 2024-09-11T06:50:44.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.405 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.405 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.821894+0000", 2024-09-11T06:50:44.405 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8766e-05 2024-09-11T06:50:44.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.405 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.405 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824392+0000", 2024-09-11T06:50:44.405 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0024977509999999999 2024-09-11T06:50:44.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.406 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.406 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824430+0000", 2024-09-11T06:50:44.406 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7812999999999999e-05 2024-09-11T06:50:44.406 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.406 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.406 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.406 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:139 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'97, mlcod=13'86)", 2024-09-11T06:50:44.406 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:50:44.406 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 97.507851603999995, 2024-09-11T06:50:44.406 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003732221, 2024-09-11T06:50:44.407 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.407 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.407 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.407 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.407 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.407 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:50:44.407 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.407 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.407 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:50:44.407 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.408 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.408 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.408 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825858+0000", 2024-09-11T06:50:44.408 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5829999999999999e-06 2024-09-11T06:50:44.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.408 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.408 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.408 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825874+0000", 2024-09-11T06:50:44.408 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5967000000000001e-05 2024-09-11T06:50:44.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.408 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.408 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.408 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825909+0000", 2024-09-11T06:50:44.409 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5692999999999999e-05 2024-09-11T06:50:44.409 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.409 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.409 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.409 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825923+0000", 2024-09-11T06:50:44.409 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4080000000000001e-05 2024-09-11T06:50:44.409 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.409 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.409 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.409 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826010+0000", 2024-09-11T06:50:44.409 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6248999999999994e-05 2024-09-11T06:50:44.409 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.409 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.409 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.409 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826038+0000", 2024-09-11T06:50:44.410 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8010000000000001e-05 2024-09-11T06:50:44.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.410 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.410 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.410 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829499+0000", 2024-09-11T06:50:44.410 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003461783 2024-09-11T06:50:44.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.410 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.410 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.410 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829588+0000", 2024-09-11T06:50:44.410 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8856000000000004e-05 2024-09-11T06:50:44.410 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.410 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.410 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.411 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:140 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'98, mlcod=13'93)", 2024-09-11T06:50:44.411 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:50:44.411 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 97.506000874999998, 2024-09-11T06:50:44.411 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0057995929999999996, 2024-09-11T06:50:44.411 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.411 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.411 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.411 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.411 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.411 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:50:44.411 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.412 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.412 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:50:44.412 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.412 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.412 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827708+0000", 2024-09-11T06:50:44.412 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.547e-06 2024-09-11T06:50:44.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.412 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.412 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827765+0000", 2024-09-11T06:50:44.412 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6266e-05 2024-09-11T06:50:44.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.413 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.413 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827791+0000", 2024-09-11T06:50:44.413 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6098999999999999e-05 2024-09-11T06:50:44.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.413 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.413 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827806+0000", 2024-09-11T06:50:44.413 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5537e-05 2024-09-11T06:50:44.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.413 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.413 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827928+0000", 2024-09-11T06:50:44.414 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000122012 2024-09-11T06:50:44.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.414 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.414 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827961+0000", 2024-09-11T06:50:44.414 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2663999999999998e-05 2024-09-11T06:50:44.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.414 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.414 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833409+0000", 2024-09-11T06:50:44.414 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054479350000000001 2024-09-11T06:50:44.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.414 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.415 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833506+0000", 2024-09-11T06:50:44.415 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7533000000000003e-05 2024-09-11T06:50:44.415 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.415 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.415 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.415 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.415 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:141 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'99, mlcod=13'93)", 2024-09-11T06:50:44.415 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:50:44.415 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 97.505107491999993, 2024-09-11T06:50:44.415 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055394709999999998, 2024-09-11T06:50:44.415 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.415 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.415 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.416 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.416 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.416 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:50:44.416 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.416 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.416 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:50:44.416 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.417 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.417 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828602+0000", 2024-09-11T06:50:44.417 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.46e-06 2024-09-11T06:50:44.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.417 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.417 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828618+0000", 2024-09-11T06:50:44.417 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6592999999999999e-05 2024-09-11T06:50:44.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.418 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.418 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828646+0000", 2024-09-11T06:50:44.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7552999999999999e-05 2024-09-11T06:50:44.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.418 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.418 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828663+0000", 2024-09-11T06:50:44.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.719e-05 2024-09-11T06:50:44.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.418 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.418 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829023+0000", 2024-09-11T06:50:44.419 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036054799999999998 2024-09-11T06:50:44.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.419 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.419 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829062+0000", 2024-09-11T06:50:44.419 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8865999999999997e-05 2024-09-11T06:50:44.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.419 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.419 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834078+0000", 2024-09-11T06:50:44.419 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050157480000000004 2024-09-11T06:50:44.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.420 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.420 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834140+0000", 2024-09-11T06:50:44.420 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1513000000000005e-05 2024-09-11T06:50:44.420 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.420 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.420 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.420 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:144 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'102, mlcod=13'98)", 2024-09-11T06:50:44.420 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:50:44.420 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 97.495946879000002, 2024-09-11T06:50:44.421 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003210348, 2024-09-11T06:50:44.421 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.421 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.421 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.421 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.421 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.421 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.421 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:50:44.421 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.422 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.422 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837761+0000", 2024-09-11T06:50:44.422 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.422 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.422 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837768+0000", 2024-09-11T06:50:44.422 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9330000000000004e-06 2024-09-11T06:50:44.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.423 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.423 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837858+0000", 2024-09-11T06:50:44.423 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0185999999999993e-05 2024-09-11T06:50:44.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.423 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.423 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837890+0000", 2024-09-11T06:50:44.423 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2437000000000001e-05 2024-09-11T06:50:44.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.423 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.424 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.837905+0000", 2024-09-11T06:50:44.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:50:44.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.424 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.424 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838019+0000", 2024-09-11T06:50:44.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000114336 2024-09-11T06:50:44.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.424 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.425 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838052+0000", 2024-09-11T06:50:44.425 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3139000000000002e-05 2024-09-11T06:50:44.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.425 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.425 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840914+0000", 2024-09-11T06:50:44.425 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028612780000000001 2024-09-11T06:50:44.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.425 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.425 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.840971+0000", 2024-09-11T06:50:44.426 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7402999999999999e-05 2024-09-11T06:50:44.426 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.426 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.426 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.426 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.426 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:145 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'103, mlcod=13'98)", 2024-09-11T06:50:44.426 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:50:44.426 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 97.495154994999993, 2024-09-11T06:50:44.426 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054455409999999999, 2024-09-11T06:50:44.426 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.426 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.426 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.426 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.427 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.427 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:50:44.427 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.427 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.427 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:50:44.427 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.427 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.427 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838554+0000", 2024-09-11T06:50:44.427 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3430000000000001e-06 2024-09-11T06:50:44.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.428 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.428 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838640+0000", 2024-09-11T06:50:44.428 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5569999999999996e-05 2024-09-11T06:50:44.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.428 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.428 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838660+0000", 2024-09-11T06:50:44.428 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0576000000000001e-05 2024-09-11T06:50:44.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.428 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.428 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838674+0000", 2024-09-11T06:50:44.428 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3529999999999999e-05 2024-09-11T06:50:44.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.429 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.429 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838838+0000", 2024-09-11T06:50:44.429 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016468799999999999 2024-09-11T06:50:44.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.429 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.429 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838885+0000", 2024-09-11T06:50:44.429 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6884e-05 2024-09-11T06:50:44.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.430 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.430 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843901+0000", 2024-09-11T06:50:44.430 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050162310000000003 2024-09-11T06:50:44.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.430 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.430 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843998+0000", 2024-09-11T06:50:44.430 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.6718999999999993e-05 2024-09-11T06:50:44.430 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.430 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.430 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.430 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4558.0:1 3.3 e56/55 3:c4c92e5a:::rbd_trash:head v 56'1, mlcod=0'0)", 2024-09-11T06:50:44.431 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:50:44.431 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.242351932999998, 2024-09-11T06:50:44.431 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0053791810000000002, 2024-09-11T06:50:44.431 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.431 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.431 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.431 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.431 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.431 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:50:44.431 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.432 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.432 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:50:44.432 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.432 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.432 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091357+0000", 2024-09-11T06:50:44.432 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.54e-07 2024-09-11T06:50:44.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.432 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.432 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091376+0000", 2024-09-11T06:50:44.433 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9769e-05 2024-09-11T06:50:44.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.433 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.433 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.433 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091396+0000", 2024-09-11T06:50:44.433 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0152999999999998e-05 2024-09-11T06:50:44.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.433 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.433 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.433 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091416+0000", 2024-09-11T06:50:44.433 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0006000000000001e-05 2024-09-11T06:50:44.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.434 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.434 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091506+0000", 2024-09-11T06:50:44.434 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9325000000000003e-05 2024-09-11T06:50:44.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.434 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.434 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091523+0000", 2024-09-11T06:50:44.434 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.77e-05 2024-09-11T06:50:44.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.434 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.434 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.096685+0000", 2024-09-11T06:50:44.435 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0051618569999999997 2024-09-11T06:50:44.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.435 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.435 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.435 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.096735+0000", 2024-09-11T06:50:44.435 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9517e-05 2024-09-11T06:50:44.435 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.435 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.435 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.435 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.435 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.435 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:44.437 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:50:44.438 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:44.435+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:44.539 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:44.535+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:44.593 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:44.593 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:44.593 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:50:44.593 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:44.605 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:50:44.610 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:50:44.610 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:50:44.625 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' rbd = rbd ']' 2024-09-11T06:50:44.625 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo rbd device --device-type nbd map testrbdnbd21607 2024-09-11T06:50:44.625 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:50:44.626 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:50:44.627 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:50:44.628 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2024-09-11T06:50:44.292486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":134808318,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":2895,"num_read_kb":110462,"num_write":714,"num_write_kb":264747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":174328190,"num_keys_recovered":22,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":653,"ondisk_log_size":653,"up":34,"acting":34,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":423768,"kb_used_data":267796,"kb_used_omap":7,"kb_used_meta":155832,"kb_avail":471435432,"statfs":{"total":483183820800,"available":482749882368,"internally_reserved":0,"allocated":274223104,"data_stored":271332422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7970,"internal_metadata":159572190},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":38,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":582,"num_read_kb":6494,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.041991"},"pg_stats":[{"pgid":"1.0","version":"14'117","reported_seq":294,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.273698+0000","last_change":"2024-09-11T06:50:37.941212+0000","last_active":"2024-09-11T06:50:43.273698+0000","last_peered":"2024-09-11T06:50:43.273698+0000","last_clean":"2024-09-11T06:50:43.273698+0000","last_became_active":"2024-09-11T06:50:37.940448+0000","last_became_peered":"2024-09-11T06:50:37.940448+0000","last_unstale":"2024-09-11T06:50:43.273698+0000","last_undegraded":"2024-09-11T06:50:43.273698+0000","last_fullsized":"2024-09-11T06:50:43.273698+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_clean_scrub_stamp":"2024-09-11T06:49:05.088839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:48:59.833858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"72'39","reported_seq":257,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.274840+0000","last_change":"2024-09-11T06:50:29.249902+0000","last_active":"2024-09-11T06:50:43.274840+0000","last_peered":"2024-09-11T06:50:43.274840+0000","last_clean":"2024-09-11T06:50:43.274840+0000","last_became_active":"2024-09-11T06:50:29.172399+0000","last_became_peered":"2024-09-11T06:50:29.172399+0000","last_unstale":"2024-09-11T06:50:43.274840+0000","last_undegraded":"2024-09-11T06:50:43.274840+0000","last_fullsized":"2024-09-11T06:50:43.274840+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:25:03.908704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011212469999999999,"stat_sum":{"num_bytes":12582950,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":74,"num_read_kb":8198,"num_write":38,"num_write_kb":28673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":12582912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"58'2","reported_seq":79,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.508598+0000","last_change":"2024-09-11T06:50:35.245582+0000","last_active":"2024-09-11T06:50:43.508598+0000","last_peered":"2024-09-11T06:50:43.508598+0000","last_clean":"2024-09-11T06:50:43.508598+0000","last_became_active":"2024-09-11T06:50:35.240946+0000","last_became_peered":"2024-09-11T06:50:35.240946+0000","last_unstale":"2024-09-11T06:50:43.508598+0000","last_undegraded":"2024-09-11T06:50:43.508598+0000","last_fullsized":"2024-09-11T06:50:43.508598+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:17:27.363513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011764950000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"68'57","reported_seq":187,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.273817+0000","last_change":"2024-09-11T06:50:35.668519+0000","last_active":"2024-09-11T06:50:43.273817+0000","last_peered":"2024-09-11T06:50:43.273817+0000","last_clean":"2024-09-11T06:50:43.273817+0000","last_became_active":"2024-09-11T06:50:35.238279+0000","last_became_peered":"2024-09-11T06:50:35.238279+0000","last_unstale":"2024-09-11T06:50:43.273817+0000","last_undegraded":"2024-09-11T06:50:43.273817+0000","last_fullsized":"2024-09-11T06:50:43.273817+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:14:06.141440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077615599999999996,"stat_sum":{"num_bytes":20971520,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":150,"num_read_kb":23060,"num_write":56,"num_write_kb":45056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":41943040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":58,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.372412+0000","last_change":"2024-09-11T06:50:19.703255+0000","last_active":"2024-09-11T06:50:43.372412+0000","last_peered":"2024-09-11T06:50:43.372412+0000","last_clean":"2024-09-11T06:50:43.372412+0000","last_became_active":"2024-09-11T06:50:17.089296+0000","last_became_peered":"2024-09-11T06:50:17.089296+0000","last_unstale":"2024-09-11T06:50:43.372412+0000","last_undegraded":"2024-09-11T06:50:43.372412+0000","last_fullsized":"2024-09-11T06:50:43.372412+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:42:18.684413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001159373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"78'106","reported_seq":982,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.275047+0000","last_change":"2024-09-11T06:49:55.948809+0000","last_active":"2024-09-11T06:50:43.275047+0000","last_peered":"2024-09-11T06:50:43.275047+0000","last_clean":"2024-09-11T06:50:43.275047+0000","last_became_active":"2024-09-11T06:49:55.948499+0000","last_became_peered":"2024-09-11T06:49:55.948499+0000","last_unstale":"2024-09-11T06:50:43.275047+0000","last_undegraded":"2024-09-11T06:50:43.275047+0000","last_fullsized":"2024-09-11T06:50:43.275047+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:14:11.313635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010163780000000001,"stat_sum":{"num_bytes":25165862,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1245,"num_read_kb":22324,"num_write":62,"num_write_kb":40966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":59,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.508536+0000","last_change":"2024-09-11T06:50:19.097182+0000","last_active":"2024-09-11T06:50:43.508536+0000","last_peered":"2024-09-11T06:50:43.508536+0000","last_clean":"2024-09-11T06:50:43.508536+0000","last_became_active":"2024-09-11T06:50:17.097783+0000","last_became_peered":"2024-09-11T06:50:17.097783+0000","last_unstale":"2024-09-11T06:50:43.508536+0000","last_undegraded":"2024-09-11T06:50:43.508536+0000","last_fullsized":"2024-09-11T06:50:43.508536+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:35:49.244377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088028100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"54'4","reported_seq":313,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.508703+0000","last_change":"2024-09-11T06:50:29.230433+0000","last_active":"2024-09-11T06:50:43.508703+0000","last_peered":"2024-09-11T06:50:43.508703+0000","last_clean":"2024-09-11T06:50:43.508703+0000","last_became_active":"2024-09-11T06:50:29.171189+0000","last_became_peered":"2024-09-11T06:50:29.171189+0000","last_unstale":"2024-09-11T06:50:43.508703+0000","last_undegraded":"2024-09-11T06:50:43.508703+0000","last_fullsized":"2024-09-11T06:50:43.508703+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:19.442991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001910244,"stat_sum":{"num_bytes":8388627,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":152,"num_read_kb":2667,"num_write":4,"num_write_kb":8194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":8388684,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"56'1","reported_seq":59,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.372426+0000","last_change":"2024-09-11T06:50:19.703261+0000","last_active":"2024-09-11T06:50:43.372426+0000","last_peered":"2024-09-11T06:50:43.372426+0000","last_clean":"2024-09-11T06:50:43.372426+0000","last_became_active":"2024-09-11T06:50:17.089261+0000","last_became_peered":"2024-09-11T06:50:17.089261+0000","last_unstale":"2024-09-11T06:50:43.372426+0000","last_undegraded":"2024-09-11T06:50:43.372426+0000","last_fullsized":"2024-09-11T06:50:43.372426+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:00:38.491173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001141853,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.274862+0000","last_change":"2024-09-11T06:50:35.238915+0000","last_active":"2024-09-11T06:50:43.274862+0000","last_peered":"2024-09-11T06:50:43.274862+0000","last_clean":"2024-09-11T06:50:43.274862+0000","last_became_active":"2024-09-11T06:50:35.238289+0000","last_became_peered":"2024-09-11T06:50:35.238289+0000","last_unstale":"2024-09-11T06:50:43.274862+0000","last_undegraded":"2024-09-11T06:50:43.274862+0000","last_fullsized":"2024-09-11T06:50:43.274862+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:13:46.102794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"72'106","reported_seq":263,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.508761+0000","last_change":"2024-09-11T06:49:56.122331+0000","last_active":"2024-09-11T06:50:43.508761+0000","last_peered":"2024-09-11T06:50:43.508761+0000","last_clean":"2024-09-11T06:50:43.508761+0000","last_became_active":"2024-09-11T06:49:56.121998+0000","last_became_peered":"2024-09-11T06:49:56.121998+0000","last_unstale":"2024-09-11T06:50:43.508761+0000","last_undegraded":"2024-09-11T06:50:43.508761+0000","last_fullsized":"2024-09-11T06:50:43.508761+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:24:38.234463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582950,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":501,"num_read_kb":8581,"num_write":170,"num_write_kb":20614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":59,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.508648+0000","last_change":"2024-09-11T06:50:19.097504+0000","last_active":"2024-09-11T06:50:43.508648+0000","last_peered":"2024-09-11T06:50:43.508648+0000","last_clean":"2024-09-11T06:50:43.508648+0000","last_became_active":"2024-09-11T06:50:17.096243+0000","last_became_peered":"2024-09-11T06:50:17.096243+0000","last_unstale":"2024-09-11T06:50:43.508648+0000","last_undegraded":"2024-09-11T06:50:43.508648+0000","last_fullsized":"2024-09-11T06:50:43.508648+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:44:45.211005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085501399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"70'66","reported_seq":157,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.273743+0000","last_change":"2024-09-11T06:50:35.337013+0000","last_active":"2024-09-11T06:50:43.273743+0000","last_peered":"2024-09-11T06:50:43.273743+0000","last_clean":"2024-09-11T06:50:43.273743+0000","last_became_active":"2024-09-11T06:50:35.236333+0000","last_became_peered":"2024-09-11T06:50:35.236333+0000","last_unstale":"2024-09-11T06:50:43.273743+0000","last_undegraded":"2024-09-11T06:50:43.273743+0000","last_fullsized":"2024-09-11T06:50:43.273743+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:40.624982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095172900000000001,"stat_sum":{"num_bytes":20971552,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":274,"num_read_kb":16513,"num_write":66,"num_write_kb":36867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":41943104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":59,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.275130+0000","last_change":"2024-09-11T06:50:19.096344+0000","last_active":"2024-09-11T06:50:43.275130+0000","last_peered":"2024-09-11T06:50:43.275130+0000","last_clean":"2024-09-11T06:50:43.275130+0000","last_became_active":"2024-09-11T06:50:17.096560+0000","last_became_peered":"2024-09-11T06:50:17.096560+0000","last_unstale":"2024-09-11T06:50:43.275130+0000","last_undegraded":"2024-09-11T06:50:43.275130+0000","last_fullsized":"2024-09-11T06:50:43.275130+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:03:19.807744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053029399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"74'43","reported_seq":199,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.508813+0000","last_change":"2024-09-11T06:50:35.320796+0000","last_active":"2024-09-11T06:50:43.508813+0000","last_peered":"2024-09-11T06:50:43.508813+0000","last_clean":"2024-09-11T06:50:43.508813+0000","last_became_active":"2024-09-11T06:50:35.241589+0000","last_became_peered":"2024-09-11T06:50:35.241589+0000","last_unstale":"2024-09-11T06:50:43.508813+0000","last_undegraded":"2024-09-11T06:50:43.508813+0000","last_fullsized":"2024-09-11T06:50:43.508813+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:20.312915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388646,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":118,"num_read_kb":8286,"num_write":59,"num_write_kb":16420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":16777292,"num_keys_recovered":6,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":59,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.273135+0000","last_change":"2024-09-11T06:50:19.688101+0000","last_active":"2024-09-11T06:50:43.273135+0000","last_peered":"2024-09-11T06:50:43.273135+0000","last_clean":"2024-09-11T06:50:43.273135+0000","last_became_active":"2024-09-11T06:50:17.096959+0000","last_became_peered":"2024-09-11T06:50:17.096959+0000","last_unstale":"2024-09-11T06:50:43.273135+0000","last_undegraded":"2024-09-11T06:50:43.273135+0000","last_fullsized":"2024-09-11T06:50:43.273135+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:54:30.491263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059833200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"78'112","reported_seq":142,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.273818+0000","last_change":"2024-09-11T06:50:35.519269+0000","last_active":"2024-09-11T06:50:43.273818+0000","last_peered":"2024-09-11T06:50:43.273818+0000","last_clean":"2024-09-11T06:50:43.273818+0000","last_became_active":"2024-09-11T06:50:35.238011+0000","last_became_peered":"2024-09-11T06:50:35.238011+0000","last_unstale":"2024-09-11T06:50:43.273818+0000","last_undegraded":"2024-09-11T06:50:43.273818+0000","last_fullsized":"2024-09-11T06:50:43.273818+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:23:49.359046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001403348,"stat_sum":{"num_bytes":25165824,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":265,"num_read_kb":20733,"num_write":98,"num_write_kb":65539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":50331648,"num_keys_recovered":16,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":134217931,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":2779,"num_read_kb":110362,"num_write":553,"num_write_kb":262329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":171966680,"num_keys_recovered":22,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":289468416,"data_stored":289407401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":533,"ondisk_log_size":533,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":0,"seq":47244640278,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640279,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56660,"kb_used_data":25464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94315180,"statfs":{"total":96636764160,"available":96578744320,"internally_reserved":0,"allocated":26075136,"data_stored":25509971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640279,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":106400,"kb_used_data":75212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94265440,"statfs":{"total":96636764160,"available":96527810560,"internally_reserved":0,"allocated":77017088,"data_stored":76432076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640280,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68956,"kb_used_data":37760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94302884,"statfs":{"total":96636764160,"available":96566153216,"internally_reserved":0,"allocated":38666240,"data_stored":38092959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640280,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118108,"kb_used_data":86916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94253732,"statfs":{"total":96636764160,"available":96515821568,"internally_reserved":0,"allocated":89001984,"data_stored":88419772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640281,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73644,"kb_used_data":42444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94298196,"statfs":{"total":96636764160,"available":96561352704,"internally_reserved":0,"allocated":43462656,"data_stored":42877644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":41959424,"data_stored":41943129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":88100864,"data_stored":88080555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":37756928,"data_stored":37748812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":75509760,"data_stored":75497542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":25165824,"data_stored":25165824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20975616,"data_stored":20971539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:50:44.629 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:50:44.630 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which rbd 2024-09-11T06:50:44.630 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/rbd 2024-09-11T06:50:44.630 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:50:44.630 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map testrbdnbd21607 2024-09-11T06:50:44.669 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:44.667+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:44.729 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:44.730 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:44.730 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:50:44.730 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:50:44.730 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:44.740 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:44.739+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:44.746 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:50:44.766 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV=/dev/nbd0 2024-09-11T06:50:44.766 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd 2024-09-11T06:50:44.766 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:50:44.766 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:50:44.767 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:50:44.767 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd0'\'']/id' 2024-09-11T06:50:44.831 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=23447 2024-09-11T06:50:44.832 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 23447 2024-09-11T06:50:44.832 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 23447 -C rbd-nbd 2024-09-11T06:50:44.838 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:50:44.838 INFO:tasks.workunit.client.0.smithi176.stdout: 23447 ? 00:00:00 rbd-nbd 2024-09-11T06:50:44.839 INFO:tasks.workunit.client.0.smithi176.stderr:+ unmap_device testrbdnbd21607 23447 2024-09-11T06:50:44.839 INFO:tasks.workunit.client.0.smithi176.stderr:+ local args=testrbdnbd21607 2024-09-11T06:50:44.839 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pid=23447 2024-09-11T06:50:44.839 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device --device-type nbd unmap testrbdnbd21607 2024-09-11T06:50:44.839 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:44.839 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:44.840 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:44.840 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:50:44.841 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:44.835+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:44.843 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:50:44.843 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:44.843 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap testrbdnbd21607 2024-09-11T06:50:44.863 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:44.864 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:50:44.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:50:44.864 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:50:44.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.864 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:5 2.2 e17/15 2:5cea7035:::rbd_info:head v 17'1, mlcod=0'0)", 2024-09-11T06:50:44.864 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:50:44.864 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 92.625428450000001, 2024-09-11T06:50:44.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0019446719999999999, 2024-09-11T06:50:44.864 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.864 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.864 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.865 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:50:44.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.866 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236084+0000", 2024-09-11T06:50:44.866 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.866 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.866 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236085+0000", 2024-09-11T06:50:44.866 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6000000000000003e-07 2024-09-11T06:50:44.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.866 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.866 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236108+0000", 2024-09-11T06:50:44.866 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3113e-05 2024-09-11T06:50:44.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.867 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.867 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236133+0000", 2024-09-11T06:50:44.867 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5723000000000001e-05 2024-09-11T06:50:44.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.867 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.867 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236154+0000", 2024-09-11T06:50:44.868 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0333999999999999e-05 2024-09-11T06:50:44.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.868 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.868 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236297+0000", 2024-09-11T06:50:44.868 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014278500000000001 2024-09-11T06:50:44.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.868 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.868 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.236383+0000", 2024-09-11T06:50:44.868 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5927999999999995e-05 2024-09-11T06:50:44.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.869 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.869 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.237950+0000", 2024-09-11T06:50:44.869 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015673690000000001 2024-09-11T06:50:44.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.869 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.869 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:12.238029+0000", 2024-09-11T06:50:44.869 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8659999999999996e-05 2024-09-11T06:50:44.869 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.870 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.870 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.870 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.870 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4230.0:7 2.2 e18/15 2:5cea7035:::rbd_info:head v 18'2, mlcod=0'0)", 2024-09-11T06:50:44.870 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:50:44.870 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 91.611103243000002, 2024-09-11T06:50:44.870 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001907192, 2024-09-11T06:50:44.870 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.870 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.870 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.871 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.871 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.871 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.871 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:50:44.871 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.871 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.871 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.871 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.871 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250409+0000", 2024-09-11T06:50:44.871 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.871 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.871 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.871 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.871 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250410+0000", 2024-09-11T06:50:44.872 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7700000000000004e-07 2024-09-11T06:50:44.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.872 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.872 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250445+0000", 2024-09-11T06:50:44.872 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4749000000000002e-05 2024-09-11T06:50:44.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.872 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.872 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250478+0000", 2024-09-11T06:50:44.872 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3269999999999998e-05 2024-09-11T06:50:44.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250501+0000", 2024-09-11T06:50:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3632999999999999e-05 2024-09-11T06:50:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250633+0000", 2024-09-11T06:50:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013143200000000001 2024-09-11T06:50:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.250700+0000", 2024-09-11T06:50:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7002999999999995e-05 2024-09-11T06:50:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.252172+0000", 2024-09-11T06:50:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014719550000000001 2024-09-11T06:50:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:13.252316+0000", 2024-09-11T06:50:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144473 2024-09-11T06:50:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4558.0:5 3.2 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-11T06:50:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:50:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 26.773094013000001, 2024-09-11T06:50:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0062353959999999998, 2024-09-11T06:50:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4558", 2024-09-11T06:50:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1478388898", 2024-09-11T06:50:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 5 2024-09-11T06:50:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:50:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:50:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088422+0000", 2024-09-11T06:50:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7859999999999998e-06 2024-09-11T06:50:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088440+0000", 2024-09-11T06:50:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8185999999999998e-05 2024-09-11T06:50:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088479+0000", 2024-09-11T06:50:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8248999999999999e-05 2024-09-11T06:50:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088497+0000", 2024-09-11T06:50:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8889000000000002e-05 2024-09-11T06:50:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.879 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088658+0000", 2024-09-11T06:50:44.879 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000160271 2024-09-11T06:50:44.879 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.879 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op must wait for map", 2024-09-11T06:50:44.879 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088668+0000", 2024-09-11T06:50:44.879 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0409e-05 2024-09-11T06:50:44.879 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.879 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.879 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092347+0000", 2024-09-11T06:50:44.880 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00367915 2024-09-11T06:50:44.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.880 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.880 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.880 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092541+0000", 2024-09-11T06:50:44.880 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000193833 2024-09-11T06:50:44.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.880 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.880 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-11T06:50:44.880 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092732+0000", 2024-09-11T06:50:44.880 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000190772 2024-09-11T06:50:44.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.881 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:44.881 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094136+0000", 2024-09-11T06:50:44.881 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014040809999999999 2024-09-11T06:50:44.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.881 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:44.881 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094506+0000", 2024-09-11T06:50:44.881 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000369959 2024-09-11T06:50:44.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.881 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.882 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094590+0000", 2024-09-11T06:50:44.882 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4033e-05 2024-09-11T06:50:44.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.882 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.882 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094654+0000", 2024-09-11T06:50:44.882 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3777999999999998e-05 2024-09-11T06:50:44.882 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.882 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.882 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.883 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:259 2.6 2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head [sparse-read 4096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:44.883 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.147544+0000", 2024-09-11T06:50:44.883 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.713968629, 2024-09-11T06:50:44.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0019824170000000002, 2024-09-11T06:50:44.883 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.883 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.883 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:44.883 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:44.883 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:44.883 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:44.883 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 259 2024-09-11T06:50:44.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.883 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.884 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.884 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.147544+0000", 2024-09-11T06:50:44.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.884 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.884 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.147544+0000", 2024-09-11T06:50:44.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.884 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.884 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.147545+0000", 2024-09-11T06:50:44.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4600000000000003e-07 2024-09-11T06:50:44.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.147549+0000", 2024-09-11T06:50:44.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8160000000000001e-06 2024-09-11T06:50:44.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.147565+0000", 2024-09-11T06:50:44.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5140000000000001e-05 2024-09-11T06:50:44.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.147574+0000", 2024-09-11T06:50:44.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2529999999999993e-06 2024-09-11T06:50:44.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.148551+0000", 2024-09-11T06:50:44.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0009775579999999999 2024-09-11T06:50:44.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.148694+0000", 2024-09-11T06:50:44.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014299399999999999 2024-09-11T06:50:44.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.887 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.149526+0000", 2024-09-11T06:50:44.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00083171000000000004 2024-09-11T06:50:44.887 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.887 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.887 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.887 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:261 2.6 2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:44.887 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.154411+0000", 2024-09-11T06:50:44.887 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.707101215000002, 2024-09-11T06:50:44.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002245024, 2024-09-11T06:50:44.887 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.887 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.887 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:44.888 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:44.888 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:44.888 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:44.888 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 261 2024-09-11T06:50:44.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.888 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.888 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.888 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154411+0000", 2024-09-11T06:50:44.888 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.888 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.889 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154411+0000", 2024-09-11T06:50:44.889 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.889 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.889 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154412+0000", 2024-09-11T06:50:44.889 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:50:44.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.889 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.889 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154416+0000", 2024-09-11T06:50:44.889 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3930000000000001e-06 2024-09-11T06:50:44.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154428+0000", 2024-09-11T06:50:44.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2099999999999999e-05 2024-09-11T06:50:44.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154438+0000", 2024-09-11T06:50:44.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2590000000000006e-06 2024-09-11T06:50:44.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.155643+0000", 2024-09-11T06:50:44.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012048689999999999 2024-09-11T06:50:44.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.155838+0000", 2024-09-11T06:50:44.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019531299999999999 2024-09-11T06:50:44.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.156656+0000", 2024-09-11T06:50:44.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00081824700000000003 2024-09-11T06:50:44.891 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.891 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.891 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.892 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:525 2.4 2:2a17619e:::rbd_data.1314f0a2bfa2.000000000000000e:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:44.892 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.530454+0000", 2024-09-11T06:50:44.892 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.33105853, 2024-09-11T06:50:44.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0019464650000000001, 2024-09-11T06:50:44.892 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.892 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.892 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:44.892 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:44.892 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:44.892 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:44.892 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 525 2024-09-11T06:50:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530454+0000", 2024-09-11T06:50:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530454+0000", 2024-09-11T06:50:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.894 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530455+0000", 2024-09-11T06:50:44.894 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4100000000000001e-06 2024-09-11T06:50:44.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.894 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.894 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.894 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530462+0000", 2024-09-11T06:50:44.894 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2760000000000003e-06 2024-09-11T06:50:44.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.894 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.894 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.894 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530486+0000", 2024-09-11T06:50:44.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3082999999999999e-05 2024-09-11T06:50:44.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.530511+0000", 2024-09-11T06:50:44.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5065000000000001e-05 2024-09-11T06:50:44.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.531637+0000", 2024-09-11T06:50:44.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011263530000000001 2024-09-11T06:50:44.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.896 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.896 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.531716+0000", 2024-09-11T06:50:44.896 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8565000000000004e-05 2024-09-11T06:50:44.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.896 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.896 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.532400+0000", 2024-09-11T06:50:44.896 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00068471299999999999 2024-09-11T06:50:44.896 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.896 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.897 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.897 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:549 2.4 2:2a17619e:::rbd_data.1314f0a2bfa2.000000000000000e:head [sparse-read 3411968~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:44.897 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.567982+0000", 2024-09-11T06:50:44.897 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.293529884000002, 2024-09-11T06:50:44.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002000216, 2024-09-11T06:50:44.897 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.898 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.898 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:44.898 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:44.898 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:44.898 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:44.898 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 549 2024-09-11T06:50:44.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.898 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.898 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.898 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.567982+0000", 2024-09-11T06:50:44.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.899 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.899 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.567982+0000", 2024-09-11T06:50:44.899 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.899 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.899 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.567984+0000", 2024-09-11T06:50:44.899 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7290000000000001e-06 2024-09-11T06:50:44.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.899 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.567994+0000", 2024-09-11T06:50:44.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4229999999999994e-06 2024-09-11T06:50:44.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.568015+0000", 2024-09-11T06:50:44.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1596000000000001e-05 2024-09-11T06:50:44.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.568032+0000", 2024-09-11T06:50:44.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7067000000000001e-05 2024-09-11T06:50:44.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.568713+0000", 2024-09-11T06:50:44.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00068109799999999999 2024-09-11T06:50:44.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.902 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.568866+0000", 2024-09-11T06:50:44.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015232399999999999 2024-09-11T06:50:44.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.902 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.902 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.569983+0000", 2024-09-11T06:50:44.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011169789999999999 2024-09-11T06:50:44.902 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.902 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.902 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.903 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:587 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-11T06:50:44.903 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:50:44.903 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.632261358000001, 2024-09-11T06:50:44.903 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0023436289999999999, 2024-09-11T06:50:44.903 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.903 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.903 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.903 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:44.903 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:44.904 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:44.904 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 587 2024-09-11T06:50:44.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.904 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.904 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.904 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:50:44.904 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.904 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.905 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:50:44.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.905 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.905 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229252+0000", 2024-09-11T06:50:44.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3799999999999999e-06 2024-09-11T06:50:44.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.905 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.905 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229259+0000", 2024-09-11T06:50:44.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4760000000000003e-06 2024-09-11T06:50:44.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.906 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229271+0000", 2024-09-11T06:50:44.906 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2313000000000001e-05 2024-09-11T06:50:44.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.906 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229279+0000", 2024-09-11T06:50:44.906 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1610000000000003e-06 2024-09-11T06:50:44.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.906 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229344+0000", 2024-09-11T06:50:44.906 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4375000000000006e-05 2024-09-11T06:50:44.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.907 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.907 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229404+0000", 2024-09-11T06:50:44.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0517999999999999e-05 2024-09-11T06:50:44.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.907 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:44.907 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229516+0000", 2024-09-11T06:50:44.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011129900000000001 2024-09-11T06:50:44.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.907 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:44.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231502+0000", 2024-09-11T06:50:44.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001986238 2024-09-11T06:50:44.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:44.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231537+0000", 2024-09-11T06:50:44.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4968000000000001e-05 2024-09-11T06:50:44.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231569+0000", 2024-09-11T06:50:44.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2459000000000002e-05 2024-09-11T06:50:44.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.909 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.909 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231595+0000", 2024-09-11T06:50:44.909 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5442000000000001e-05 2024-09-11T06:50:44.909 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.909 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.909 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.909 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:15 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:44.909 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.709418+0000", 2024-09-11T06:50:44.910 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.152094204999999, 2024-09-11T06:50:44.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027504280000000001, 2024-09-11T06:50:44.910 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.910 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.910 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.910 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:44.910 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:44.910 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:44.910 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 15 2024-09-11T06:50:44.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.910 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.911 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.911 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709418+0000", 2024-09-11T06:50:44.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.911 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.911 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709418+0000", 2024-09-11T06:50:44.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.911 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.911 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709420+0000", 2024-09-11T06:50:44.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2299999999999998e-06 2024-09-11T06:50:44.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.912 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.912 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709432+0000", 2024-09-11T06:50:44.912 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1897e-05 2024-09-11T06:50:44.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.912 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.912 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.912 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709464+0000", 2024-09-11T06:50:44.912 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1819000000000001e-05 2024-09-11T06:50:44.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.912 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.912 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.913 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709479+0000", 2024-09-11T06:50:44.913 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4853e-05 2024-09-11T06:50:44.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.913 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709624+0000", 2024-09-11T06:50:44.913 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144912 2024-09-11T06:50:44.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.913 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709773+0000", 2024-09-11T06:50:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000149529 2024-09-11T06:50:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.710055+0000", 2024-09-11T06:50:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000281692 2024-09-11T06:50:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.711412+0000", 2024-09-11T06:50:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001356931 2024-09-11T06:50:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:44.915 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.711890+0000", 2024-09-11T06:50:44.915 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00047768700000000002 2024-09-11T06:50:44.915 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.915 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.915 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.712050+0000", 2024-09-11T06:50:44.915 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016041499999999999 2024-09-11T06:50:44.915 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.915 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.915 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.712169+0000", 2024-09-11T06:50:44.915 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011846299999999999 2024-09-11T06:50:44.916 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.916 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.916 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.916 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.916 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:18 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:44.916 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.713502+0000", 2024-09-11T06:50:44.916 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.148010779, 2024-09-11T06:50:44.916 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032323069999999998, 2024-09-11T06:50:44.916 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.916 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.916 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.916 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:44.917 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:44.917 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:44.917 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18 2024-09-11T06:50:44.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.917 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.917 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.917 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.917 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713502+0000", 2024-09-11T06:50:44.917 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.917 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.917 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.917 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713502+0000", 2024-09-11T06:50:44.917 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.918 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.918 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713503+0000", 2024-09-11T06:50:44.918 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8199999999999999e-06 2024-09-11T06:50:44.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.918 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.918 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713513+0000", 2024-09-11T06:50:44.918 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8539999999999992e-06 2024-09-11T06:50:44.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.918 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.918 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713540+0000", 2024-09-11T06:50:44.919 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6594999999999999e-05 2024-09-11T06:50:44.919 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.919 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.919 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.919 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713553+0000", 2024-09-11T06:50:44.919 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2843999999999999e-05 2024-09-11T06:50:44.919 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.919 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.919 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.919 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713686+0000", 2024-09-11T06:50:44.919 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013294899999999999 2024-09-11T06:50:44.919 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.920 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.920 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.920 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713937+0000", 2024-09-11T06:50:44.920 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025110600000000002 2024-09-11T06:50:44.920 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.920 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.920 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:44.920 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.714412+0000", 2024-09-11T06:50:44.920 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00047562 2024-09-11T06:50:44.920 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.920 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.920 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:44.920 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.715867+0000", 2024-09-11T06:50:44.921 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001455004 2024-09-11T06:50:44.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.921 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.921 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:44.921 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.716569+0000", 2024-09-11T06:50:44.921 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070159400000000005 2024-09-11T06:50:44.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.921 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.921 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.921 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.716660+0000", 2024-09-11T06:50:44.921 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1403000000000006e-05 2024-09-11T06:50:44.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.921 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.921 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.922 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.716734+0000", 2024-09-11T06:50:44.922 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3517999999999996e-05 2024-09-11T06:50:44.922 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.922 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.922 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.922 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.922 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:23 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:44.922 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.723392+0000", 2024-09-11T06:50:44.922 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.138120183, 2024-09-11T06:50:44.922 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027098489999999999, 2024-09-11T06:50:44.922 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.922 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.922 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.923 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:44.923 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:44.923 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:44.923 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23 2024-09-11T06:50:44.923 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.923 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.923 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.923 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.923 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723392+0000", 2024-09-11T06:50:44.923 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.923 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.923 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.923 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.923 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723392+0000", 2024-09-11T06:50:44.924 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.924 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.924 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.924 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723394+0000", 2024-09-11T06:50:44.924 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0669999999999999e-06 2024-09-11T06:50:44.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.924 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.924 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.924 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723406+0000", 2024-09-11T06:50:44.924 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1367e-05 2024-09-11T06:50:44.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.924 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.925 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.925 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723436+0000", 2024-09-11T06:50:44.925 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0874999999999998e-05 2024-09-11T06:50:44.925 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.925 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.925 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.925 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723451+0000", 2024-09-11T06:50:44.925 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4192999999999999e-05 2024-09-11T06:50:44.925 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.925 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.925 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.925 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723577+0000", 2024-09-11T06:50:44.925 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000126267 2024-09-11T06:50:44.925 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.926 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.926 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.926 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723718+0000", 2024-09-11T06:50:44.926 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000141206 2024-09-11T06:50:44.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.926 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.926 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:44.926 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723982+0000", 2024-09-11T06:50:44.926 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026338499999999999 2024-09-11T06:50:44.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.926 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.926 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:44.926 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.725377+0000", 2024-09-11T06:50:44.927 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013956999999999999 2024-09-11T06:50:44.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.927 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.927 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:44.927 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.725908+0000", 2024-09-11T06:50:44.927 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00053036200000000004 2024-09-11T06:50:44.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.927 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.927 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.927 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.725999+0000", 2024-09-11T06:50:44.927 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1016999999999999e-05 2024-09-11T06:50:44.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.927 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.927 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.928 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726102+0000", 2024-09-11T06:50:44.928 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010341 2024-09-11T06:50:44.928 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.928 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.928 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.928 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:15 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:44.928 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.868921+0000", 2024-09-11T06:50:44.928 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 13.992591142, 2024-09-11T06:50:44.929 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0026879, 2024-09-11T06:50:44.929 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.929 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.929 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.929 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:44.929 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:44.929 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:44.929 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 15 2024-09-11T06:50:44.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.929 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.930 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.930 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868921+0000", 2024-09-11T06:50:44.930 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.930 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.930 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868921+0000", 2024-09-11T06:50:44.930 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.930 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.930 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868923+0000", 2024-09-11T06:50:44.930 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8959999999999999e-06 2024-09-11T06:50:44.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.931 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.931 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.931 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868934+0000", 2024-09-11T06:50:44.931 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1083e-05 2024-09-11T06:50:44.931 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.931 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.931 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.931 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868965+0000", 2024-09-11T06:50:44.931 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0833000000000001e-05 2024-09-11T06:50:44.931 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.931 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.931 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.931 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868990+0000", 2024-09-11T06:50:44.932 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4545999999999998e-05 2024-09-11T06:50:44.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.932 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.932 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.869124+0000", 2024-09-11T06:50:44.932 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013489599999999999 2024-09-11T06:50:44.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.932 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.932 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.869270+0000", 2024-09-11T06:50:44.932 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000145976 2024-09-11T06:50:44.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.932 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:44.933 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.869538+0000", 2024-09-11T06:50:44.933 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026780599999999999 2024-09-11T06:50:44.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.933 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:44.933 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.870691+0000", 2024-09-11T06:50:44.933 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00115277 2024-09-11T06:50:44.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.933 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:44.933 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.871456+0000", 2024-09-11T06:50:44.934 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00076473800000000005 2024-09-11T06:50:44.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.934 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.934 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.871543+0000", 2024-09-11T06:50:44.934 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7040999999999996e-05 2024-09-11T06:50:44.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.934 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.934 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.871609+0000", 2024-09-11T06:50:44.934 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6315000000000005e-05 2024-09-11T06:50:44.934 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.934 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.934 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.935 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.935 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:20 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:44.935 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.877429+0000", 2024-09-11T06:50:44.935 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 13.984082853, 2024-09-11T06:50:44.935 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032382629999999999, 2024-09-11T06:50:44.935 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.935 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.935 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.935 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:44.935 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:44.936 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:44.936 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 20 2024-09-11T06:50:44.936 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.936 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.936 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.936 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877429+0000", 2024-09-11T06:50:44.936 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.936 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.936 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.937 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877429+0000", 2024-09-11T06:50:44.937 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.937 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.937 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.937 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877431+0000", 2024-09-11T06:50:44.937 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9429999999999999e-06 2024-09-11T06:50:44.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.937 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.937 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.937 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877444+0000", 2024-09-11T06:50:44.937 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2396e-05 2024-09-11T06:50:44.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.938 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.938 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877474+0000", 2024-09-11T06:50:44.938 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0239999999999998e-05 2024-09-11T06:50:44.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.938 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.938 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877496+0000", 2024-09-11T06:50:44.938 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1914999999999999e-05 2024-09-11T06:50:44.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.939 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.939 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.939 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877616+0000", 2024-09-11T06:50:44.939 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000119814 2024-09-11T06:50:44.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.939 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.939 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.939 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877836+0000", 2024-09-11T06:50:44.939 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022029600000000001 2024-09-11T06:50:44.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.939 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.940 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:44.940 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.878482+0000", 2024-09-11T06:50:44.940 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00064605200000000004 2024-09-11T06:50:44.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.940 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:44.940 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.879804+0000", 2024-09-11T06:50:44.940 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013216460000000001 2024-09-11T06:50:44.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.941 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:44.941 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.880519+0000", 2024-09-11T06:50:44.941 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00071547600000000005 2024-09-11T06:50:44.941 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.941 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.941 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.941 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.880606+0000", 2024-09-11T06:50:44.941 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6907e-05 2024-09-11T06:50:44.941 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.941 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.941 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.941 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.880668+0000", 2024-09-11T06:50:44.941 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1577999999999999e-05 2024-09-11T06:50:44.942 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.942 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.942 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.942 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.942 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.942 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:23 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:44.942 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.885067+0000", 2024-09-11T06:50:44.942 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 13.976445199, 2024-09-11T06:50:44.942 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028127320000000001, 2024-09-11T06:50:44.942 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.942 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.942 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.942 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:44.943 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:44.943 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:44.943 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23 2024-09-11T06:50:44.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.943 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.943 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.943 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.943 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885067+0000", 2024-09-11T06:50:44.943 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.943 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.943 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.943 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885067+0000", 2024-09-11T06:50:44.943 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885069+0000", 2024-09-11T06:50:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2000000000000001e-06 2024-09-11T06:50:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885081+0000", 2024-09-11T06:50:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1209999999999999e-05 2024-09-11T06:50:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885111+0000", 2024-09-11T06:50:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0318999999999999e-05 2024-09-11T06:50:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885132+0000", 2024-09-11T06:50:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0649000000000002e-05 2024-09-11T06:50:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885243+0000", 2024-09-11T06:50:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000111284 2024-09-11T06:50:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885387+0000", 2024-09-11T06:50:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144222 2024-09-11T06:50:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885699+0000", 2024-09-11T06:50:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00031239800000000003 2024-09-11T06:50:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.886817+0000", 2024-09-11T06:50:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001117191 2024-09-11T06:50:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.887684+0000", 2024-09-11T06:50:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00086699499999999996 2024-09-11T06:50:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.887772+0000", 2024-09-11T06:50:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8224000000000001e-05 2024-09-11T06:50:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.887880+0000", 2024-09-11T06:50:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010804 2024-09-11T06:50:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "MOSDPGPull(2.0 e70/69 cost 1048576)", 2024-09-11T06:50:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:35.528435+0000", 2024-09-11T06:50:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.3330774630000004, 2024-09-11T06:50:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027390520000000001, 2024-09-11T06:50:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-11T06:50:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528435+0000", 2024-09-11T06:50:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528435+0000", 2024-09-11T06:50:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528437+0000", 2024-09-11T06:50:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5959999999999998e-06 2024-09-11T06:50:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528465+0000", 2024-09-11T06:50:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7436e-05 2024-09-11T06:50:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528499+0000", 2024-09-11T06:50:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3599000000000001e-05 2024-09-11T06:50:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528515+0000", 2024-09-11T06:50:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.677e-05 2024-09-11T06:50:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.952 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.952 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.952 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528652+0000", 2024-09-11T06:50:44.952 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000136546 2024-09-11T06:50:44.952 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.952 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.952 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.952 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.531174+0000", 2024-09-11T06:50:44.952 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0025221050000000002 2024-09-11T06:50:44.952 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.952 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.952 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.952 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "MOSDPGPull(2.0 e70/69 cost 1048576)", 2024-09-11T06:50:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:35.535973+0000", 2024-09-11T06:50:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.3255394710000008, 2024-09-11T06:50:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0020988000000000001, 2024-09-11T06:50:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-11T06:50:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.535973+0000", 2024-09-11T06:50:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.535973+0000", 2024-09-11T06:50:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.535975+0000", 2024-09-11T06:50:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.714e-06 2024-09-11T06:50:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.535994+0000", 2024-09-11T06:50:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9899e-05 2024-09-11T06:50:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.536023+0000", 2024-09-11T06:50:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8619000000000002e-05 2024-09-11T06:50:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.536036+0000", 2024-09-11T06:50:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.261e-05 2024-09-11T06:50:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.536164+0000", 2024-09-11T06:50:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012810999999999999 2024-09-11T06:50:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.538072+0000", 2024-09-11T06:50:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001907848 2024-09-11T06:50:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "MOSDPGPull(2.0 e70/69 cost 1048576)", 2024-09-11T06:50:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:35.607413+0000", 2024-09-11T06:50:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.2540993090000008, 2024-09-11T06:50:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0021226639999999998, 2024-09-11T06:50:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-11T06:50:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.607413+0000", 2024-09-11T06:50:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.607413+0000", 2024-09-11T06:50:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.607415+0000", 2024-09-11T06:50:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7069999999999999e-06 2024-09-11T06:50:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.607436+0000", 2024-09-11T06:50:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1151000000000001e-05 2024-09-11T06:50:44.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.959 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.959 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.959 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.607456+0000", 2024-09-11T06:50:44.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0559000000000001e-05 2024-09-11T06:50:44.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.959 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.959 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.959 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.607466+0000", 2024-09-11T06:50:44.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3419999999999997e-06 2024-09-11T06:50:44.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.959 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.959 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.960 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.607646+0000", 2024-09-11T06:50:44.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017984000000000001 2024-09-11T06:50:44.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.960 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.960 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.609536+0000", 2024-09-11T06:50:44.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001890065 2024-09-11T06:50:44.960 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.960 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.960 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.960 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4828.0:15 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-11T06:50:44.960 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:36.253730+0000", 2024-09-11T06:50:44.961 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.6077821199999995, 2024-09-11T06:50:44.961 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0025257589999999998, 2024-09-11T06:50:44.961 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.961 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.961 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.961 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:44.961 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4828", 2024-09-11T06:50:44.961 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2410288650", 2024-09-11T06:50:44.961 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 15 2024-09-11T06:50:44.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.961 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.961 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.961 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253730+0000", 2024-09-11T06:50:44.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253730+0000", 2024-09-11T06:50:44.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253732+0000", 2024-09-11T06:50:44.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2400000000000002e-06 2024-09-11T06:50:44.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.963 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253744+0000", 2024-09-11T06:50:44.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1316e-05 2024-09-11T06:50:44.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.963 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253771+0000", 2024-09-11T06:50:44.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.724e-05 2024-09-11T06:50:44.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.964 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.964 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.964 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253785+0000", 2024-09-11T06:50:44.964 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3786e-05 2024-09-11T06:50:44.964 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.964 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.964 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.964 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253886+0000", 2024-09-11T06:50:44.964 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000100981 2024-09-11T06:50:44.964 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.964 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.964 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.964 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.254029+0000", 2024-09-11T06:50:44.964 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014311900000000001 2024-09-11T06:50:44.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.965 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.965 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:44.965 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.254317+0000", 2024-09-11T06:50:44.965 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00028809300000000001 2024-09-11T06:50:44.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.965 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.965 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:44.965 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.255403+0000", 2024-09-11T06:50:44.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001085639 2024-09-11T06:50:44.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.966 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:44.966 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.256107+0000", 2024-09-11T06:50:44.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070468799999999997 2024-09-11T06:50:44.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.966 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.966 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.256191+0000", 2024-09-11T06:50:44.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3714999999999994e-05 2024-09-11T06:50:44.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.967 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.967 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.256256+0000", 2024-09-11T06:50:44.967 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4942000000000003e-05 2024-09-11T06:50:44.967 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.967 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.967 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.967 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5313.0:26 2.6 2:7b712441:ns::rbd_object_map.11adc335555f:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:50:44.967 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.260230+0000", 2024-09-11T06:50:44.968 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.6012827119999997, 2024-09-11T06:50:44.968 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002457201, 2024-09-11T06:50:44.968 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.968 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.968 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.968 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:44.968 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5313", 2024-09-11T06:50:44.968 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/624314229", 2024-09-11T06:50:44.968 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:50:44.968 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.968 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.968 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.969 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.969 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260230+0000", 2024-09-11T06:50:44.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.969 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.969 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260230+0000", 2024-09-11T06:50:44.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.970 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.970 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260232+0000", 2024-09-11T06:50:44.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.956e-06 2024-09-11T06:50:44.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.970 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.970 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260243+0000", 2024-09-11T06:50:44.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1687e-05 2024-09-11T06:50:44.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260269+0000", 2024-09-11T06:50:44.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6211999999999999e-05 2024-09-11T06:50:44.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260284+0000", 2024-09-11T06:50:44.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4053000000000001e-05 2024-09-11T06:50:44.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260391+0000", 2024-09-11T06:50:44.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000107278 2024-09-11T06:50:44.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260541+0000", 2024-09-11T06:50:44.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015033000000000001 2024-09-11T06:50:44.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 3", 2024-09-11T06:50:44.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260951+0000", 2024-09-11T06:50:44.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040981299999999997 2024-09-11T06:50:44.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:44.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262399+0000", 2024-09-11T06:50:44.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014475930000000001 2024-09-11T06:50:44.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:44.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262531+0000", 2024-09-11T06:50:44.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013239 2024-09-11T06:50:44.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262623+0000", 2024-09-11T06:50:44.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2361000000000006e-05 2024-09-11T06:50:44.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262687+0000", 2024-09-11T06:50:44.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3528000000000006e-05 2024-09-11T06:50:44.974 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.974 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.974 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.974 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5313.0:31 2.6 2:7b712441:ns::rbd_object_map.11adc335555f:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:50:44.974 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.266495+0000", 2024-09-11T06:50:44.975 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.5950169379999997, 2024-09-11T06:50:44.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0026763870000000001, 2024-09-11T06:50:44.975 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:44.975 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:44.975 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:44.975 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:44.975 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5313", 2024-09-11T06:50:44.975 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/624314229", 2024-09-11T06:50:44.975 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 31 2024-09-11T06:50:44.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.975 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:44.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:44.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266495+0000", 2024-09-11T06:50:44.976 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.976 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.976 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:44.976 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266495+0000", 2024-09-11T06:50:44.976 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:44.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.976 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.976 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:44.976 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266498+0000", 2024-09-11T06:50:44.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2369999999999999e-06 2024-09-11T06:50:44.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:44.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266509+0000", 2024-09-11T06:50:44.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1459e-05 2024-09-11T06:50:44.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:44.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266540+0000", 2024-09-11T06:50:44.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1189999999999998e-05 2024-09-11T06:50:44.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.978 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:44.978 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266563+0000", 2024-09-11T06:50:44.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2529000000000001e-05 2024-09-11T06:50:44.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.978 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:44.978 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266708+0000", 2024-09-11T06:50:44.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014511 2024-09-11T06:50:44.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:44.979 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266962+0000", 2024-09-11T06:50:44.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000253897 2024-09-11T06:50:44.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 3", 2024-09-11T06:50:44.979 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.267430+0000", 2024-09-11T06:50:44.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00046799899999999998 2024-09-11T06:50:44.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:44.980 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.268765+0000", 2024-09-11T06:50:44.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013350359999999999 2024-09-11T06:50:44.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.980 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.980 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:44.980 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.269019+0000", 2024-09-11T06:50:44.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025397000000000001 2024-09-11T06:50:44.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.980 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.980 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:44.980 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.269105+0000", 2024-09-11T06:50:44.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5952000000000006e-05 2024-09-11T06:50:44.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:44.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:44.981 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:44.981 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.269172+0000", 2024-09-11T06:50:44.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7008000000000006e-05 2024-09-11T06:50:44.981 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.981 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.981 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.981 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:44.981 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:44.981 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:44.984 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:50:44.984 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^23447\b' 2024-09-11T06:50:44.984 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^23447\b' 2024-09-11T06:50:44.984 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:44.984 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -C rbd-nbd 2024-09-11T06:50:44.984 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^ *23447\b' 2024-09-11T06:50:44.984 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^ *23447\b' 2024-09-11T06:50:44.985 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:50:44.985 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:50:44.986 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:50:44.986 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2024-09-11T06:50:44.292486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":134808318,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":2895,"num_read_kb":110462,"num_write":714,"num_write_kb":264747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":174328190,"num_keys_recovered":22,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":653,"ondisk_log_size":653,"up":34,"acting":34,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":423768,"kb_used_data":267796,"kb_used_omap":7,"kb_used_meta":155832,"kb_avail":471435432,"statfs":{"total":483183820800,"available":482749882368,"internally_reserved":0,"allocated":274223104,"data_stored":271332422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7970,"internal_metadata":159572190},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":38,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":582,"num_read_kb":6494,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.041991"},"pg_stats":[{"pgid":"1.0","version":"14'117","reported_seq":294,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.273698+0000","last_change":"2024-09-11T06:50:37.941212+0000","last_active":"2024-09-11T06:50:43.273698+0000","last_peered":"2024-09-11T06:50:43.273698+0000","last_clean":"2024-09-11T06:50:43.273698+0000","last_became_active":"2024-09-11T06:50:37.940448+0000","last_became_peered":"2024-09-11T06:50:37.940448+0000","last_unstale":"2024-09-11T06:50:43.273698+0000","last_undegraded":"2024-09-11T06:50:43.273698+0000","last_fullsized":"2024-09-11T06:50:43.273698+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_clean_scrub_stamp":"2024-09-11T06:49:05.088839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:48:59.833858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"72'39","reported_seq":257,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.274840+0000","last_change":"2024-09-11T06:50:29.249902+0000","last_active":"2024-09-11T06:50:43.274840+0000","last_peered":"2024-09-11T06:50:43.274840+0000","last_clean":"2024-09-11T06:50:43.274840+0000","last_became_active":"2024-09-11T06:50:29.172399+0000","last_became_peered":"2024-09-11T06:50:29.172399+0000","last_unstale":"2024-09-11T06:50:43.274840+0000","last_undegraded":"2024-09-11T06:50:43.274840+0000","last_fullsized":"2024-09-11T06:50:43.274840+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:25:03.908704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011212469999999999,"stat_sum":{"num_bytes":12582950,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":74,"num_read_kb":8198,"num_write":38,"num_write_kb":28673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":12582912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"58'2","reported_seq":79,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.508598+0000","last_change":"2024-09-11T06:50:35.245582+0000","last_active":"2024-09-11T06:50:43.508598+0000","last_peered":"2024-09-11T06:50:43.508598+0000","last_clean":"2024-09-11T06:50:43.508598+0000","last_became_active":"2024-09-11T06:50:35.240946+0000","last_became_peered":"2024-09-11T06:50:35.240946+0000","last_unstale":"2024-09-11T06:50:43.508598+0000","last_undegraded":"2024-09-11T06:50:43.508598+0000","last_fullsized":"2024-09-11T06:50:43.508598+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:17:27.363513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011764950000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"68'57","reported_seq":187,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.273817+0000","last_change":"2024-09-11T06:50:35.668519+0000","last_active":"2024-09-11T06:50:43.273817+0000","last_peered":"2024-09-11T06:50:43.273817+0000","last_clean":"2024-09-11T06:50:43.273817+0000","last_became_active":"2024-09-11T06:50:35.238279+0000","last_became_peered":"2024-09-11T06:50:35.238279+0000","last_unstale":"2024-09-11T06:50:43.273817+0000","last_undegraded":"2024-09-11T06:50:43.273817+0000","last_fullsized":"2024-09-11T06:50:43.273817+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:14:06.141440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077615599999999996,"stat_sum":{"num_bytes":20971520,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":150,"num_read_kb":23060,"num_write":56,"num_write_kb":45056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":41943040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":58,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.372412+0000","last_change":"2024-09-11T06:50:19.703255+0000","last_active":"2024-09-11T06:50:43.372412+0000","last_peered":"2024-09-11T06:50:43.372412+0000","last_clean":"2024-09-11T06:50:43.372412+0000","last_became_active":"2024-09-11T06:50:17.089296+0000","last_became_peered":"2024-09-11T06:50:17.089296+0000","last_unstale":"2024-09-11T06:50:43.372412+0000","last_undegraded":"2024-09-11T06:50:43.372412+0000","last_fullsized":"2024-09-11T06:50:43.372412+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:42:18.684413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001159373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"78'106","reported_seq":982,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.275047+0000","last_change":"2024-09-11T06:49:55.948809+0000","last_active":"2024-09-11T06:50:43.275047+0000","last_peered":"2024-09-11T06:50:43.275047+0000","last_clean":"2024-09-11T06:50:43.275047+0000","last_became_active":"2024-09-11T06:49:55.948499+0000","last_became_peered":"2024-09-11T06:49:55.948499+0000","last_unstale":"2024-09-11T06:50:43.275047+0000","last_undegraded":"2024-09-11T06:50:43.275047+0000","last_fullsized":"2024-09-11T06:50:43.275047+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:14:11.313635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010163780000000001,"stat_sum":{"num_bytes":25165862,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1245,"num_read_kb":22324,"num_write":62,"num_write_kb":40966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":59,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.508536+0000","last_change":"2024-09-11T06:50:19.097182+0000","last_active":"2024-09-11T06:50:43.508536+0000","last_peered":"2024-09-11T06:50:43.508536+0000","last_clean":"2024-09-11T06:50:43.508536+0000","last_became_active":"2024-09-11T06:50:17.097783+0000","last_became_peered":"2024-09-11T06:50:17.097783+0000","last_unstale":"2024-09-11T06:50:43.508536+0000","last_undegraded":"2024-09-11T06:50:43.508536+0000","last_fullsized":"2024-09-11T06:50:43.508536+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:35:49.244377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088028100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"54'4","reported_seq":313,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.508703+0000","last_change":"2024-09-11T06:50:29.230433+0000","last_active":"2024-09-11T06:50:43.508703+0000","last_peered":"2024-09-11T06:50:43.508703+0000","last_clean":"2024-09-11T06:50:43.508703+0000","last_became_active":"2024-09-11T06:50:29.171189+0000","last_became_peered":"2024-09-11T06:50:29.171189+0000","last_unstale":"2024-09-11T06:50:43.508703+0000","last_undegraded":"2024-09-11T06:50:43.508703+0000","last_fullsized":"2024-09-11T06:50:43.508703+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:19.442991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001910244,"stat_sum":{"num_bytes":8388627,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":152,"num_read_kb":2667,"num_write":4,"num_write_kb":8194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":8388684,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"56'1","reported_seq":59,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.372426+0000","last_change":"2024-09-11T06:50:19.703261+0000","last_active":"2024-09-11T06:50:43.372426+0000","last_peered":"2024-09-11T06:50:43.372426+0000","last_clean":"2024-09-11T06:50:43.372426+0000","last_became_active":"2024-09-11T06:50:17.089261+0000","last_became_peered":"2024-09-11T06:50:17.089261+0000","last_unstale":"2024-09-11T06:50:43.372426+0000","last_undegraded":"2024-09-11T06:50:43.372426+0000","last_fullsized":"2024-09-11T06:50:43.372426+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:00:38.491173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001141853,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.274862+0000","last_change":"2024-09-11T06:50:35.238915+0000","last_active":"2024-09-11T06:50:43.274862+0000","last_peered":"2024-09-11T06:50:43.274862+0000","last_clean":"2024-09-11T06:50:43.274862+0000","last_became_active":"2024-09-11T06:50:35.238289+0000","last_became_peered":"2024-09-11T06:50:35.238289+0000","last_unstale":"2024-09-11T06:50:43.274862+0000","last_undegraded":"2024-09-11T06:50:43.274862+0000","last_fullsized":"2024-09-11T06:50:43.274862+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:13:46.102794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"72'106","reported_seq":263,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.508761+0000","last_change":"2024-09-11T06:49:56.122331+0000","last_active":"2024-09-11T06:50:43.508761+0000","last_peered":"2024-09-11T06:50:43.508761+0000","last_clean":"2024-09-11T06:50:43.508761+0000","last_became_active":"2024-09-11T06:49:56.121998+0000","last_became_peered":"2024-09-11T06:49:56.121998+0000","last_unstale":"2024-09-11T06:50:43.508761+0000","last_undegraded":"2024-09-11T06:50:43.508761+0000","last_fullsized":"2024-09-11T06:50:43.508761+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:24:38.234463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582950,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":501,"num_read_kb":8581,"num_write":170,"num_write_kb":20614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":59,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.508648+0000","last_change":"2024-09-11T06:50:19.097504+0000","last_active":"2024-09-11T06:50:43.508648+0000","last_peered":"2024-09-11T06:50:43.508648+0000","last_clean":"2024-09-11T06:50:43.508648+0000","last_became_active":"2024-09-11T06:50:17.096243+0000","last_became_peered":"2024-09-11T06:50:17.096243+0000","last_unstale":"2024-09-11T06:50:43.508648+0000","last_undegraded":"2024-09-11T06:50:43.508648+0000","last_fullsized":"2024-09-11T06:50:43.508648+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:44:45.211005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085501399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"70'66","reported_seq":157,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.273743+0000","last_change":"2024-09-11T06:50:35.337013+0000","last_active":"2024-09-11T06:50:43.273743+0000","last_peered":"2024-09-11T06:50:43.273743+0000","last_clean":"2024-09-11T06:50:43.273743+0000","last_became_active":"2024-09-11T06:50:35.236333+0000","last_became_peered":"2024-09-11T06:50:35.236333+0000","last_unstale":"2024-09-11T06:50:43.273743+0000","last_undegraded":"2024-09-11T06:50:43.273743+0000","last_fullsized":"2024-09-11T06:50:43.273743+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:40.624982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095172900000000001,"stat_sum":{"num_bytes":20971552,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":274,"num_read_kb":16513,"num_write":66,"num_write_kb":36867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":41943104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":59,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.275130+0000","last_change":"2024-09-11T06:50:19.096344+0000","last_active":"2024-09-11T06:50:43.275130+0000","last_peered":"2024-09-11T06:50:43.275130+0000","last_clean":"2024-09-11T06:50:43.275130+0000","last_became_active":"2024-09-11T06:50:17.096560+0000","last_became_peered":"2024-09-11T06:50:17.096560+0000","last_unstale":"2024-09-11T06:50:43.275130+0000","last_undegraded":"2024-09-11T06:50:43.275130+0000","last_fullsized":"2024-09-11T06:50:43.275130+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:03:19.807744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053029399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"74'43","reported_seq":199,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.508813+0000","last_change":"2024-09-11T06:50:35.320796+0000","last_active":"2024-09-11T06:50:43.508813+0000","last_peered":"2024-09-11T06:50:43.508813+0000","last_clean":"2024-09-11T06:50:43.508813+0000","last_became_active":"2024-09-11T06:50:35.241589+0000","last_became_peered":"2024-09-11T06:50:35.241589+0000","last_unstale":"2024-09-11T06:50:43.508813+0000","last_undegraded":"2024-09-11T06:50:43.508813+0000","last_fullsized":"2024-09-11T06:50:43.508813+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:20.312915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388646,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":118,"num_read_kb":8286,"num_write":59,"num_write_kb":16420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":16777292,"num_keys_recovered":6,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":59,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.273135+0000","last_change":"2024-09-11T06:50:19.688101+0000","last_active":"2024-09-11T06:50:43.273135+0000","last_peered":"2024-09-11T06:50:43.273135+0000","last_clean":"2024-09-11T06:50:43.273135+0000","last_became_active":"2024-09-11T06:50:17.096959+0000","last_became_peered":"2024-09-11T06:50:17.096959+0000","last_unstale":"2024-09-11T06:50:43.273135+0000","last_undegraded":"2024-09-11T06:50:43.273135+0000","last_fullsized":"2024-09-11T06:50:43.273135+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:54:30.491263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059833200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"78'112","reported_seq":142,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-11T06:50:43.273818+0000","last_change":"2024-09-11T06:50:35.519269+0000","last_active":"2024-09-11T06:50:43.273818+0000","last_peered":"2024-09-11T06:50:43.273818+0000","last_clean":"2024-09-11T06:50:43.273818+0000","last_became_active":"2024-09-11T06:50:35.238011+0000","last_became_peered":"2024-09-11T06:50:35.238011+0000","last_unstale":"2024-09-11T06:50:43.273818+0000","last_undegraded":"2024-09-11T06:50:43.273818+0000","last_fullsized":"2024-09-11T06:50:43.273818+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:23:49.359046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001403348,"stat_sum":{"num_bytes":25165824,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":265,"num_read_kb":20733,"num_write":98,"num_write_kb":65539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":50331648,"num_keys_recovered":16,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":134217931,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":2779,"num_read_kb":110362,"num_write":553,"num_write_kb":262329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":171966680,"num_keys_recovered":22,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":289468416,"data_stored":289407401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":533,"ondisk_log_size":533,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":0,"seq":47244640278,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640279,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56660,"kb_used_data":25464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94315180,"statfs":{"total":96636764160,"available":96578744320,"internally_reserved":0,"allocated":26075136,"data_stored":25509971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640279,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":106400,"kb_used_data":75212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94265440,"statfs":{"total":96636764160,"available":96527810560,"internally_reserved":0,"allocated":77017088,"data_stored":76432076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640280,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68956,"kb_used_data":37760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94302884,"statfs":{"total":96636764160,"available":96566153216,"internally_reserved":0,"allocated":38666240,"data_stored":38092959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640280,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118108,"kb_used_data":86916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94253732,"statfs":{"total":96636764160,"available":96515821568,"internally_reserved":0,"allocated":89001984,"data_stored":88419772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640281,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73644,"kb_used_data":42444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94298196,"statfs":{"total":96636764160,"available":96561352704,"internally_reserved":0,"allocated":43462656,"data_stored":42877644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":41959424,"data_stored":41943129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":88100864,"data_stored":88080555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":37756928,"data_stored":37748812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":75509760,"data_stored":75497542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":25165824,"data_stored":25165824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20975616,"data_stored":20971539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:50:44.987 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-11T06:50:44.987 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T06:50:44.988 INFO:tasks.thrashosds.thrasher:Adding osd 5 2024-09-11T06:50:44.988 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 5 2024-09-11T06:50:44.988 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:44.979+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:44.990 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:44.991 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 0.5 2024-09-11T06:50:45.084 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:45.079+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:45.153 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:45.154 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:45.154 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:50:45.154 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:45.167 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:50:45.185 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:45.183+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:45.286 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:45.283+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:45.321 INFO:tasks.ceph.mon.a.smithi071.stderr:2024-09-11T06:50:45.315+0000 7f1296554640 -1 mon.a@0(leader).osd e79 definitely_dead 0 2024-09-11T06:50:45.328 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:45.328 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:45.328 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:50:45.328 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:50:45.328 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:45.341 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:50:45.387 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:45.383+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:45.488 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:45.483+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:45.493 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV= 2024-09-11T06:50:45.493 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd config global set global rbd_default_pool new_default_pool21607 2024-09-11T06:50:45.505 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:45.505 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:50:45.505 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:50:45.505 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:50:45.505 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.505 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch watch cookie 139735284585312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:45.506 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.521153+0000", 2024-09-11T06:50:45.506 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.979575369999999, 2024-09-11T06:50:45.506 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028996400000000002, 2024-09-11T06:50:45.506 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:45.506 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:45.506 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:45.506 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:45.506 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:45.506 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:45.506 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:50:45.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.506 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:45.506 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.507 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:45.507 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521153+0000", 2024-09-11T06:50:45.507 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.507 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:45.507 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521153+0000", 2024-09-11T06:50:45.507 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.507 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:45.507 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521154+0000", 2024-09-11T06:50:45.507 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.77e-06 2024-09-11T06:50:45.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:45.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521162+0000", 2024-09-11T06:50:45.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6569999999999997e-06 2024-09-11T06:50:45.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:45.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521184+0000", 2024-09-11T06:50:45.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1736e-05 2024-09-11T06:50:45.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:45.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521200+0000", 2024-09-11T06:50:45.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5953000000000001e-05 2024-09-11T06:50:45.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:45.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521328+0000", 2024-09-11T06:50:45.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000128489 2024-09-11T06:50:45.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:45.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521565+0000", 2024-09-11T06:50:45.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000236357 2024-09-11T06:50:45.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:45.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521991+0000", 2024-09-11T06:50:45.510 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00042691099999999998 2024-09-11T06:50:45.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.510 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:45.510 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.523245+0000", 2024-09-11T06:50:45.510 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001253964 2024-09-11T06:50:45.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.510 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:45.510 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.523847+0000", 2024-09-11T06:50:45.510 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00060132899999999995 2024-09-11T06:50:45.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.511 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:45.511 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.523938+0000", 2024-09-11T06:50:45.511 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1577999999999996e-05 2024-09-11T06:50:45.511 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.511 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.511 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:45.511 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.524052+0000", 2024-09-11T06:50:45.511 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000113896 2024-09-11T06:50:45.511 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.511 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:45.511 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.511 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.511 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.511 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch watch cookie 140250278007648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:45.511 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.701683+0000", 2024-09-11T06:50:45.512 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.799044755000001, 2024-09-11T06:50:45.512 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002794693, 2024-09-11T06:50:45.512 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:45.512 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:45.512 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:45.512 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:45.512 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:45.512 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:45.512 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:50:45.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.512 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:45.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.512 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:45.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.701683+0000", 2024-09-11T06:50:45.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.513 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:45.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.701683+0000", 2024-09-11T06:50:45.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.513 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:45.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.701685+0000", 2024-09-11T06:50:45.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6500000000000001e-06 2024-09-11T06:50:45.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.514 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:45.514 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.701694+0000", 2024-09-11T06:50:45.514 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0759999999999992e-06 2024-09-11T06:50:45.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.514 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.514 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:45.514 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.701717+0000", 2024-09-11T06:50:45.514 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.349e-05 2024-09-11T06:50:45.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.514 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.514 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:45.514 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.701730+0000", 2024-09-11T06:50:45.515 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2649e-05 2024-09-11T06:50:45.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.515 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:45.515 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.701853+0000", 2024-09-11T06:50:45.515 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012253000000000001 2024-09-11T06:50:45.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.515 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:45.515 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.702079+0000", 2024-09-11T06:50:45.515 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022646999999999999 2024-09-11T06:50:45.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.516 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:45.516 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.702508+0000", 2024-09-11T06:50:45.516 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00042935500000000003 2024-09-11T06:50:45.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.516 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.516 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:45.516 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.703905+0000", 2024-09-11T06:50:45.516 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001396102 2024-09-11T06:50:45.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.516 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.516 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:45.516 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.704285+0000", 2024-09-11T06:50:45.516 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000380436 2024-09-11T06:50:45.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.517 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:45.517 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.704374+0000", 2024-09-11T06:50:45.517 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8813999999999994e-05 2024-09-11T06:50:45.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.517 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:45.517 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.704478+0000", 2024-09-11T06:50:45.517 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000104121 2024-09-11T06:50:45.517 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.517 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:45.517 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.518 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:14 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:45.518 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.706169+0000", 2024-09-11T06:50:45.518 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.794558701, 2024-09-11T06:50:45.518 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0029019599999999999, 2024-09-11T06:50:45.518 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:45.518 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:45.518 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:45.518 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:45.518 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:45.518 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:45.518 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 14 2024-09-11T06:50:45.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.519 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:45.519 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.519 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:45.519 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706169+0000", 2024-09-11T06:50:45.519 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.519 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.519 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.519 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:45.519 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706169+0000", 2024-09-11T06:50:45.519 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.519 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.519 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.519 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:45.520 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706172+0000", 2024-09-11T06:50:45.520 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.34e-06 2024-09-11T06:50:45.520 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.520 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.520 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:45.520 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706183+0000", 2024-09-11T06:50:45.520 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1556000000000001e-05 2024-09-11T06:50:45.520 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.520 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.520 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:45.520 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706215+0000", 2024-09-11T06:50:45.520 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2116000000000002e-05 2024-09-11T06:50:45.521 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.521 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.521 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:45.521 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706232+0000", 2024-09-11T06:50:45.521 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7073000000000001e-05 2024-09-11T06:50:45.521 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.521 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.521 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:45.521 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706356+0000", 2024-09-11T06:50:45.521 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012411599999999999 2024-09-11T06:50:45.521 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.521 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.521 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:45.521 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706591+0000", 2024-09-11T06:50:45.522 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000234897 2024-09-11T06:50:45.522 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.522 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.522 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:45.522 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.707060+0000", 2024-09-11T06:50:45.522 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00046906299999999998 2024-09-11T06:50:45.522 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.522 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.522 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:45.522 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.708179+0000", 2024-09-11T06:50:45.522 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001118232 2024-09-11T06:50:45.522 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.522 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.522 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:45.523 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.708894+0000", 2024-09-11T06:50:45.523 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00071581900000000003 2024-09-11T06:50:45.523 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.523 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.523 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:45.523 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.708983+0000", 2024-09-11T06:50:45.523 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8270999999999996e-05 2024-09-11T06:50:45.523 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.523 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.523 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:45.523 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709071+0000", 2024-09-11T06:50:45.524 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8477e-05 2024-09-11T06:50:45.524 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.524 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:45.524 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.524 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.524 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.524 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:20 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:45.524 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.717149+0000", 2024-09-11T06:50:45.524 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.783578533, 2024-09-11T06:50:45.524 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002913297, 2024-09-11T06:50:45.524 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:45.524 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:45.525 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:45.525 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:45.525 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:45.525 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:45.525 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 20 2024-09-11T06:50:45.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.525 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:45.525 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.525 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:45.525 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717149+0000", 2024-09-11T06:50:45.525 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.525 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.525 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:45.526 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717149+0000", 2024-09-11T06:50:45.526 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.526 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:45.526 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717152+0000", 2024-09-11T06:50:45.526 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.21e-06 2024-09-11T06:50:45.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.526 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:45.526 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717164+0000", 2024-09-11T06:50:45.526 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1912999999999999e-05 2024-09-11T06:50:45.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.527 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:45.527 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717191+0000", 2024-09-11T06:50:45.527 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7526e-05 2024-09-11T06:50:45.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.527 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.527 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:45.527 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717206+0000", 2024-09-11T06:50:45.527 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5113000000000001e-05 2024-09-11T06:50:45.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.527 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.527 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:45.527 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717355+0000", 2024-09-11T06:50:45.527 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014896500000000001 2024-09-11T06:50:45.528 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.528 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.528 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:45.528 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717607+0000", 2024-09-11T06:50:45.528 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000252017 2024-09-11T06:50:45.528 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.528 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.528 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:45.528 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.718128+0000", 2024-09-11T06:50:45.528 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00052114499999999998 2024-09-11T06:50:45.528 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.528 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.528 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:45.528 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.719432+0000", 2024-09-11T06:50:45.529 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013041629999999999 2024-09-11T06:50:45.529 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.529 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.529 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:45.529 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.719864+0000", 2024-09-11T06:50:45.529 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00043115299999999998 2024-09-11T06:50:45.529 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.529 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.529 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:45.529 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.719949+0000", 2024-09-11T06:50:45.529 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4862000000000002e-05 2024-09-11T06:50:45.529 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.529 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.529 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:45.530 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.720063+0000", 2024-09-11T06:50:45.530 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011423 2024-09-11T06:50:45.530 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.530 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:45.530 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.530 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.530 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:24 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:45.530 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.726441+0000", 2024-09-11T06:50:45.530 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.774287036, 2024-09-11T06:50:45.530 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027674599999999998, 2024-09-11T06:50:45.530 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:45.530 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:45.530 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:45.531 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:45.531 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:45.531 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:45.531 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 24 2024-09-11T06:50:45.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.531 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:45.531 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.531 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:45.531 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726441+0000", 2024-09-11T06:50:45.531 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.531 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.531 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:45.531 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726441+0000", 2024-09-11T06:50:45.532 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.532 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.532 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:45.532 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726443+0000", 2024-09-11T06:50:45.532 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3769999999999999e-06 2024-09-11T06:50:45.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.532 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.532 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:45.532 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726456+0000", 2024-09-11T06:50:45.532 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2616000000000001e-05 2024-09-11T06:50:45.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.532 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.533 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:45.533 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726485+0000", 2024-09-11T06:50:45.533 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9195000000000001e-05 2024-09-11T06:50:45.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.533 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.533 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:45.533 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726499+0000", 2024-09-11T06:50:45.533 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4087000000000001e-05 2024-09-11T06:50:45.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.533 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.533 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:45.533 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726618+0000", 2024-09-11T06:50:45.533 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000118337 2024-09-11T06:50:45.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.534 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.534 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:45.534 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726772+0000", 2024-09-11T06:50:45.534 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015453799999999999 2024-09-11T06:50:45.534 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.534 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.534 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:45.534 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.727048+0000", 2024-09-11T06:50:45.534 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00027612900000000001 2024-09-11T06:50:45.534 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.534 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.534 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:45.534 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.728406+0000", 2024-09-11T06:50:45.535 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013579880000000001 2024-09-11T06:50:45.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.535 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.535 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:45.535 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.728878+0000", 2024-09-11T06:50:45.535 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00047132599999999999 2024-09-11T06:50:45.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.535 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.535 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:45.535 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729044+0000", 2024-09-11T06:50:45.535 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016672900000000001 2024-09-11T06:50:45.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.535 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.535 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:45.535 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729208+0000", 2024-09-11T06:50:45.536 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016413800000000001 2024-09-11T06:50:45.536 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.536 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:45.536 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.536 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.536 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:26 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 140250278007648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:45.536 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:50:45.536 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.771095398, 2024-09-11T06:50:45.537 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034645140000000001, 2024-09-11T06:50:45.537 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:45.537 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:45.537 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:45.537 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:45.537 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:45.537 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:45.537 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:50:45.537 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.537 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:45.537 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.537 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:45.537 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:50:45.537 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.537 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.538 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:45.538 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:50:45.538 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.538 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:45.538 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729634+0000", 2024-09-11T06:50:45.538 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0470000000000001e-06 2024-09-11T06:50:45.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.538 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:45.538 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729640+0000", 2024-09-11T06:50:45.538 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8860000000000001e-06 2024-09-11T06:50:45.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.539 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:45.539 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729654+0000", 2024-09-11T06:50:45.539 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:50:45.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.539 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:45.539 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729664+0000", 2024-09-11T06:50:45.539 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.594e-06 2024-09-11T06:50:45.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.539 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:45.540 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730422+0000", 2024-09-11T06:50:45.540 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000758648 2024-09-11T06:50:45.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.540 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:45.540 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730571+0000", 2024-09-11T06:50:45.540 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000148932 2024-09-11T06:50:45.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.540 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:45.540 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730936+0000", 2024-09-11T06:50:45.540 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036510900000000002 2024-09-11T06:50:45.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.541 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:45.541 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732159+0000", 2024-09-11T06:50:45.541 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001222302 2024-09-11T06:50:45.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.541 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.541 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:45.541 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732763+0000", 2024-09-11T06:50:45.541 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00060421599999999998 2024-09-11T06:50:45.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.541 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.541 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:45.541 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732916+0000", 2024-09-11T06:50:45.541 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015264 2024-09-11T06:50:45.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.542 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:45.542 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733097+0000", 2024-09-11T06:50:45.542 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000181504 2024-09-11T06:50:45.542 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.542 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:45.542 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.542 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:27 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:45.542 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:50:45.542 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.770121496, 2024-09-11T06:50:45.542 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003072852, 2024-09-11T06:50:45.543 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:45.543 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:45.543 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:45.543 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:45.543 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:45.543 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:45.543 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27 2024-09-11T06:50:45.543 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.543 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:45.543 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.543 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:45.543 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:50:45.543 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.543 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.544 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:45.544 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:50:45.544 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.544 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:45.544 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730608+0000", 2024-09-11T06:50:45.544 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8369999999999999e-06 2024-09-11T06:50:45.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.544 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:45.544 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730617+0000", 2024-09-11T06:50:45.544 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1859999999999992e-06 2024-09-11T06:50:45.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.545 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:45.545 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730636+0000", 2024-09-11T06:50:45.545 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9296e-05 2024-09-11T06:50:45.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.545 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:45.545 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730674+0000", 2024-09-11T06:50:45.545 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8609000000000003e-05 2024-09-11T06:50:45.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.545 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:45.545 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731496+0000", 2024-09-11T06:50:45.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00082112299999999999 2024-09-11T06:50:45.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.546 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:45.546 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731722+0000", 2024-09-11T06:50:45.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022651400000000001 2024-09-11T06:50:45.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.546 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:45.546 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733215+0000", 2024-09-11T06:50:45.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001493257 2024-09-11T06:50:45.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.546 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:45.546 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733452+0000", 2024-09-11T06:50:45.547 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023631600000000001 2024-09-11T06:50:45.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.547 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:45.547 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733679+0000", 2024-09-11T06:50:45.547 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022771400000000001 2024-09-11T06:50:45.547 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.547 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:45.547 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.547 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:73 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:45.547 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:50:45.547 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.770037212, 2024-09-11T06:50:45.548 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033454439999999999, 2024-09-11T06:50:45.548 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:45.548 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:45.548 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:45.548 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:45.548 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:45.548 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:45.548 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73 2024-09-11T06:50:45.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.548 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:45.548 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.548 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:45.548 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:50:45.548 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:45.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:50:45.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:45.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730694+0000", 2024-09-11T06:50:45.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5099999999999999e-06 2024-09-11T06:50:45.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:45.550 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730708+0000", 2024-09-11T06:50:45.550 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3237e-05 2024-09-11T06:50:45.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.550 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:45.550 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730738+0000", 2024-09-11T06:50:45.550 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0688999999999997e-05 2024-09-11T06:50:45.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.550 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:45.551 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730759+0000", 2024-09-11T06:50:45.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0818999999999998e-05 2024-09-11T06:50:45.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.551 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:45.551 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731841+0000", 2024-09-11T06:50:45.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010821819999999999 2024-09-11T06:50:45.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.551 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:45.551 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732092+0000", 2024-09-11T06:50:45.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025086299999999999 2024-09-11T06:50:45.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.552 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.552 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:45.552 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733730+0000", 2024-09-11T06:50:45.552 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0016374600000000001 2024-09-11T06:50:45.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.552 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.552 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:45.552 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733866+0000", 2024-09-11T06:50:45.552 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000136666 2024-09-11T06:50:45.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.552 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.552 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:45.552 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.734036+0000", 2024-09-11T06:50:45.553 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000170018 2024-09-11T06:50:45.553 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.553 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:45.553 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.553 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch watch cookie 139740787512064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:45.553 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.861993+0000", 2024-09-11T06:50:45.553 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.638735046000001, 2024-09-11T06:50:45.553 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027042189999999999, 2024-09-11T06:50:45.553 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:45.553 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:45.553 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:45.553 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:45.554 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:45.554 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:45.554 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:50:45.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.554 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:45.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.554 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:45.554 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.861993+0000", 2024-09-11T06:50:45.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.554 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:45.554 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.861993+0000", 2024-09-11T06:50:45.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.555 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.555 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:45.555 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.861995+0000", 2024-09-11T06:50:45.555 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2529999999999999e-06 2024-09-11T06:50:45.555 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.555 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.555 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:45.555 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862006+0000", 2024-09-11T06:50:45.555 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0509999999999999e-05 2024-09-11T06:50:45.555 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.555 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.555 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:45.555 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862037+0000", 2024-09-11T06:50:45.555 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1396000000000002e-05 2024-09-11T06:50:45.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.556 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.556 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:45.556 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862056+0000", 2024-09-11T06:50:45.556 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.859e-05 2024-09-11T06:50:45.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.556 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.556 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:45.556 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862168+0000", 2024-09-11T06:50:45.556 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000111996 2024-09-11T06:50:45.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.556 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.556 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:45.556 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862304+0000", 2024-09-11T06:50:45.557 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000136716 2024-09-11T06:50:45.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.557 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.557 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:45.557 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862544+0000", 2024-09-11T06:50:45.557 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024005000000000001 2024-09-11T06:50:45.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.557 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.557 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:45.557 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.863704+0000", 2024-09-11T06:50:45.558 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011596499999999999 2024-09-11T06:50:45.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.558 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.558 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:45.558 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.864508+0000", 2024-09-11T06:50:45.558 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00080411999999999997 2024-09-11T06:50:45.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.558 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.558 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:45.558 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.864597+0000", 2024-09-11T06:50:45.558 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8660999999999997e-05 2024-09-11T06:50:45.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.558 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.558 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:45.559 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.864697+0000", 2024-09-11T06:50:45.559 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000100277 2024-09-11T06:50:45.559 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.559 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:45.559 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.559 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.559 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.559 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:18 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:45.559 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:50:45.559 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.626620022999999, 2024-09-11T06:50:45.559 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0031646579999999999, 2024-09-11T06:50:45.559 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:45.559 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:45.559 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:45.559 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:45.560 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:45.560 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:45.560 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18 2024-09-11T06:50:45.560 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.560 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:45.560 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.560 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:45.560 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:50:45.560 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.560 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.560 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.560 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:45.560 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:50:45.560 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.561 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.561 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:45.561 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874109+0000", 2024-09-11T06:50:45.561 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.426e-06 2024-09-11T06:50:45.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.561 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.561 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:45.561 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874118+0000", 2024-09-11T06:50:45.561 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6929999999999996e-06 2024-09-11T06:50:45.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.561 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.561 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:45.561 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874134+0000", 2024-09-11T06:50:45.561 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5503e-05 2024-09-11T06:50:45.562 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.562 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.562 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:45.562 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874144+0000", 2024-09-11T06:50:45.562 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0207000000000001e-05 2024-09-11T06:50:45.562 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.562 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.562 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:45.562 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874545+0000", 2024-09-11T06:50:45.562 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040120800000000001 2024-09-11T06:50:45.562 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.562 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.562 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:45.562 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874686+0000", 2024-09-11T06:50:45.563 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014128599999999999 2024-09-11T06:50:45.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.563 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:45.563 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875023+0000", 2024-09-11T06:50:45.563 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033706600000000001 2024-09-11T06:50:45.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.563 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:45.563 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.876217+0000", 2024-09-11T06:50:45.563 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011938999999999999 2024-09-11T06:50:45.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.563 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:45.563 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877042+0000", 2024-09-11T06:50:45.564 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00082499899999999998 2024-09-11T06:50:45.564 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.564 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.564 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:45.564 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877128+0000", 2024-09-11T06:50:45.564 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5903999999999996e-05 2024-09-11T06:50:45.564 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.564 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.564 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:45.564 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877273+0000", 2024-09-11T06:50:45.564 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144466 2024-09-11T06:50:45.564 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.564 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:45.564 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.564 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.565 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.565 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:126 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:45.565 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.875489+0000", 2024-09-11T06:50:45.565 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.625238737, 2024-09-11T06:50:45.565 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002819996, 2024-09-11T06:50:45.565 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:45.565 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:45.565 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:45.565 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:45.565 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:45.565 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:45.565 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 126 2024-09-11T06:50:45.565 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.566 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:45.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.566 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:45.566 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875489+0000", 2024-09-11T06:50:45.566 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.566 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.566 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:45.566 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875489+0000", 2024-09-11T06:50:45.566 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.566 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.566 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:45.566 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875492+0000", 2024-09-11T06:50:45.566 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2600000000000001e-06 2024-09-11T06:50:45.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.567 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:45.567 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875507+0000", 2024-09-11T06:50:45.567 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4619e-05 2024-09-11T06:50:45.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.567 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:45.567 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875537+0000", 2024-09-11T06:50:45.567 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.023e-05 2024-09-11T06:50:45.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.567 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:45.567 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875587+0000", 2024-09-11T06:50:45.567 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9694999999999998e-05 2024-09-11T06:50:45.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:45.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875834+0000", 2024-09-11T06:50:45.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024709999999999999 2024-09-11T06:50:45.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:45.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.876037+0000", 2024-09-11T06:50:45.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00020236000000000001 2024-09-11T06:50:45.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:45.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877776+0000", 2024-09-11T06:50:45.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00173942 2024-09-11T06:50:45.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.569 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:45.569 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877998+0000", 2024-09-11T06:50:45.569 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022171100000000001 2024-09-11T06:50:45.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.569 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:45.569 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.878309+0000", 2024-09-11T06:50:45.569 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000311601 2024-09-11T06:50:45.569 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.569 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:45.569 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.570 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:24 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:45.570 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:50:45.570 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.612114317, 2024-09-11T06:50:45.570 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002980271, 2024-09-11T06:50:45.570 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:45.570 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:45.570 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:45.570 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:45.570 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:45.570 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:45.570 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 24 2024-09-11T06:50:45.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.570 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:45.571 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.571 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:45.571 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:50:45.571 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.571 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.571 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:45.571 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:50:45.571 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.571 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.571 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:45.571 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888616+0000", 2024-09-11T06:50:45.571 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0030000000000001e-06 2024-09-11T06:50:45.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.572 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:45.572 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888626+0000", 2024-09-11T06:50:45.572 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0502999999999999e-05 2024-09-11T06:50:45.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.572 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:45.572 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888655+0000", 2024-09-11T06:50:45.572 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8419999999999999e-05 2024-09-11T06:50:45.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.572 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:45.572 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888688+0000", 2024-09-11T06:50:45.572 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3212e-05 2024-09-11T06:50:45.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.573 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.573 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:45.573 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888829+0000", 2024-09-11T06:50:45.573 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014114299999999999 2024-09-11T06:50:45.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.573 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.573 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:45.573 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.889082+0000", 2024-09-11T06:50:45.573 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025279499999999999 2024-09-11T06:50:45.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.573 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.573 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:45.573 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.889577+0000", 2024-09-11T06:50:45.574 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00049536299999999997 2024-09-11T06:50:45.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.574 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.574 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:45.574 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.890811+0000", 2024-09-11T06:50:45.574 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001233858 2024-09-11T06:50:45.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.574 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.574 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:45.574 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891368+0000", 2024-09-11T06:50:45.574 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00055677400000000001 2024-09-11T06:50:45.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.574 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.575 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:45.575 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891456+0000", 2024-09-11T06:50:45.575 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8246999999999997e-05 2024-09-11T06:50:45.575 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.575 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.575 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:45.575 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891594+0000", 2024-09-11T06:50:45.575 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013795300000000001 2024-09-11T06:50:45.575 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.575 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:45.575 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.575 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.575 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.575 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:26 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 139740787512064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:45.575 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:50:45.576 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.608411672999999, 2024-09-11T06:50:45.576 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003613733, 2024-09-11T06:50:45.576 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:45.576 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:45.576 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:45.576 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:45.576 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:45.576 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:45.576 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:50:45.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.576 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:45.576 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.576 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:45.577 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:50:45.577 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.577 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.577 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:45.577 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:50:45.577 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.577 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.577 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:45.577 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892318+0000", 2024-09-11T06:50:45.577 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.623e-06 2024-09-11T06:50:45.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.578 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.578 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:45.578 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892325+0000", 2024-09-11T06:50:45.578 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:50:45.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.578 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.578 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:45.578 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892346+0000", 2024-09-11T06:50:45.578 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1246e-05 2024-09-11T06:50:45.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.578 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.578 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:45.578 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892362+0000", 2024-09-11T06:50:45.578 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5409000000000001e-05 2024-09-11T06:50:45.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.579 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.579 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:45.579 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892883+0000", 2024-09-11T06:50:45.579 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00052087599999999998 2024-09-11T06:50:45.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.579 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.579 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:45.579 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893123+0000", 2024-09-11T06:50:45.579 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000240769 2024-09-11T06:50:45.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.579 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.579 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:45.579 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893526+0000", 2024-09-11T06:50:45.580 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040300200000000002 2024-09-11T06:50:45.580 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.580 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.580 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:45.580 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894820+0000", 2024-09-11T06:50:45.580 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012934159999999999 2024-09-11T06:50:45.580 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.580 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.580 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:45.580 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895608+0000", 2024-09-11T06:50:45.580 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00078788100000000004 2024-09-11T06:50:45.580 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.580 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.580 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:45.581 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895724+0000", 2024-09-11T06:50:45.581 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011595999999999999 2024-09-11T06:50:45.581 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.581 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.581 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:45.581 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895930+0000", 2024-09-11T06:50:45.581 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000206391 2024-09-11T06:50:45.581 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.581 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:45.581 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.581 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.581 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.581 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:133 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:45.582 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:50:45.582 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.607613656, 2024-09-11T06:50:45.582 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035017360000000001, 2024-09-11T06:50:45.582 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:45.582 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:45.582 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:45.582 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:45.582 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:45.584 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:45.584 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 133 2024-09-11T06:50:45.584 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.584 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:45.584 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.584 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:45.584 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:50:45.584 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.584 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.585 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.585 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:45.585 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:50:45.585 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.585 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.585 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:45.585 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893116+0000", 2024-09-11T06:50:45.585 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1629999999999999e-06 2024-09-11T06:50:45.586 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.586 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.586 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:45.586 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893128+0000", 2024-09-11T06:50:45.586 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.11e-05 2024-09-11T06:50:45.586 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.586 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.586 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:45.586 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893156+0000", 2024-09-11T06:50:45.587 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8266e-05 2024-09-11T06:50:45.587 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.587 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.587 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:45.587 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893171+0000", 2024-09-11T06:50:45.587 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5426000000000001e-05 2024-09-11T06:50:45.587 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.587 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.587 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:45.587 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894561+0000", 2024-09-11T06:50:45.587 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013899959999999999 2024-09-11T06:50:45.588 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.588 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.588 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:45.588 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894781+0000", 2024-09-11T06:50:45.588 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021987100000000001 2024-09-11T06:50:45.588 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.588 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.588 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:45.588 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896349+0000", 2024-09-11T06:50:45.588 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015681779999999999 2024-09-11T06:50:45.588 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.588 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.589 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:45.589 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896458+0000", 2024-09-11T06:50:45.589 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000108187 2024-09-11T06:50:45.589 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.589 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.589 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:45.589 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896616+0000", 2024-09-11T06:50:45.589 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015854899999999999 2024-09-11T06:50:45.589 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.589 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:45.589 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.590 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.590 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.590 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:27 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:45.590 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:50:45.590 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.607602276, 2024-09-11T06:50:45.590 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003151989, 2024-09-11T06:50:45.590 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:45.590 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:45.590 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:45.590 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:45.590 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:45.591 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:45.591 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27 2024-09-11T06:50:45.591 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.591 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:45.591 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.591 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:45.591 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:50:45.591 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.592 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.592 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.592 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:45.592 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:50:45.592 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.592 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.592 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.592 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:45.593 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893128+0000", 2024-09-11T06:50:45.593 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.013e-06 2024-09-11T06:50:45.593 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.593 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.593 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:45.593 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893136+0000", 2024-09-11T06:50:45.593 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4190000000000002e-06 2024-09-11T06:50:45.593 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.593 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.593 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:45.593 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893158+0000", 2024-09-11T06:50:45.594 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1506999999999999e-05 2024-09-11T06:50:45.594 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.594 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.594 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:45.594 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893170+0000", 2024-09-11T06:50:45.594 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2741999999999999e-05 2024-09-11T06:50:45.594 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.594 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.594 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:45.594 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894255+0000", 2024-09-11T06:50:45.595 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001084453 2024-09-11T06:50:45.595 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.595 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.595 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:45.595 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894461+0000", 2024-09-11T06:50:45.595 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00020586700000000001 2024-09-11T06:50:45.595 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.595 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.595 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:45.595 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896033+0000", 2024-09-11T06:50:45.595 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001572092 2024-09-11T06:50:45.595 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.595 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.596 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:45.596 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896149+0000", 2024-09-11T06:50:45.596 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116636 2024-09-11T06:50:45.596 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.596 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.596 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:45.596 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896278+0000", 2024-09-11T06:50:45.596 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012826 2024-09-11T06:50:45.596 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.596 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:45.596 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.597 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.597 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.597 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:176 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch reconnect cookie 139735284585312 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:45.597 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:50:45.597 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.576235456999999, 2024-09-11T06:50:45.597 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033620960000000002, 2024-09-11T06:50:45.597 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:45.597 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:45.597 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:45.597 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:45.597 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:45.597 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:45.597 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 176 2024-09-11T06:50:45.598 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.598 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:45.598 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.598 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:45.598 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:50:45.598 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.598 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.598 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.598 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:45.598 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:50:45.598 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.598 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.598 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.599 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:45.599 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924494+0000", 2024-09-11T06:50:45.599 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.967e-06 2024-09-11T06:50:45.599 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.599 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.599 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:45.599 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924501+0000", 2024-09-11T06:50:45.599 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2659999999999998e-06 2024-09-11T06:50:45.599 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.599 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.599 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:45.599 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924520+0000", 2024-09-11T06:50:45.599 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9038999999999998e-05 2024-09-11T06:50:45.599 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.600 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.600 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:45.600 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924535+0000", 2024-09-11T06:50:45.600 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5359999999999999e-05 2024-09-11T06:50:45.600 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.600 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.600 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:45.600 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.925544+0000", 2024-09-11T06:50:45.600 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010085109999999999 2024-09-11T06:50:45.600 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.601 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.601 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:45.601 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.925671+0000", 2024-09-11T06:50:45.601 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012756599999999999 2024-09-11T06:50:45.601 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.601 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.601 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:45.601 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.927855+0000", 2024-09-11T06:50:45.601 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0021833870000000002 2024-09-11T06:50:45.601 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.601 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:45.601 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.602 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.602 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.602 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:186 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 139735284585312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:45.602 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.981668+0000", 2024-09-11T06:50:45.602 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.519059876, 2024-09-11T06:50:45.602 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0026256180000000001, 2024-09-11T06:50:45.602 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:45.602 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:45.602 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:45.602 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:45.602 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:45.602 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:45.602 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 186 2024-09-11T06:50:45.602 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.603 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:45.603 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.603 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:45.603 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.981668+0000", 2024-09-11T06:50:45.603 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.603 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.603 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.603 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:45.603 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.981668+0000", 2024-09-11T06:50:45.603 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.603 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.603 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.603 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:45.604 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.981670+0000", 2024-09-11T06:50:45.604 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3640000000000002e-06 2024-09-11T06:50:45.604 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.604 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.604 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:45.604 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.981682+0000", 2024-09-11T06:50:45.604 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1036000000000001e-05 2024-09-11T06:50:45.604 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.604 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.604 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:45.604 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.981713+0000", 2024-09-11T06:50:45.604 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1569000000000002e-05 2024-09-11T06:50:45.604 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.605 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.605 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:45.605 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.981728+0000", 2024-09-11T06:50:45.605 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5302999999999999e-05 2024-09-11T06:50:45.605 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.605 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.605 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:45.605 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.981858+0000", 2024-09-11T06:50:45.605 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012991599999999999 2024-09-11T06:50:45.605 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.605 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.605 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:45.606 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.981992+0000", 2024-09-11T06:50:45.606 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000133996 2024-09-11T06:50:45.606 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.606 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.606 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:45.606 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.982247+0000", 2024-09-11T06:50:45.606 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025445300000000003 2024-09-11T06:50:45.606 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.606 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.606 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:45.606 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.983436+0000", 2024-09-11T06:50:45.606 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011896089999999999 2024-09-11T06:50:45.606 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.606 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.607 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:45.607 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.984108+0000", 2024-09-11T06:50:45.607 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00067153099999999995 2024-09-11T06:50:45.607 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.607 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.607 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:45.607 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.984195+0000", 2024-09-11T06:50:45.607 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6804000000000004e-05 2024-09-11T06:50:45.607 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.607 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.607 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:45.607 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.984294+0000", 2024-09-11T06:50:45.607 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9036999999999994e-05 2024-09-11T06:50:45.607 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.607 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:45.608 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.608 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.608 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.608 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4828.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch watch cookie 140587969811264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-11T06:50:45.608 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:50:45.608 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.2545324989999997, 2024-09-11T06:50:45.608 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032778859999999998, 2024-09-11T06:50:45.608 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:45.608 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:45.608 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:45.608 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:45.608 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4828", 2024-09-11T06:50:45.609 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2410288650", 2024-09-11T06:50:45.609 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:50:45.609 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.609 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:45.609 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.609 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:45.609 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:50:45.609 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.609 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.609 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.609 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:45.610 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:50:45.610 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.610 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.610 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.610 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:45.610 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246198+0000", 2024-09-11T06:50:45.610 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.667e-06 2024-09-11T06:50:45.610 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.610 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.610 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:45.611 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246209+0000", 2024-09-11T06:50:45.611 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0409e-05 2024-09-11T06:50:45.611 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.611 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.611 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:45.611 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246241+0000", 2024-09-11T06:50:45.611 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2592999999999998e-05 2024-09-11T06:50:45.611 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.611 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.611 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:45.612 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246260+0000", 2024-09-11T06:50:45.612 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9343000000000001e-05 2024-09-11T06:50:45.612 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.612 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.612 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:45.612 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246466+0000", 2024-09-11T06:50:45.612 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000205701 2024-09-11T06:50:45.612 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.613 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.613 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:45.613 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246713+0000", 2024-09-11T06:50:45.613 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024646999999999999 2024-09-11T06:50:45.613 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.613 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.613 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:45.613 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.247133+0000", 2024-09-11T06:50:45.613 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000420536 2024-09-11T06:50:45.613 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.613 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.613 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:45.614 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.248541+0000", 2024-09-11T06:50:45.614 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001407801 2024-09-11T06:50:45.614 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.614 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.614 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:45.614 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249241+0000", 2024-09-11T06:50:45.614 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070043500000000001 2024-09-11T06:50:45.614 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.615 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.615 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:45.615 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249326+0000", 2024-09-11T06:50:45.615 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5037999999999995e-05 2024-09-11T06:50:45.615 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.615 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.615 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:45.615 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249473+0000", 2024-09-11T06:50:45.615 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014689299999999999 2024-09-11T06:50:45.615 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.615 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:45.615 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.615 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.615 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.615 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5313.0:25 2.1 2:967e1786:ns::rbd_object_map.11adc335555f.0000000000000004:head [writefull 0~38] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:50:45.616 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.256788+0000", 2024-09-11T06:50:45.616 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.2439394999999998, 2024-09-11T06:50:45.616 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0030446449999999999, 2024-09-11T06:50:45.616 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:45.616 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:45.616 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:45.616 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:45.616 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5313", 2024-09-11T06:50:45.617 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/624314229", 2024-09-11T06:50:45.617 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 25 2024-09-11T06:50:45.617 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.617 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:45.617 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.617 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:45.617 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256788+0000", 2024-09-11T06:50:45.617 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.617 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.617 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.618 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:45.618 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256788+0000", 2024-09-11T06:50:45.618 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.618 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.618 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.618 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:45.618 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256791+0000", 2024-09-11T06:50:45.618 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5770000000000001e-06 2024-09-11T06:50:45.618 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.618 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.619 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:45.619 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256802+0000", 2024-09-11T06:50:45.619 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0742e-05 2024-09-11T06:50:45.619 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.619 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.619 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:45.619 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256823+0000", 2024-09-11T06:50:45.619 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0999999999999999e-05 2024-09-11T06:50:45.619 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.619 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.620 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:45.620 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256841+0000", 2024-09-11T06:50:45.620 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8099000000000001e-05 2024-09-11T06:50:45.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.620 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.620 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:45.620 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256988+0000", 2024-09-11T06:50:45.620 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014756400000000001 2024-09-11T06:50:45.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.620 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.620 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:45.620 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.257234+0000", 2024-09-11T06:50:45.620 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024524399999999999 2024-09-11T06:50:45.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.621 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.621 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:45.621 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.257446+0000", 2024-09-11T06:50:45.621 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000212241 2024-09-11T06:50:45.621 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.621 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.621 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:45.621 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259292+0000", 2024-09-11T06:50:45.621 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001845937 2024-09-11T06:50:45.621 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.621 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.621 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:45.621 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259681+0000", 2024-09-11T06:50:45.621 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00038940699999999998 2024-09-11T06:50:45.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.622 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.622 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:45.622 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259770+0000", 2024-09-11T06:50:45.622 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8385000000000002e-05 2024-09-11T06:50:45.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.622 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.622 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:45.622 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259833+0000", 2024-09-11T06:50:45.622 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3448999999999995e-05 2024-09-11T06:50:45.622 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.622 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:45.622 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.622 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.623 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5328.0:14 2.1 2:967e1786:ns::rbd_object_map.11adc335555f.0000000000000004:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:50:45.623 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.396257+0000", 2024-09-11T06:50:45.623 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.104471051, 2024-09-11T06:50:45.623 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032125370000000001, 2024-09-11T06:50:45.623 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:45.623 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:45.623 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:45.623 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:45.623 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5328", 2024-09-11T06:50:45.623 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2386493698", 2024-09-11T06:50:45.624 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 14 2024-09-11T06:50:45.624 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.624 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:45.624 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.624 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:45.624 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396257+0000", 2024-09-11T06:50:45.624 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.624 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.624 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.624 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:45.624 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396257+0000", 2024-09-11T06:50:45.624 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:45.625 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.625 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.625 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:45.625 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396259+0000", 2024-09-11T06:50:45.625 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4830000000000002e-06 2024-09-11T06:50:45.625 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.625 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.625 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:45.625 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396271+0000", 2024-09-11T06:50:45.625 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.115e-05 2024-09-11T06:50:45.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.626 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.626 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:45.626 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396292+0000", 2024-09-11T06:50:45.626 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1761999999999999e-05 2024-09-11T06:50:45.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.626 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.626 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:45.626 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396314+0000", 2024-09-11T06:50:45.626 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1982999999999999e-05 2024-09-11T06:50:45.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.627 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.627 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:45.627 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396409+0000", 2024-09-11T06:50:45.627 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4250000000000001e-05 2024-09-11T06:50:45.627 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.627 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.627 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:45.627 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396464+0000", 2024-09-11T06:50:45.627 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5884000000000002e-05 2024-09-11T06:50:45.627 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:45.627 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:45.627 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:45.627 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.399469+0000", 2024-09-11T06:50:45.628 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0030050250000000001 2024-09-11T06:50:45.628 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.628 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:45.628 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.628 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:45.628 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:45.628 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:45.630 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:50:45.631 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd create --size 10M testrbdnbd21607 2024-09-11T06:50:45.631 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo rbd device --device-type nbd map testrbdnbd21607 2024-09-11T06:50:45.631 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:50:45.631 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:50:45.631 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:50:45.631 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which rbd 2024-09-11T06:50:45.631 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/rbd 2024-09-11T06:50:45.632 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:50:45.632 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map testrbdnbd21607 2024-09-11T06:50:45.632 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:45.627+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:45.731 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:45.727+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:45.748 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:50:45.750 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV=/dev/nbd0 2024-09-11T06:50:45.750 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid new_default_pool21607 2024-09-11T06:50:45.751 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=new_default_pool21607 2024-09-11T06:50:45.751 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:50:45.751 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:50:45.751 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''new_default_pool21607'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd0'\'']/id' 2024-09-11T06:50:45.762 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:45.762 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:45.762 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:50:45.762 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:45.777 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:50:45.815 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=23573 2024-09-11T06:50:45.815 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 23573 2024-09-11T06:50:45.816 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 23573 -C rbd-nbd 2024-09-11T06:50:45.821 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:50:45.822 INFO:tasks.workunit.client.0.smithi176.stdout: 23573 ? 00:00:00 rbd-nbd 2024-09-11T06:50:45.822 INFO:tasks.workunit.client.0.smithi176.stderr:+ unmap_device testrbdnbd21607 23573 2024-09-11T06:50:45.822 INFO:tasks.workunit.client.0.smithi176.stderr:+ local args=testrbdnbd21607 2024-09-11T06:50:45.822 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pid=23573 2024-09-11T06:50:45.823 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device --device-type nbd unmap testrbdnbd21607 2024-09-11T06:50:45.823 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:45.823 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:45.823 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:45.823 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:50:45.825 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:50:45.825 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:45.825 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap testrbdnbd21607 2024-09-11T06:50:45.832 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:45.827+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:45.903 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:50:45.904 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:50:45.912 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:50:45.912 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^23573\b' 2024-09-11T06:50:45.912 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^23573\b' 2024-09-11T06:50:45.919 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:50:45.920 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:50:45.932 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:45.927+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:45.935 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:50:45.936 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:50:45.947 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:45.948 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:45.948 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:50:45.948 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:50:45.948 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:45.955 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:50:45.956 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:50:45.962 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:50:45.965 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:45.965 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^ *23573\b' 2024-09-11T06:50:45.965 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^ *23573\b' 2024-09-11T06:50:45.965 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -C rbd-nbd 2024-09-11T06:50:45.973 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:45.973 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 0.5 2024-09-11T06:50:45.974 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:50:45.974 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:50:45.993 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:50:45.994 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:50:46.033 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:46.031+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:46.084 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:46.084 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:46.099 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:50:46.135 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:46.131+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:46.236 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:46.231+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:46.262 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:46.262 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:46.277 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:50:46.290 INFO:teuthology.orchestra.run.smithi071.stderr:marked in osd.5. 2024-09-11T06:50:46.314 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T06:50:46.320 INFO:tasks.thrashosds.thrasher:Added osd 5 2024-09-11T06:50:46.361 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:46.359+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:46.404 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:46.404 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:46.417 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:50:46.437 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:46.435+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:46.475 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV= 2024-09-11T06:50:46.475 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd config global rm global rbd_default_pool 2024-09-11T06:50:46.532 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo rbd device --device-type nbd map rbd/testrbdnbd21607 2024-09-11T06:50:46.532 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:50:46.532 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:50:46.533 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:50:46.533 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which rbd 2024-09-11T06:50:46.534 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/rbd 2024-09-11T06:50:46.534 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:50:46.534 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map rbd/testrbdnbd21607 2024-09-11T06:50:46.539 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:46.535+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:46.541 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:46.541 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:46.555 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:50:46.648 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:46.647+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:46.723 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:46.723 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:46.737 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:50:46.741 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:46.735+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:46.841 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:46.839+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:46.903 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:46.903 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:46.917 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:50:46.960 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:46.959+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:47.043 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:47.039+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:47.083 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:47.083 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:47.098 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:50:47.143 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:47.143+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:47.230 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:47.231 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:47.244 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:50:47.245 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:47.243+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:47.299 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is set 2024-09-11T06:50:47.345 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:47.339+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:47.346 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV=/dev/nbd0 2024-09-11T06:50:47.346 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd 2024-09-11T06:50:47.346 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:50:47.347 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:50:47.347 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd0'\'']/id' 2024-09-11T06:50:47.347 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:50:47.377 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:47.377 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:47.392 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:50:47.418 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=23667 2024-09-11T06:50:47.418 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 23667 2024-09-11T06:50:47.419 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 23667 -C rbd-nbd 2024-09-11T06:50:47.424 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:50:47.425 INFO:tasks.workunit.client.0.smithi176.stdout: 23667 ? 00:00:00 rbd-nbd 2024-09-11T06:50:47.425 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo kill 23667 2024-09-11T06:50:47.425 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:47.425 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:47.427 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:47.427 INFO:tasks.workunit.client.0.smithi176.stderr:++ which kill 2024-09-11T06:50:47.429 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/kill 2024-09-11T06:50:47.429 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:47.429 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/kill 23667 2024-09-11T06:50:47.443 INFO:tasks.workunit.client.0.smithi176.stderr:++ seq 10 2024-09-11T06:50:47.446 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:47.443+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:47.447 INFO:tasks.workunit.client.0.smithi176.stderr:+ for i in `seq 10` 2024-09-11T06:50:47.448 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:50:47.448 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^23667 *rbd *testrbdnbd21607' 2024-09-11T06:50:47.448 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^23667 *rbd *testrbdnbd21607' 2024-09-11T06:50:47.516 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:47.516 INFO:tasks.workunit.client.0.smithi176.stderr:+ break 2024-09-11T06:50:47.517 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^23667 *rbd *testrbdnbd21607' 2024-09-11T06:50:47.517 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:50:47.517 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^23667 *rbd *testrbdnbd21607' 2024-09-11T06:50:47.547 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:47.543+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:47.559 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:47.559 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:47.573 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:50:47.579 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:47.580 INFO:tasks.workunit.client.0.smithi176.stderr:+ QUIESCE_HOOK=/tmp/tmp.IGWYJYR7sz/quiesce.sh 2024-09-11T06:50:47.580 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo rbd device --device-type nbd map --quiesce --quiesce-hook /tmp/tmp.IGWYJYR7sz/quiesce.sh rbd/testrbdnbd21607 2024-09-11T06:50:47.580 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:50:47.580 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:50:47.581 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:50:47.581 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which rbd 2024-09-11T06:50:47.582 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/rbd 2024-09-11T06:50:47.583 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:50:47.583 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map --quiesce --quiesce-hook /tmp/tmp.IGWYJYR7sz/quiesce.sh rbd/testrbdnbd21607 2024-09-11T06:50:47.648 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:47.647+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:47.721 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV=/dev/nbd1 2024-09-11T06:50:47.722 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd 2024-09-11T06:50:47.722 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:50:47.722 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:50:47.722 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:50:47.722 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd1'\'']/id' 2024-09-11T06:50:47.742 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:47.743 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:47.748 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:47.747+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:47.757 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:50:47.813 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=23729 2024-09-11T06:50:47.813 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 23729 2024-09-11T06:50:47.813 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 23729 -C rbd-nbd 2024-09-11T06:50:47.819 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:50:47.819 INFO:tasks.workunit.client.0.smithi176.stdout: 23729 ? 00:00:00 rbd-nbd 2024-09-11T06:50:47.820 INFO:tasks.workunit.client.0.smithi176.stderr:+ test '!' -e /tmp/tmp.IGWYJYR7sz/quiesce.sh 2024-09-11T06:50:47.820 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false rbd snap create rbd/testrbdnbd21607@quiesce1 2024-09-11T06:50:47.820 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd snap create rbd/testrbdnbd21607@quiesce1 2024-09-11T06:50:47.850 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:47.847+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:47.877 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:47.877 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:47.891 INFO:tasks.workunit.client.0.smithi176.stderr: Creating snap: 10% complete...2024-09-11T06:50:47.887+0000 7efff7620640 -1 librbd::ImageWatcher: 0x7effdc0075b0 failed to notify quiesce: (1) Operation not permitted 2024-09-11T06:50:47.892 INFO:tasks.workunit.client.0.smithi176.stderr:2024-09-11T06:50:47.887+0000 7efff7620640 -1 librbd::SnapshotCreateRequest: failed to notify quiesce: (1) Operation not permitted 2024-09-11T06:50:47.893 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:50:47.894 INFO:tasks.workunit.client.0.smithi176.stderr: Creating snap: 10% complete...failed. 2024-09-11T06:50:47.894 INFO:tasks.workunit.client.0.smithi176.stderr:rbd: failed to create snapshot: (1) Operation not permitted 2024-09-11T06:50:47.906 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:47.907 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo dd if=/tmp/tmp.IGWYJYR7sz/data of=/dev/nbd1 bs=1M count=1 oflag=direct 2024-09-11T06:50:47.907 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:47.907 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:47.907 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:47.908 INFO:tasks.workunit.client.0.smithi176.stderr:++ which dd 2024-09-11T06:50:47.910 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/dd 2024-09-11T06:50:47.910 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:47.910 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/dd if=/tmp/tmp.IGWYJYR7sz/data of=/dev/nbd1 bs=1M count=1 oflag=direct 2024-09-11T06:50:47.950 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:47.947+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:47.954 INFO:tasks.workunit.client.0.smithi176.stderr:1+0 records in 2024-09-11T06:50:47.954 INFO:tasks.workunit.client.0.smithi176.stderr:1+0 records out 2024-09-11T06:50:47.954 INFO:tasks.workunit.client.0.smithi176.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0384686 s, 27.3 MB/s 2024-09-11T06:50:47.955 INFO:tasks.workunit.client.0.smithi176.stderr:+ touch /tmp/tmp.IGWYJYR7sz/quiesce.sh 2024-09-11T06:50:47.956 INFO:tasks.workunit.client.0.smithi176.stderr:+ chmod +x /tmp/tmp.IGWYJYR7sz/quiesce.sh 2024-09-11T06:50:47.958 INFO:tasks.workunit.client.0.smithi176.stderr:+ cat 2024-09-11T06:50:47.959 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd snap create rbd/testrbdnbd21607@quiesce1 2024-09-11T06:50:48.011 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:50:48.051 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:48.047+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:48.063 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:48.063 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:48.078 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:50:48.151 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:50:48.151 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:50:48.153 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:48.151+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:48.167 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:50:48.168 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:50:48.184 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:50:48.185 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:50:48.195 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:48.195 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:48.195 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:50:48.195 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:48.205 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:50:48.206 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:50:48.209 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:50:48.226 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:50:48.226 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:50:48.248 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:50:48.248 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:50:48.253 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:48.251+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:48.324 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:48.324 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:48.324 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:50:48.324 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:50:48.324 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:48.338 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:50:48.355 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:48.351+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:48.456 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:48.455+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:48.459 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:48.459 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:50:48.459 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:50:48.460 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:50:48.460 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.460 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch watch cookie 139735284585312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:48.460 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.521153+0000", 2024-09-11T06:50:48.460 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 17.934198938000002, 2024-09-11T06:50:48.460 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028996400000000002, 2024-09-11T06:50:48.460 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:48.460 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:48.460 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:48.460 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:48.460 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:48.460 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:48.460 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:50:48.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.461 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:48.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.461 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:48.461 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521153+0000", 2024-09-11T06:50:48.461 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.461 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:48.461 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521153+0000", 2024-09-11T06:50:48.461 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.461 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:48.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521154+0000", 2024-09-11T06:50:48.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.77e-06 2024-09-11T06:50:48.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:48.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521162+0000", 2024-09-11T06:50:48.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6569999999999997e-06 2024-09-11T06:50:48.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:48.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521184+0000", 2024-09-11T06:50:48.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1736e-05 2024-09-11T06:50:48.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:48.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521200+0000", 2024-09-11T06:50:48.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5953000000000001e-05 2024-09-11T06:50:48.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:48.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521328+0000", 2024-09-11T06:50:48.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000128489 2024-09-11T06:50:48.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:48.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521565+0000", 2024-09-11T06:50:48.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000236357 2024-09-11T06:50:48.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:48.464 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.521991+0000", 2024-09-11T06:50:48.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00042691099999999998 2024-09-11T06:50:48.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:48.464 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.523245+0000", 2024-09-11T06:50:48.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001253964 2024-09-11T06:50:48.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:48.464 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.523847+0000", 2024-09-11T06:50:48.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00060132899999999995 2024-09-11T06:50:48.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.465 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:48.465 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.523938+0000", 2024-09-11T06:50:48.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1577999999999996e-05 2024-09-11T06:50:48.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.465 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:48.465 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.524052+0000", 2024-09-11T06:50:48.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000113896 2024-09-11T06:50:48.465 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.465 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:48.465 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.466 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch watch cookie 140250278007648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:48.466 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.701683+0000", 2024-09-11T06:50:48.466 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 17.753668322999999, 2024-09-11T06:50:48.466 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002794693, 2024-09-11T06:50:48.466 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:48.466 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:48.466 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:48.466 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:48.466 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:48.466 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:48.466 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:50:48.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.466 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:48.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:48.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.701683+0000", 2024-09-11T06:50:48.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:48.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.701683+0000", 2024-09-11T06:50:48.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:48.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.701685+0000", 2024-09-11T06:50:48.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6500000000000001e-06 2024-09-11T06:50:48.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:48.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.701694+0000", 2024-09-11T06:50:48.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0759999999999992e-06 2024-09-11T06:50:48.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:48.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.701717+0000", 2024-09-11T06:50:48.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.349e-05 2024-09-11T06:50:48.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.469 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:48.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.701730+0000", 2024-09-11T06:50:48.469 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2649e-05 2024-09-11T06:50:48.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.469 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:48.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.701853+0000", 2024-09-11T06:50:48.469 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012253000000000001 2024-09-11T06:50:48.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.469 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:48.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.702079+0000", 2024-09-11T06:50:48.470 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022646999999999999 2024-09-11T06:50:48.470 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.470 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.470 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:48.470 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.702508+0000", 2024-09-11T06:50:48.470 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00042935500000000003 2024-09-11T06:50:48.470 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.470 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.470 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:48.471 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.703905+0000", 2024-09-11T06:50:48.471 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001396102 2024-09-11T06:50:48.471 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.471 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.471 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:48.471 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.704285+0000", 2024-09-11T06:50:48.472 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000380436 2024-09-11T06:50:48.472 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.472 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.472 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:48.472 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.704374+0000", 2024-09-11T06:50:48.472 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8813999999999994e-05 2024-09-11T06:50:48.472 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.472 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.472 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:48.472 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.704478+0000", 2024-09-11T06:50:48.472 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000104121 2024-09-11T06:50:48.472 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.473 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:48.473 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.473 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.473 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:14 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:48.473 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.706169+0000", 2024-09-11T06:50:48.474 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 17.749182268999999, 2024-09-11T06:50:48.474 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0029019599999999999, 2024-09-11T06:50:48.474 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:48.474 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:48.474 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:48.474 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:48.474 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:48.474 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:48.474 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 14 2024-09-11T06:50:48.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.474 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:48.474 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.474 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:48.475 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706169+0000", 2024-09-11T06:50:48.475 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.475 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:48.475 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706169+0000", 2024-09-11T06:50:48.475 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.475 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:48.475 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706172+0000", 2024-09-11T06:50:48.475 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.34e-06 2024-09-11T06:50:48.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.476 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:48.476 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706183+0000", 2024-09-11T06:50:48.476 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1556000000000001e-05 2024-09-11T06:50:48.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.476 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.476 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:48.476 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706215+0000", 2024-09-11T06:50:48.476 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2116000000000002e-05 2024-09-11T06:50:48.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.476 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.476 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:48.476 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706232+0000", 2024-09-11T06:50:48.477 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7073000000000001e-05 2024-09-11T06:50:48.477 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.477 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:48.477 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706356+0000", 2024-09-11T06:50:48.477 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012411599999999999 2024-09-11T06:50:48.477 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.477 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:48.477 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.706591+0000", 2024-09-11T06:50:48.477 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000234897 2024-09-11T06:50:48.477 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.477 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:48.478 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.707060+0000", 2024-09-11T06:50:48.478 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00046906299999999998 2024-09-11T06:50:48.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.478 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.478 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:48.478 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.708179+0000", 2024-09-11T06:50:48.478 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001118232 2024-09-11T06:50:48.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.478 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.478 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:48.478 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.708894+0000", 2024-09-11T06:50:48.478 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00071581900000000003 2024-09-11T06:50:48.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.478 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.479 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:48.479 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.708983+0000", 2024-09-11T06:50:48.479 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8270999999999996e-05 2024-09-11T06:50:48.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.479 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.479 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:48.479 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709071+0000", 2024-09-11T06:50:48.479 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8477e-05 2024-09-11T06:50:48.479 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.479 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:48.479 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.479 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.479 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:20 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:48.480 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.717149+0000", 2024-09-11T06:50:48.480 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 17.738202100999999, 2024-09-11T06:50:48.480 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002913297, 2024-09-11T06:50:48.480 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:48.480 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:48.480 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:48.480 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:48.480 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:48.480 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:48.480 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 20 2024-09-11T06:50:48.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.480 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:48.480 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.480 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:48.481 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717149+0000", 2024-09-11T06:50:48.481 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.481 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.481 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:48.481 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717149+0000", 2024-09-11T06:50:48.481 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.481 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.481 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:48.481 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717152+0000", 2024-09-11T06:50:48.481 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.21e-06 2024-09-11T06:50:48.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.482 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:48.482 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717164+0000", 2024-09-11T06:50:48.482 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1912999999999999e-05 2024-09-11T06:50:48.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.482 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:48.482 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717191+0000", 2024-09-11T06:50:48.482 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7526e-05 2024-09-11T06:50:48.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.482 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:48.482 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717206+0000", 2024-09-11T06:50:48.482 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5113000000000001e-05 2024-09-11T06:50:48.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.483 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.483 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:48.483 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717355+0000", 2024-09-11T06:50:48.483 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014896500000000001 2024-09-11T06:50:48.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.483 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.483 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:48.483 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717607+0000", 2024-09-11T06:50:48.483 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000252017 2024-09-11T06:50:48.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.483 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.483 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:48.483 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.718128+0000", 2024-09-11T06:50:48.484 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00052114499999999998 2024-09-11T06:50:48.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.484 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.484 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:48.484 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.719432+0000", 2024-09-11T06:50:48.484 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013041629999999999 2024-09-11T06:50:48.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.484 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.484 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:48.484 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.719864+0000", 2024-09-11T06:50:48.484 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00043115299999999998 2024-09-11T06:50:48.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.484 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.484 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:48.485 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.719949+0000", 2024-09-11T06:50:48.485 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4862000000000002e-05 2024-09-11T06:50:48.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.485 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.485 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:48.485 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.720063+0000", 2024-09-11T06:50:48.485 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011423 2024-09-11T06:50:48.485 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.485 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:48.485 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.485 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.485 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:24 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:48.485 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.726441+0000", 2024-09-11T06:50:48.486 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 17.728910603999999, 2024-09-11T06:50:48.486 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027674599999999998, 2024-09-11T06:50:48.486 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:48.486 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:48.486 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:48.486 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:48.486 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:48.486 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:48.486 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 24 2024-09-11T06:50:48.486 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.486 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:48.486 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.486 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:48.486 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726441+0000", 2024-09-11T06:50:48.487 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.487 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.487 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.487 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:48.487 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726441+0000", 2024-09-11T06:50:48.487 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.487 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.487 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.487 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:48.487 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726443+0000", 2024-09-11T06:50:48.488 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3769999999999999e-06 2024-09-11T06:50:48.488 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.488 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.488 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:48.488 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726456+0000", 2024-09-11T06:50:48.488 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2616000000000001e-05 2024-09-11T06:50:48.488 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.488 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.488 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:48.488 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726485+0000", 2024-09-11T06:50:48.488 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9195000000000001e-05 2024-09-11T06:50:48.488 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.488 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.489 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:48.489 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726499+0000", 2024-09-11T06:50:48.489 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4087000000000001e-05 2024-09-11T06:50:48.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.489 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.489 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:48.489 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726618+0000", 2024-09-11T06:50:48.489 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000118337 2024-09-11T06:50:48.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.489 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.489 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:48.490 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726772+0000", 2024-09-11T06:50:48.490 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015453799999999999 2024-09-11T06:50:48.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.490 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.490 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:48.490 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.727048+0000", 2024-09-11T06:50:48.490 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00027612900000000001 2024-09-11T06:50:48.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.490 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.490 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:48.490 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.728406+0000", 2024-09-11T06:50:48.490 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013579880000000001 2024-09-11T06:50:48.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.491 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:48.491 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.728878+0000", 2024-09-11T06:50:48.491 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00047132599999999999 2024-09-11T06:50:48.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.491 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:48.491 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729044+0000", 2024-09-11T06:50:48.491 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016672900000000001 2024-09-11T06:50:48.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.492 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:48.492 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729208+0000", 2024-09-11T06:50:48.492 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016413800000000001 2024-09-11T06:50:48.492 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.492 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:48.492 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.492 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.492 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.492 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:26 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 140250278007648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:48.492 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:50:48.492 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 17.725718965999999, 2024-09-11T06:50:48.492 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034645140000000001, 2024-09-11T06:50:48.492 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:48.493 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:48.493 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:48.493 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:48.493 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:48.493 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:48.493 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:50:48.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.493 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:48.493 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.493 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:48.494 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:50:48.494 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.494 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.494 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:48.494 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:50:48.494 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.494 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.495 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:48.495 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729634+0000", 2024-09-11T06:50:48.495 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0470000000000001e-06 2024-09-11T06:50:48.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.495 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.495 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:48.495 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729640+0000", 2024-09-11T06:50:48.495 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8860000000000001e-06 2024-09-11T06:50:48.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.495 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.495 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:48.495 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729654+0000", 2024-09-11T06:50:48.495 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:50:48.496 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.496 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.496 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:48.496 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729664+0000", 2024-09-11T06:50:48.496 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.594e-06 2024-09-11T06:50:48.496 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.496 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.496 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:48.496 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730422+0000", 2024-09-11T06:50:48.496 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000758648 2024-09-11T06:50:48.496 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.496 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.497 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:48.497 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730571+0000", 2024-09-11T06:50:48.497 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000148932 2024-09-11T06:50:48.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.497 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.497 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:48.497 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730936+0000", 2024-09-11T06:50:48.497 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036510900000000002 2024-09-11T06:50:48.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.497 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.497 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:48.497 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732159+0000", 2024-09-11T06:50:48.497 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001222302 2024-09-11T06:50:48.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.498 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:48.498 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732763+0000", 2024-09-11T06:50:48.498 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00060421599999999998 2024-09-11T06:50:48.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.498 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:48.498 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732916+0000", 2024-09-11T06:50:48.498 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015264 2024-09-11T06:50:48.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.498 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:48.498 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733097+0000", 2024-09-11T06:50:48.498 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000181504 2024-09-11T06:50:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:27 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:50:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 17.724745064, 2024-09-11T06:50:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003072852, 2024-09-11T06:50:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:48.500 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:48.500 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27 2024-09-11T06:50:48.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.500 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:48.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.500 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:48.500 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:50:48.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.500 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:48.500 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:50:48.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.501 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.501 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:48.501 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730608+0000", 2024-09-11T06:50:48.501 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8369999999999999e-06 2024-09-11T06:50:48.501 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.501 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.501 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:48.501 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730617+0000", 2024-09-11T06:50:48.501 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1859999999999992e-06 2024-09-11T06:50:48.501 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.501 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.501 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:48.501 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730636+0000", 2024-09-11T06:50:48.501 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9296e-05 2024-09-11T06:50:48.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.502 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.502 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:48.502 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730674+0000", 2024-09-11T06:50:48.502 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8609000000000003e-05 2024-09-11T06:50:48.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.502 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.502 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:48.502 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731496+0000", 2024-09-11T06:50:48.502 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00082112299999999999 2024-09-11T06:50:48.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:48.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731722+0000", 2024-09-11T06:50:48.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022651400000000001 2024-09-11T06:50:48.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:48.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733215+0000", 2024-09-11T06:50:48.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001493257 2024-09-11T06:50:48.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:48.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733452+0000", 2024-09-11T06:50:48.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023631600000000001 2024-09-11T06:50:48.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.504 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:48.504 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733679+0000", 2024-09-11T06:50:48.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022771400000000001 2024-09-11T06:50:48.504 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.504 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:48.504 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.504 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:73 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:48.504 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:50:48.505 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 17.724660780000001, 2024-09-11T06:50:48.505 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033454439999999999, 2024-09-11T06:50:48.505 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:48.505 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:48.505 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:48.505 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:48.505 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:48.505 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:48.505 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73 2024-09-11T06:50:48.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.505 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:48.505 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.505 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:48.505 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:50:48.506 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.506 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.506 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:48.506 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:50:48.506 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.506 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.506 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:48.506 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730694+0000", 2024-09-11T06:50:48.506 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5099999999999999e-06 2024-09-11T06:50:48.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.507 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:48.507 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730708+0000", 2024-09-11T06:50:48.507 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3237e-05 2024-09-11T06:50:48.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.507 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:48.507 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730738+0000", 2024-09-11T06:50:48.507 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0688999999999997e-05 2024-09-11T06:50:48.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.507 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:48.507 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730759+0000", 2024-09-11T06:50:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0818999999999998e-05 2024-09-11T06:50:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731841+0000", 2024-09-11T06:50:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010821819999999999 2024-09-11T06:50:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732092+0000", 2024-09-11T06:50:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025086299999999999 2024-09-11T06:50:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:48.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733730+0000", 2024-09-11T06:50:48.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0016374600000000001 2024-09-11T06:50:48.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:48.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733866+0000", 2024-09-11T06:50:48.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000136666 2024-09-11T06:50:48.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:48.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.734036+0000", 2024-09-11T06:50:48.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000170018 2024-09-11T06:50:48.509 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.509 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:48.509 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.510 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch watch cookie 139740787512064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:48.510 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.861993+0000", 2024-09-11T06:50:48.510 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 17.593358614, 2024-09-11T06:50:48.510 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027042189999999999, 2024-09-11T06:50:48.510 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:48.510 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:48.510 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:48.510 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:48.510 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:48.510 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:48.510 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:50:48.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.511 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:48.511 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.511 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:48.511 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.861993+0000", 2024-09-11T06:50:48.511 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.511 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.511 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.511 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:48.511 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.861993+0000", 2024-09-11T06:50:48.511 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.511 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.511 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.511 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:48.511 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.861995+0000", 2024-09-11T06:50:48.512 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2529999999999999e-06 2024-09-11T06:50:48.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.512 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:48.512 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862006+0000", 2024-09-11T06:50:48.512 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0509999999999999e-05 2024-09-11T06:50:48.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.512 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:48.512 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862037+0000", 2024-09-11T06:50:48.512 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1396000000000002e-05 2024-09-11T06:50:48.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.512 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:48.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862056+0000", 2024-09-11T06:50:48.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.859e-05 2024-09-11T06:50:48.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.513 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:48.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862168+0000", 2024-09-11T06:50:48.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000111996 2024-09-11T06:50:48.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.513 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:48.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862304+0000", 2024-09-11T06:50:48.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000136716 2024-09-11T06:50:48.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.514 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:48.514 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.862544+0000", 2024-09-11T06:50:48.514 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024005000000000001 2024-09-11T06:50:48.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.514 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.514 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:48.514 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.863704+0000", 2024-09-11T06:50:48.514 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011596499999999999 2024-09-11T06:50:48.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.514 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.514 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:48.514 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.864508+0000", 2024-09-11T06:50:48.514 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00080411999999999997 2024-09-11T06:50:48.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.515 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:48.515 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.864597+0000", 2024-09-11T06:50:48.515 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8660999999999997e-05 2024-09-11T06:50:48.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.515 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:48.515 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.864697+0000", 2024-09-11T06:50:48.515 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000100277 2024-09-11T06:50:48.515 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.515 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:48.515 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.516 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:18 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:48.516 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:50:48.516 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 17.581243591, 2024-09-11T06:50:48.516 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0031646579999999999, 2024-09-11T06:50:48.516 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:48.516 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:48.516 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:48.516 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:48.516 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:48.516 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:48.516 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18 2024-09-11T06:50:48.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.516 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:48.516 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.517 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:48.517 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:50:48.517 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.517 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:48.517 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:50:48.517 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.517 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:48.517 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874109+0000", 2024-09-11T06:50:48.517 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.426e-06 2024-09-11T06:50:48.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.518 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.518 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:48.518 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874118+0000", 2024-09-11T06:50:48.518 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6929999999999996e-06 2024-09-11T06:50:48.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.518 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.518 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:48.518 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874134+0000", 2024-09-11T06:50:48.518 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5503e-05 2024-09-11T06:50:48.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.518 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.518 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:48.518 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874144+0000", 2024-09-11T06:50:48.519 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0207000000000001e-05 2024-09-11T06:50:48.519 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.519 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.519 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:48.519 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874545+0000", 2024-09-11T06:50:48.519 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040120800000000001 2024-09-11T06:50:48.519 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.519 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.519 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:48.519 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874686+0000", 2024-09-11T06:50:48.519 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014128599999999999 2024-09-11T06:50:48.519 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.519 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.520 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:48.520 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875023+0000", 2024-09-11T06:50:48.520 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033706600000000001 2024-09-11T06:50:48.520 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.520 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.520 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:48.520 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.876217+0000", 2024-09-11T06:50:48.520 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011938999999999999 2024-09-11T06:50:48.520 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.520 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.521 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:48.521 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877042+0000", 2024-09-11T06:50:48.521 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00082499899999999998 2024-09-11T06:50:48.521 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.521 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.521 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:48.521 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877128+0000", 2024-09-11T06:50:48.521 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5903999999999996e-05 2024-09-11T06:50:48.521 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.521 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.521 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:48.521 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877273+0000", 2024-09-11T06:50:48.521 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144466 2024-09-11T06:50:48.522 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.522 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:48.522 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.522 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.522 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.522 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:126 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:48.522 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.875489+0000", 2024-09-11T06:50:48.522 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 17.579862304999999, 2024-09-11T06:50:48.522 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002819996, 2024-09-11T06:50:48.522 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:48.522 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:48.523 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:48.523 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:48.523 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:48.523 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:48.523 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 126 2024-09-11T06:50:48.523 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.523 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:48.523 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.523 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:48.523 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875489+0000", 2024-09-11T06:50:48.523 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.523 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.524 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.524 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:48.524 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875489+0000", 2024-09-11T06:50:48.524 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.524 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.524 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.524 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:48.524 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875492+0000", 2024-09-11T06:50:48.524 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2600000000000001e-06 2024-09-11T06:50:48.524 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.524 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.525 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:48.525 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875507+0000", 2024-09-11T06:50:48.525 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4619e-05 2024-09-11T06:50:48.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.525 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.525 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:48.525 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875537+0000", 2024-09-11T06:50:48.525 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.023e-05 2024-09-11T06:50:48.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.525 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.525 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:48.525 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875587+0000", 2024-09-11T06:50:48.525 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9694999999999998e-05 2024-09-11T06:50:48.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.526 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:48.526 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875834+0000", 2024-09-11T06:50:48.526 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024709999999999999 2024-09-11T06:50:48.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.526 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:48.526 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.876037+0000", 2024-09-11T06:50:48.526 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00020236000000000001 2024-09-11T06:50:48.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.527 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.527 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:48.527 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877776+0000", 2024-09-11T06:50:48.527 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00173942 2024-09-11T06:50:48.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.527 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.527 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:48.527 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877998+0000", 2024-09-11T06:50:48.527 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022171100000000001 2024-09-11T06:50:48.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.527 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.527 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:48.527 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.878309+0000", 2024-09-11T06:50:48.527 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000311601 2024-09-11T06:50:48.528 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.528 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:48.528 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.528 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.528 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.528 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:24 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:48.528 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:50:48.528 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 17.566737884999998, 2024-09-11T06:50:48.528 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002980271, 2024-09-11T06:50:48.528 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:48.528 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:48.528 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:48.529 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:48.529 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:48.529 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:48.529 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 24 2024-09-11T06:50:48.529 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.529 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:48.529 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.529 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:48.529 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:50:48.529 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.529 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.529 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.529 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:50:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888616+0000", 2024-09-11T06:50:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0030000000000001e-06 2024-09-11T06:50:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888626+0000", 2024-09-11T06:50:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0502999999999999e-05 2024-09-11T06:50:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888655+0000", 2024-09-11T06:50:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8419999999999999e-05 2024-09-11T06:50:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888688+0000", 2024-09-11T06:50:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3212e-05 2024-09-11T06:50:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888829+0000", 2024-09-11T06:50:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014114299999999999 2024-09-11T06:50:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.532 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:48.532 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.889082+0000", 2024-09-11T06:50:48.532 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025279499999999999 2024-09-11T06:50:48.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.532 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.532 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:48.532 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.889577+0000", 2024-09-11T06:50:48.532 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00049536299999999997 2024-09-11T06:50:48.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.532 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.532 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:48.532 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.890811+0000", 2024-09-11T06:50:48.532 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001233858 2024-09-11T06:50:48.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.533 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.533 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:48.533 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891368+0000", 2024-09-11T06:50:48.533 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00055677400000000001 2024-09-11T06:50:48.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.533 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.533 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:48.533 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891456+0000", 2024-09-11T06:50:48.533 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8246999999999997e-05 2024-09-11T06:50:48.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.533 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.533 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:48.533 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891594+0000", 2024-09-11T06:50:48.534 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013795300000000001 2024-09-11T06:50:48.534 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.534 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:48.534 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.534 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.534 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.534 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:26 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 139740787512064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:48.534 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:50:48.534 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 17.563035241000001, 2024-09-11T06:50:48.534 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003613733, 2024-09-11T06:50:48.534 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:48.534 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:48.534 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:50:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:50:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:50:48.536 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.536 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.536 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:48.536 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892318+0000", 2024-09-11T06:50:48.536 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.623e-06 2024-09-11T06:50:48.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.536 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.536 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:48.536 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892325+0000", 2024-09-11T06:50:48.536 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:50:48.537 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.537 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.537 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:48.537 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892346+0000", 2024-09-11T06:50:48.537 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1246e-05 2024-09-11T06:50:48.537 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.537 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.537 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:48.537 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892362+0000", 2024-09-11T06:50:48.537 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5409000000000001e-05 2024-09-11T06:50:48.537 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.537 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.537 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:48.537 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892883+0000", 2024-09-11T06:50:48.538 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00052087599999999998 2024-09-11T06:50:48.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.538 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:48.538 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893123+0000", 2024-09-11T06:50:48.538 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000240769 2024-09-11T06:50:48.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.538 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:48.538 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893526+0000", 2024-09-11T06:50:48.538 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040300200000000002 2024-09-11T06:50:48.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.539 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:48.539 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894820+0000", 2024-09-11T06:50:48.539 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012934159999999999 2024-09-11T06:50:48.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.539 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:48.539 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895608+0000", 2024-09-11T06:50:48.539 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00078788100000000004 2024-09-11T06:50:48.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.539 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:48.539 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895724+0000", 2024-09-11T06:50:48.539 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011595999999999999 2024-09-11T06:50:48.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.540 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:48.540 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895930+0000", 2024-09-11T06:50:48.540 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000206391 2024-09-11T06:50:48.540 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.540 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:48.540 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.540 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:133 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:48.540 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:50:48.540 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 17.562237224, 2024-09-11T06:50:48.541 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035017360000000001, 2024-09-11T06:50:48.541 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:48.541 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:48.541 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:48.541 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:48.541 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:48.541 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:48.541 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 133 2024-09-11T06:50:48.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.541 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:48.541 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.541 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:48.541 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:50:48.541 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.542 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:48.542 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:50:48.542 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.542 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:48.542 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893116+0000", 2024-09-11T06:50:48.542 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1629999999999999e-06 2024-09-11T06:50:48.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.542 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:48.542 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893128+0000", 2024-09-11T06:50:48.542 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.11e-05 2024-09-11T06:50:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893156+0000", 2024-09-11T06:50:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8266e-05 2024-09-11T06:50:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893171+0000", 2024-09-11T06:50:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5426000000000001e-05 2024-09-11T06:50:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894561+0000", 2024-09-11T06:50:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013899959999999999 2024-09-11T06:50:48.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.544 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:48.544 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894781+0000", 2024-09-11T06:50:48.544 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021987100000000001 2024-09-11T06:50:48.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.544 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:48.544 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896349+0000", 2024-09-11T06:50:48.544 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015681779999999999 2024-09-11T06:50:48.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.544 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:48.544 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896458+0000", 2024-09-11T06:50:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000108187 2024-09-11T06:50:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896616+0000", 2024-09-11T06:50:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015854899999999999 2024-09-11T06:50:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:27 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:50:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 17.562225844, 2024-09-11T06:50:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003151989, 2024-09-11T06:50:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27 2024-09-11T06:50:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:50:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.547 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:48.547 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:50:48.547 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.547 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:48.547 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893128+0000", 2024-09-11T06:50:48.547 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.013e-06 2024-09-11T06:50:48.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.547 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:48.547 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893136+0000", 2024-09-11T06:50:48.547 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4190000000000002e-06 2024-09-11T06:50:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893158+0000", 2024-09-11T06:50:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1506999999999999e-05 2024-09-11T06:50:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893170+0000", 2024-09-11T06:50:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2741999999999999e-05 2024-09-11T06:50:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894255+0000", 2024-09-11T06:50:48.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001084453 2024-09-11T06:50:48.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:48.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894461+0000", 2024-09-11T06:50:48.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00020586700000000001 2024-09-11T06:50:48.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:48.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896033+0000", 2024-09-11T06:50:48.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001572092 2024-09-11T06:50:48.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896149+0000", 2024-09-11T06:50:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116636 2024-09-11T06:50:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896278+0000", 2024-09-11T06:50:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012826 2024-09-11T06:50:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:176 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch reconnect cookie 139735284585312 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:50:48.551 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 17.530859025000002, 2024-09-11T06:50:48.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033620960000000002, 2024-09-11T06:50:48.551 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:48.551 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:48.551 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:48.551 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:48.551 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:48.551 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:48.551 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 176 2024-09-11T06:50:48.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.551 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:48.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.551 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:48.551 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:50:48.552 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.552 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.552 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:48.552 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:50:48.552 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.552 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.552 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:48.552 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924494+0000", 2024-09-11T06:50:48.552 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.967e-06 2024-09-11T06:50:48.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.552 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.552 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:48.553 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924501+0000", 2024-09-11T06:50:48.553 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2659999999999998e-06 2024-09-11T06:50:48.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.553 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:48.553 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924520+0000", 2024-09-11T06:50:48.553 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9038999999999998e-05 2024-09-11T06:50:48.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.553 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:48.554 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924535+0000", 2024-09-11T06:50:48.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5359999999999999e-05 2024-09-11T06:50:48.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.554 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:48.554 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.925544+0000", 2024-09-11T06:50:48.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010085109999999999 2024-09-11T06:50:48.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.554 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:48.554 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.925671+0000", 2024-09-11T06:50:48.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012756599999999999 2024-09-11T06:50:48.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.555 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:48.555 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.927855+0000", 2024-09-11T06:50:48.555 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0021833870000000002 2024-09-11T06:50:48.555 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.555 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:48.555 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.555 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.555 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.555 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4828.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch watch cookie 140587969811264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-11T06:50:48.555 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:50:48.555 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.209156067, 2024-09-11T06:50:48.555 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032778859999999998, 2024-09-11T06:50:48.555 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:48.555 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:48.556 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:48.556 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:48.556 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4828", 2024-09-11T06:50:48.556 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2410288650", 2024-09-11T06:50:48.556 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:50:48.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.556 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:48.556 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.556 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:48.556 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:50:48.556 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.556 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.556 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:48.557 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:50:48.557 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.557 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.557 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:48.557 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246198+0000", 2024-09-11T06:50:48.557 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.667e-06 2024-09-11T06:50:48.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.557 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.557 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:48.557 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246209+0000", 2024-09-11T06:50:48.557 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0409e-05 2024-09-11T06:50:48.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.557 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.558 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:48.558 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246241+0000", 2024-09-11T06:50:48.558 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2592999999999998e-05 2024-09-11T06:50:48.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.558 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.558 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:48.558 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246260+0000", 2024-09-11T06:50:48.558 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9343000000000001e-05 2024-09-11T06:50:48.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.558 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.558 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:48.558 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246466+0000", 2024-09-11T06:50:48.558 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000205701 2024-09-11T06:50:48.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.559 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.559 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:48.559 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246713+0000", 2024-09-11T06:50:48.559 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024646999999999999 2024-09-11T06:50:48.559 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.559 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.559 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:48.559 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.247133+0000", 2024-09-11T06:50:48.559 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000420536 2024-09-11T06:50:48.559 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.559 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.559 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:48.559 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.248541+0000", 2024-09-11T06:50:48.559 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001407801 2024-09-11T06:50:48.560 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.560 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.560 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:48.560 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249241+0000", 2024-09-11T06:50:48.560 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070043500000000001 2024-09-11T06:50:48.560 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.560 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.560 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:48.560 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249326+0000", 2024-09-11T06:50:48.560 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5037999999999995e-05 2024-09-11T06:50:48.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.561 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.561 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:48.561 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249473+0000", 2024-09-11T06:50:48.561 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014689299999999999 2024-09-11T06:50:48.561 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.561 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:48.561 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.561 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.561 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5313.0:25 2.1 2:967e1786:ns::rbd_object_map.11adc335555f.0000000000000004:head [writefull 0~38] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:50:48.561 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.256788+0000", 2024-09-11T06:50:48.561 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.1985630680000003, 2024-09-11T06:50:48.562 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0030446449999999999, 2024-09-11T06:50:48.562 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:48.562 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:48.562 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:48.562 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:48.562 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5313", 2024-09-11T06:50:48.562 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/624314229", 2024-09-11T06:50:48.562 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 25 2024-09-11T06:50:48.562 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.562 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:48.562 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.563 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:48.563 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256788+0000", 2024-09-11T06:50:48.563 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.563 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:48.563 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256788+0000", 2024-09-11T06:50:48.563 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.564 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.564 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:48.564 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256791+0000", 2024-09-11T06:50:48.564 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5770000000000001e-06 2024-09-11T06:50:48.564 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.564 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.564 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:48.564 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256802+0000", 2024-09-11T06:50:48.564 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0742e-05 2024-09-11T06:50:48.565 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.565 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.565 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:48.565 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256823+0000", 2024-09-11T06:50:48.565 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0999999999999999e-05 2024-09-11T06:50:48.565 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.565 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.565 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:48.566 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256841+0000", 2024-09-11T06:50:48.566 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8099000000000001e-05 2024-09-11T06:50:48.566 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.566 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:48.566 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256988+0000", 2024-09-11T06:50:48.566 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014756400000000001 2024-09-11T06:50:48.566 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.566 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:48.566 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.257234+0000", 2024-09-11T06:50:48.566 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024524399999999999 2024-09-11T06:50:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.257446+0000", 2024-09-11T06:50:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000212241 2024-09-11T06:50:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259292+0000", 2024-09-11T06:50:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001845937 2024-09-11T06:50:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259681+0000", 2024-09-11T06:50:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00038940699999999998 2024-09-11T06:50:48.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:48.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259770+0000", 2024-09-11T06:50:48.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8385000000000002e-05 2024-09-11T06:50:48.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:48.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259833+0000", 2024-09-11T06:50:48.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3448999999999995e-05 2024-09-11T06:50:48.568 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.568 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:48.569 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.569 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5328.0:14 2.1 2:967e1786:ns::rbd_object_map.11adc335555f.0000000000000004:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:50:48.569 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.396257+0000", 2024-09-11T06:50:48.569 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.0590946189999997, 2024-09-11T06:50:48.569 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032125370000000001, 2024-09-11T06:50:48.569 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:48.569 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:48.569 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:48.569 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:48.569 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5328", 2024-09-11T06:50:48.569 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2386493698", 2024-09-11T06:50:48.569 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 14 2024-09-11T06:50:48.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.570 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:48.570 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.570 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:48.570 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396257+0000", 2024-09-11T06:50:48.570 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.570 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.570 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:48.570 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396257+0000", 2024-09-11T06:50:48.570 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.570 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.571 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:48.571 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396259+0000", 2024-09-11T06:50:48.571 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4830000000000002e-06 2024-09-11T06:50:48.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.571 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.571 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:48.572 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396271+0000", 2024-09-11T06:50:48.572 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.115e-05 2024-09-11T06:50:48.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.572 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:48.572 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396292+0000", 2024-09-11T06:50:48.572 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1761999999999999e-05 2024-09-11T06:50:48.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.572 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:48.572 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396314+0000", 2024-09-11T06:50:48.572 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1982999999999999e-05 2024-09-11T06:50:48.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.573 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.573 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:48.573 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396409+0000", 2024-09-11T06:50:48.573 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4250000000000001e-05 2024-09-11T06:50:48.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.573 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.573 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:48.574 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396464+0000", 2024-09-11T06:50:48.574 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5884000000000002e-05 2024-09-11T06:50:48.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.574 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.574 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:48.574 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.399469+0000", 2024-09-11T06:50:48.574 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0030050250000000001 2024-09-11T06:50:48.574 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.574 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:48.574 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.574 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.574 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:6 3.5 3:b2cc3336:::rbd_id.testrbdnbd21607:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:50:48.575 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.591899+0000", 2024-09-11T06:50:48.575 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 2.8634521180000001, 2024-09-11T06:50:48.575 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032382999999999999, 2024-09-11T06:50:48.575 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:48.575 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:48.575 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:48.575 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:48.575 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:50:48.575 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:50:48.575 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 6 2024-09-11T06:50:48.575 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.575 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:48.575 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.576 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:48.576 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591899+0000", 2024-09-11T06:50:48.576 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.576 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.576 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:48.576 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591899+0000", 2024-09-11T06:50:48.576 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:48.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.576 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.576 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:48.576 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591902+0000", 2024-09-11T06:50:48.577 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0770000000000002e-06 2024-09-11T06:50:48.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.577 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.577 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:48.577 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591913+0000", 2024-09-11T06:50:48.577 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1182999999999999e-05 2024-09-11T06:50:48.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.578 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.578 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:48.578 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591940+0000", 2024-09-11T06:50:48.578 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7233000000000002e-05 2024-09-11T06:50:48.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.578 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.579 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:48.579 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591954+0000", 2024-09-11T06:50:48.579 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4503e-05 2024-09-11T06:50:48.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.579 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.579 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:48.579 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.592071+0000", 2024-09-11T06:50:48.579 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116731 2024-09-11T06:50:48.580 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.580 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.580 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:48.580 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.592274+0000", 2024-09-11T06:50:48.580 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00020320900000000001 2024-09-11T06:50:48.580 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.581 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.581 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 3", 2024-09-11T06:50:48.581 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.592576+0000", 2024-09-11T06:50:48.581 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030179000000000002 2024-09-11T06:50:48.581 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.582 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.582 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:48.582 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.594072+0000", 2024-09-11T06:50:48.582 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001495496 2024-09-11T06:50:48.582 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.582 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.582 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:48.582 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.594945+0000", 2024-09-11T06:50:48.582 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00087379799999999996 2024-09-11T06:50:48.582 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.583 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.583 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:48.583 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.595023+0000", 2024-09-11T06:50:48.583 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7751999999999997e-05 2024-09-11T06:50:48.583 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:48.583 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:48.583 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:48.583 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.595138+0000", 2024-09-11T06:50:48.583 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000114528 2024-09-11T06:50:48.583 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.584 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:48.584 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.584 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:48.584 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:48.584 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:48.586 INFO:tasks.workunit.client.0.smithi176.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2024-09-11T06:50:48.586 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo dd if=/tmp/tmp.IGWYJYR7sz/data of=/dev/nbd1 bs=1M count=1 oflag=direct 2024-09-11T06:50:48.586 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:48.586 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:48.586 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:48.587 INFO:tasks.workunit.client.0.smithi176.stderr:++ which dd 2024-09-11T06:50:48.587 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/dd 2024-09-11T06:50:48.587 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:48.587 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/dd if=/tmp/tmp.IGWYJYR7sz/data of=/dev/nbd1 bs=1M count=1 oflag=direct 2024-09-11T06:50:48.587 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:50:48.588 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:48.583+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:48.611 INFO:tasks.workunit.client.0.smithi176.stderr:1+0 records in 2024-09-11T06:50:48.611 INFO:tasks.workunit.client.0.smithi176.stderr:1+0 records out 2024-09-11T06:50:48.611 INFO:tasks.workunit.client.0.smithi176.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022593 s, 46.4 MB/s 2024-09-11T06:50:48.612 INFO:tasks.workunit.client.0.smithi176.stderr:++ cat /tmp/tmp.IGWYJYR7sz/quiesce 2024-09-11T06:50:48.614 INFO:tasks.workunit.client.0.smithi176.stderr:+ test /dev/nbd1 = /dev/nbd1 2024-09-11T06:50:48.614 INFO:tasks.workunit.client.0.smithi176.stderr:++ cat /tmp/tmp.IGWYJYR7sz/unquiesce 2024-09-11T06:50:48.616 INFO:tasks.workunit.client.0.smithi176.stderr:+ test /dev/nbd1 = /dev/nbd1 2024-09-11T06:50:48.616 INFO:tasks.workunit.client.0.smithi176.stderr:+ touch /tmp/tmp.IGWYJYR7sz/quiesce.sh 2024-09-11T06:50:48.617 INFO:tasks.workunit.client.0.smithi176.stderr:+ chmod +x /tmp/tmp.IGWYJYR7sz/quiesce.sh 2024-09-11T06:50:48.618 INFO:tasks.workunit.client.0.smithi176.stderr:+ cat 2024-09-11T06:50:48.619 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false rbd snap create rbd/testrbdnbd21607@quiesce2 2024-09-11T06:50:48.619 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd snap create rbd/testrbdnbd21607@quiesce2 2024-09-11T06:50:48.686 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:48.683+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:48.697 INFO:tasks.workunit.client.0.smithi176.stderr: Creating snap: 10% complete... Creating snap: 10% complete...failed. 2024-09-11T06:50:48.697 INFO:tasks.workunit.client.0.smithi176.stderr:rbd: failed to create snapshot: (22) Invalid argument 2024-09-11T06:50:48.707 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:50:48.707 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo dd if=/tmp/tmp.IGWYJYR7sz/data of=/dev/nbd1 bs=1M count=1 oflag=direct 2024-09-11T06:50:48.707 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:48.707 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:48.709 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:48.710 INFO:tasks.workunit.client.0.smithi176.stderr:++ which dd 2024-09-11T06:50:48.711 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/dd 2024-09-11T06:50:48.711 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:48.711 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/dd if=/tmp/tmp.IGWYJYR7sz/data of=/dev/nbd1 bs=1M count=1 oflag=direct 2024-09-11T06:50:48.718 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:48.718 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:48.718 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:50:48.718 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:48.731 INFO:tasks.workunit.client.0.smithi176.stderr:1+0 records in 2024-09-11T06:50:48.731 INFO:tasks.workunit.client.0.smithi176.stderr:1+0 records out 2024-09-11T06:50:48.731 INFO:tasks.workunit.client.0.smithi176.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126749 s, 82.7 MB/s 2024-09-11T06:50:48.732 INFO:tasks.workunit.client.0.smithi176.stderr:+ cat 2024-09-11T06:50:48.734 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:50:48.735 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd snap create rbd/testrbdnbd21607@quiesce2 2024-09-11T06:50:48.787 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:48.783+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:48.888 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:48.887+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:48.894 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:48.894 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:48.894 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:50:48.894 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:50:48.894 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:48.909 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:50:48.996 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:48.995+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:49.075 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:49.075 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:49.075 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:49.075 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:50:49.076 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.076 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:49.076 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:49.076 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 102.800459072, 2024-09-11T06:50:49.076 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283336, 2024-09-11T06:50:49.076 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.076 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.076 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:49.076 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:49.076 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:49.076 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:49.076 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 87 2024-09-11T06:50:49.076 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.077 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.077 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.077 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.077 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:49.077 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.077 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.077 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.077 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.077 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:49.077 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.077 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.077 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.077 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.077 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271768+0000", 2024-09-11T06:50:49.078 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7599999999999996e-07 2024-09-11T06:50:49.078 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.078 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.078 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.078 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271860+0000", 2024-09-11T06:50:49.078 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2639999999999994e-05 2024-09-11T06:50:49.078 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.078 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.078 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.078 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271873+0000", 2024-09-11T06:50:49.078 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3139000000000001e-05 2024-09-11T06:50:49.078 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.078 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.078 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.079 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271882+0000", 2024-09-11T06:50:49.079 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2369999999999992e-06 2024-09-11T06:50:49.079 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.079 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.079 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.079 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277150+0000", 2024-09-11T06:50:49.079 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052684419999999999 2024-09-11T06:50:49.079 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.079 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.079 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:49.079 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277273+0000", 2024-09-11T06:50:49.079 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012278899999999999 2024-09-11T06:50:49.079 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.079 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.079 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:49.080 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277466+0000", 2024-09-11T06:50:49.080 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193578 2024-09-11T06:50:49.080 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.080 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.080 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:49.080 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.280808+0000", 2024-09-11T06:50:49.080 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003341971 2024-09-11T06:50:49.080 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.080 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.080 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:49.080 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284419+0000", 2024-09-11T06:50:49.080 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003610982 2024-09-11T06:50:49.080 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.081 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.081 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:49.081 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284531+0000", 2024-09-11T06:50:49.081 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011121299999999999 2024-09-11T06:50:49.081 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.081 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.081 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.081 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284600+0000", 2024-09-11T06:50:49.081 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9592999999999996e-05 2024-09-11T06:50:49.081 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.081 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.081 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.081 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.081 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.082 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:49.082 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:49.082 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 102.80032882, 2024-09-11T06:50:49.082 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283314, 2024-09-11T06:50:49.082 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.082 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.083 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:49.083 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:49.083 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:49.083 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:49.083 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 88 2024-09-11T06:50:49.083 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.083 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.083 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.083 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.083 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:49.083 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.083 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.083 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.083 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.084 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:49.084 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.084 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.084 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.084 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.084 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271898+0000", 2024-09-11T06:50:49.084 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2999999999999999e-07 2024-09-11T06:50:49.084 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.084 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.084 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.084 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271904+0000", 2024-09-11T06:50:49.084 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.767e-06 2024-09-11T06:50:49.084 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.084 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.085 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.085 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271913+0000", 2024-09-11T06:50:49.085 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7200000000000001e-06 2024-09-11T06:50:49.085 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.085 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.085 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.085 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271920+0000", 2024-09-11T06:50:49.085 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7100000000000001e-06 2024-09-11T06:50:49.085 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.085 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.085 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.085 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278684+0000", 2024-09-11T06:50:49.085 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0067642650000000002 2024-09-11T06:50:49.085 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.086 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.086 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:49.086 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278838+0000", 2024-09-11T06:50:49.086 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015371499999999999 2024-09-11T06:50:49.086 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.086 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.086 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:49.086 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.279040+0000", 2024-09-11T06:50:49.086 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020207499999999999 2024-09-11T06:50:49.086 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.086 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.086 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:49.086 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.281448+0000", 2024-09-11T06:50:49.086 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002407686 2024-09-11T06:50:49.087 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.087 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.087 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:49.087 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284615+0000", 2024-09-11T06:50:49.087 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031675499999999999 2024-09-11T06:50:49.087 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.087 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.087 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:49.087 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284682+0000", 2024-09-11T06:50:49.087 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7032000000000005e-05 2024-09-11T06:50:49.087 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.087 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.087 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.087 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284730+0000", 2024-09-11T06:50:49.087 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7790000000000002e-05 2024-09-11T06:50:49.088 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.088 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.088 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.088 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.088 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.088 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:49.088 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:49.088 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 102.63147961999999, 2024-09-11T06:50:49.088 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014944941, 2024-09-11T06:50:49.088 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.088 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.088 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:49.088 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:49.088 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:49.089 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:49.089 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 111 2024-09-11T06:50:49.089 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.089 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.089 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.089 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.089 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:49.089 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.089 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.089 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.089 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.089 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:49.090 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.090 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.090 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.090 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.090 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440748+0000", 2024-09-11T06:50:49.090 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:50:49.090 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.090 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.090 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.090 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440760+0000", 2024-09-11T06:50:49.090 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1897e-05 2024-09-11T06:50:49.090 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.090 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.090 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.091 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440782+0000", 2024-09-11T06:50:49.091 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2552999999999999e-05 2024-09-11T06:50:49.091 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.091 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.091 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.091 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440793+0000", 2024-09-11T06:50:49.091 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0643e-05 2024-09-11T06:50:49.091 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.091 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.091 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.091 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449755+0000", 2024-09-11T06:50:49.091 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089622620000000004 2024-09-11T06:50:49.091 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.091 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.092 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:49.092 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449918+0000", 2024-09-11T06:50:49.092 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163086 2024-09-11T06:50:49.092 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.092 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.092 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:49.092 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450150+0000", 2024-09-11T06:50:49.092 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023204500000000001 2024-09-11T06:50:49.092 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.092 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.092 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:49.092 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454533+0000", 2024-09-11T06:50:49.092 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0043831349999999998 2024-09-11T06:50:49.093 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.093 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.093 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:49.093 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455511+0000", 2024-09-11T06:50:49.093 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0009776050000000001 2024-09-11T06:50:49.093 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.093 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.093 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:49.093 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455597+0000", 2024-09-11T06:50:49.093 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6446000000000005e-05 2024-09-11T06:50:49.093 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.093 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.093 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.093 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455691+0000", 2024-09-11T06:50:49.093 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3832000000000006e-05 2024-09-11T06:50:49.094 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.094 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.094 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.094 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.094 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.094 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:49.094 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:49.094 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 102.63140985699999, 2024-09-11T06:50:49.094 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018885411000000001, 2024-09-11T06:50:49.094 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.094 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.094 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:49.094 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:49.094 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:49.095 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:49.095 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 112 2024-09-11T06:50:49.095 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.095 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.095 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.095 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.095 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:49.095 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.095 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.095 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.095 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.095 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:49.095 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.095 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.096 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.096 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.096 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440817+0000", 2024-09-11T06:50:49.096 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:50:49.096 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.096 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.096 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.096 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441003+0000", 2024-09-11T06:50:49.096 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018661200000000001 2024-09-11T06:50:49.096 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.096 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.096 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.096 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441027+0000", 2024-09-11T06:50:49.096 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3920000000000001e-05 2024-09-11T06:50:49.097 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.097 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.097 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.097 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441062+0000", 2024-09-11T06:50:49.097 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4663000000000003e-05 2024-09-11T06:50:49.097 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.097 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.097 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.097 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451981+0000", 2024-09-11T06:50:49.097 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010918926000000001 2024-09-11T06:50:49.097 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.097 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.097 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:49.097 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452111+0000", 2024-09-11T06:50:49.098 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129529 2024-09-11T06:50:49.098 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.098 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.098 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:49.098 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452311+0000", 2024-09-11T06:50:49.098 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020013799999999999 2024-09-11T06:50:49.098 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.098 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.098 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:49.098 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455809+0000", 2024-09-11T06:50:49.098 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003498443 2024-09-11T06:50:49.098 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.098 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.098 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:49.099 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459566+0000", 2024-09-11T06:50:49.099 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003757271 2024-09-11T06:50:49.099 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.099 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.099 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:49.099 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459640+0000", 2024-09-11T06:50:49.099 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3949000000000006e-05 2024-09-11T06:50:49.099 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.099 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.099 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.099 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459701+0000", 2024-09-11T06:50:49.099 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1117000000000005e-05 2024-09-11T06:50:49.099 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.099 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.099 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.100 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.100 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.100 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:49.100 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:49.100 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 102.631131006, 2024-09-11T06:50:49.100 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018760915, 2024-09-11T06:50:49.100 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.100 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.100 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:49.100 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:49.100 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:49.100 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:49.100 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 113 2024-09-11T06:50:49.100 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.101 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.101 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.101 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.101 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:49.101 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.101 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.101 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.101 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.101 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:49.101 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.101 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.101 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.101 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.101 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441096+0000", 2024-09-11T06:50:49.102 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4130000000000001e-06 2024-09-11T06:50:49.102 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.102 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.102 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.102 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441110+0000", 2024-09-11T06:50:49.102 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3900000000000001e-05 2024-09-11T06:50:49.102 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.102 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.102 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.102 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441131+0000", 2024-09-11T06:50:49.102 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0636999999999999e-05 2024-09-11T06:50:49.102 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.102 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.102 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.102 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441147+0000", 2024-09-11T06:50:49.103 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6512999999999999e-05 2024-09-11T06:50:49.103 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.103 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.103 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.103 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453274+0000", 2024-09-11T06:50:49.103 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012126764999999999 2024-09-11T06:50:49.103 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.103 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.103 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:49.103 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453498+0000", 2024-09-11T06:50:49.103 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000223568 2024-09-11T06:50:49.103 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.103 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.103 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:49.104 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453872+0000", 2024-09-11T06:50:49.104 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037467099999999998 2024-09-11T06:50:49.104 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.104 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.104 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:49.104 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458709+0000", 2024-09-11T06:50:49.104 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0048365680000000003 2024-09-11T06:50:49.104 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.104 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.104 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:49.104 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459716+0000", 2024-09-11T06:50:49.104 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010075240000000001 2024-09-11T06:50:49.104 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.104 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.105 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:49.105 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459779+0000", 2024-09-11T06:50:49.105 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2336999999999996e-05 2024-09-11T06:50:49.105 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.105 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.105 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.105 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459856+0000", 2024-09-11T06:50:49.105 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7019000000000002e-05 2024-09-11T06:50:49.105 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.105 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.105 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.105 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.105 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.105 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:49.106 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:49.106 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 102.631054267, 2024-09-11T06:50:49.106 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018963114, 2024-09-11T06:50:49.106 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.106 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.106 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:49.106 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:49.106 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:49.106 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:49.106 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 114 2024-09-11T06:50:49.106 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.106 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.106 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.106 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.107 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:49.107 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.107 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.107 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.107 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.107 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:49.107 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.107 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.107 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.107 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.107 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:49.107 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9299999999999996e-07 2024-09-11T06:50:49.107 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.107 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.107 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.108 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441180+0000", 2024-09-11T06:50:49.108 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7200000000000006e-06 2024-09-11T06:50:49.108 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.108 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.108 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.108 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441265+0000", 2024-09-11T06:50:49.108 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4753000000000007e-05 2024-09-11T06:50:49.108 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.108 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.109 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.109 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441288+0000", 2024-09-11T06:50:49.109 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2860000000000001e-05 2024-09-11T06:50:49.109 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.109 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.109 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.109 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454585+0000", 2024-09-11T06:50:49.109 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013297286 2024-09-11T06:50:49.109 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.109 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.109 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:49.109 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454714+0000", 2024-09-11T06:50:49.109 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000128465 2024-09-11T06:50:49.110 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.110 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.110 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:49.110 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454909+0000", 2024-09-11T06:50:49.110 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019522499999999999 2024-09-11T06:50:49.110 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.110 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.110 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:49.110 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459868+0000", 2024-09-11T06:50:49.110 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0049591239999999997 2024-09-11T06:50:49.110 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.110 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.110 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:49.110 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459988+0000", 2024-09-11T06:50:49.111 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000120532 2024-09-11T06:50:49.111 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.111 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.111 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:49.111 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460071+0000", 2024-09-11T06:50:49.111 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2256000000000007e-05 2024-09-11T06:50:49.111 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.111 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.111 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.111 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460135+0000", 2024-09-11T06:50:49.111 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3999999999999997e-05 2024-09-11T06:50:49.111 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.111 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.111 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.111 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.112 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.112 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:49.112 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:49.112 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 102.630896638, 2024-09-11T06:50:49.112 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.019036849000000002, 2024-09-11T06:50:49.112 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.112 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.112 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:49.112 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:49.112 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:49.112 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:49.112 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 115 2024-09-11T06:50:49.112 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.113 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.113 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.113 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.113 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:49.113 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.113 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.113 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.113 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.113 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:49.113 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.113 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.113 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.113 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.113 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441330+0000", 2024-09-11T06:50:49.113 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1930000000000001e-06 2024-09-11T06:50:49.114 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.114 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.114 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.114 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441544+0000", 2024-09-11T06:50:49.114 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021405799999999999 2024-09-11T06:50:49.114 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.114 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.114 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.114 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441572+0000", 2024-09-11T06:50:49.114 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7317e-05 2024-09-11T06:50:49.114 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.114 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.114 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.114 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441593+0000", 2024-09-11T06:50:49.115 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1702999999999999e-05 2024-09-11T06:50:49.115 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.115 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.115 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.115 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455891+0000", 2024-09-11T06:50:49.115 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01429708 2024-09-11T06:50:49.115 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.115 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.115 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:49.115 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456046+0000", 2024-09-11T06:50:49.115 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000155682 2024-09-11T06:50:49.115 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.115 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.115 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:49.115 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456242+0000", 2024-09-11T06:50:49.116 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000195692 2024-09-11T06:50:49.116 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.116 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.116 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:49.116 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459876+0000", 2024-09-11T06:50:49.116 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036336480000000002 2024-09-11T06:50:49.116 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.116 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.116 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:49.116 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460245+0000", 2024-09-11T06:50:49.116 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036901700000000002 2024-09-11T06:50:49.116 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.116 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.116 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:49.117 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460316+0000", 2024-09-11T06:50:49.117 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1846000000000003e-05 2024-09-11T06:50:49.117 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.117 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.117 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.117 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460366+0000", 2024-09-11T06:50:49.117 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9613000000000001e-05 2024-09-11T06:50:49.117 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.117 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.117 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.117 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.117 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.118 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:49.118 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:49.118 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 102.63057455800001, 2024-09-11T06:50:49.118 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01904432, 2024-09-11T06:50:49.118 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.118 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.118 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:49.118 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:49.118 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:49.118 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:49.118 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 116 2024-09-11T06:50:49.118 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.118 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.118 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.119 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.119 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:49.119 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.119 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.119 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.119 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.119 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:49.119 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.119 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.119 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.119 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.119 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441653+0000", 2024-09-11T06:50:49.119 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:50:49.119 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.119 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.120 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.120 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441665+0000", 2024-09-11T06:50:49.120 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1653e-05 2024-09-11T06:50:49.120 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.120 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.120 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.120 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441684+0000", 2024-09-11T06:50:49.120 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9490000000000001e-05 2024-09-11T06:50:49.120 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.120 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.120 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.120 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441699+0000", 2024-09-11T06:50:49.120 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5044e-05 2024-09-11T06:50:49.120 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.121 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.121 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.121 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457297+0000", 2024-09-11T06:50:49.121 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015598241000000001 2024-09-11T06:50:49.121 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.121 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.121 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:49.121 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457577+0000", 2024-09-11T06:50:49.121 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027930799999999998 2024-09-11T06:50:49.121 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.121 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.121 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:49.121 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457957+0000", 2024-09-11T06:50:49.121 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00038088 2024-09-11T06:50:49.122 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.122 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.122 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:49.122 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460432+0000", 2024-09-11T06:50:49.122 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024743349999999998 2024-09-11T06:50:49.122 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.122 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.122 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:49.122 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460584+0000", 2024-09-11T06:50:49.122 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015232900000000001 2024-09-11T06:50:49.122 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.122 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.122 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:49.122 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460646+0000", 2024-09-11T06:50:49.123 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1883000000000004e-05 2024-09-11T06:50:49.123 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.123 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.123 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.123 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460696+0000", 2024-09-11T06:50:49.123 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9589999999999998e-05 2024-09-11T06:50:49.123 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.123 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.123 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.123 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.123 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.123 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:49.123 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:49.123 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 102.63050249200001, 2024-09-11T06:50:49.123 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.021834473, 2024-09-11T06:50:49.124 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.124 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.124 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:49.124 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:49.124 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:49.124 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:49.124 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 117 2024-09-11T06:50:49.124 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.124 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.124 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.124 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.124 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:49.124 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.124 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.125 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.125 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.125 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:49.125 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.125 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.125 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.125 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.125 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441725+0000", 2024-09-11T06:50:49.125 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5069999999999999e-06 2024-09-11T06:50:49.125 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.125 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.125 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.125 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441738+0000", 2024-09-11T06:50:49.125 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2813000000000001e-05 2024-09-11T06:50:49.126 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.126 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.126 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.126 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441759+0000", 2024-09-11T06:50:49.126 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1246999999999998e-05 2024-09-11T06:50:49.126 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.126 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.126 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.126 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441779+0000", 2024-09-11T06:50:49.126 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0392999999999999e-05 2024-09-11T06:50:49.126 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.126 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.126 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.126 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458804+0000", 2024-09-11T06:50:49.126 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.017024182999999998 2024-09-11T06:50:49.127 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.127 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.127 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:49.127 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458977+0000", 2024-09-11T06:50:49.127 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000173926 2024-09-11T06:50:49.127 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.127 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.127 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:49.127 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459170+0000", 2024-09-11T06:50:49.127 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000192184 2024-09-11T06:50:49.127 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.127 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.127 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:49.127 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.462300+0000", 2024-09-11T06:50:49.128 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031304969999999999 2024-09-11T06:50:49.128 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.128 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.128 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:49.128 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463408+0000", 2024-09-11T06:50:49.128 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011077839999999999 2024-09-11T06:50:49.128 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.128 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.128 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:49.128 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463497+0000", 2024-09-11T06:50:49.128 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8828999999999999e-05 2024-09-11T06:50:49.128 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.128 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.128 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.129 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463558+0000", 2024-09-11T06:50:49.129 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1110000000000003e-05 2024-09-11T06:50:49.129 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.129 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.129 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.129 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.129 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.129 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:49.129 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:49.129 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 102.25386099000001, 2024-09-11T06:50:49.129 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013136297, 2024-09-11T06:50:49.129 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.129 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.129 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:49.130 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:49.130 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:49.130 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:49.130 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 138 2024-09-11T06:50:49.130 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.130 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.130 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.130 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.130 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:49.130 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.130 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.130 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.130 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.130 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:49.130 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.131 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.131 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.131 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.131 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:49.131 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.89e-07 2024-09-11T06:50:49.131 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.131 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.131 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.131 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818369+0000", 2024-09-11T06:50:49.131 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4170000000000001e-06 2024-09-11T06:50:49.131 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.131 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.131 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.131 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818376+0000", 2024-09-11T06:50:49.132 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4270000000000002e-06 2024-09-11T06:50:49.132 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.132 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.132 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.132 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818383+0000", 2024-09-11T06:50:49.132 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3060000000000003e-06 2024-09-11T06:50:49.132 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.132 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.132 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.132 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824783+0000", 2024-09-11T06:50:49.132 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064003020000000001 2024-09-11T06:50:49.132 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.132 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.133 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:49.133 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825013+0000", 2024-09-11T06:50:49.133 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023039499999999999 2024-09-11T06:50:49.133 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.133 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.133 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:49.133 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825484+0000", 2024-09-11T06:50:49.133 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000470562 2024-09-11T06:50:49.133 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.133 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.133 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:49.133 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828633+0000", 2024-09-11T06:50:49.133 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031496689999999999 2024-09-11T06:50:49.133 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.133 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.134 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:49.134 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831413+0000", 2024-09-11T06:50:49.134 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027793219999999999 2024-09-11T06:50:49.134 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.134 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.134 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:49.134 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831450+0000", 2024-09-11T06:50:49.134 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7370999999999998e-05 2024-09-11T06:50:49.134 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.134 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.134 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.134 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831501+0000", 2024-09-11T06:50:49.134 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1036999999999999e-05 2024-09-11T06:50:49.134 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.134 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.135 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.135 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.135 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.135 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:49.135 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:49.135 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 102.25383651, 2024-09-11T06:50:49.135 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013223011, 2024-09-11T06:50:49.135 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.135 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.135 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:49.135 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:49.135 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:49.135 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:49.135 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 139 2024-09-11T06:50:49.136 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.136 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.136 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.136 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.136 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:49.136 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.136 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.136 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.136 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.136 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:49.136 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.136 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.136 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.136 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.137 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818390+0000", 2024-09-11T06:50:49.137 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5600000000000001e-07 2024-09-11T06:50:49.137 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.137 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.137 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.137 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818394+0000", 2024-09-11T06:50:49.137 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5929999999999999e-06 2024-09-11T06:50:49.137 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.137 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.137 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.137 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818399+0000", 2024-09-11T06:50:49.137 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7740000000000004e-06 2024-09-11T06:50:49.137 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.137 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.138 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.138 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818412+0000", 2024-09-11T06:50:49.138 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2625e-05 2024-09-11T06:50:49.138 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.138 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.138 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.138 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826205+0000", 2024-09-11T06:50:49.138 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0077930730000000002 2024-09-11T06:50:49.138 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.138 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.138 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:49.138 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826281+0000", 2024-09-11T06:50:49.138 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5876000000000003e-05 2024-09-11T06:50:49.138 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.139 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.139 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:49.139 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826384+0000", 2024-09-11T06:50:49.139 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103838 2024-09-11T06:50:49.139 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.139 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.139 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:49.139 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830642+0000", 2024-09-11T06:50:49.139 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004257083 2024-09-11T06:50:49.139 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.139 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.139 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:49.139 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831509+0000", 2024-09-11T06:50:49.139 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086730900000000003 2024-09-11T06:50:49.139 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.140 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.140 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:49.140 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831547+0000", 2024-09-11T06:50:49.140 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7793000000000002e-05 2024-09-11T06:50:49.140 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.140 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.140 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.140 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831612+0000", 2024-09-11T06:50:49.140 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5691000000000005e-05 2024-09-11T06:50:49.140 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.140 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.140 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.140 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.140 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.141 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:49.141 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:49.141 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 102.25380781600001, 2024-09-11T06:50:49.141 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016098024999999998, 2024-09-11T06:50:49.141 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.141 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.141 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:49.141 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:49.141 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:49.141 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:49.141 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 140 2024-09-11T06:50:49.141 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.141 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.141 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.142 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.142 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:49.142 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.142 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.142 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.142 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.142 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:49.142 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.142 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.142 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.142 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.142 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:49.142 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5199999999999998e-07 2024-09-11T06:50:49.142 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.142 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.143 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.143 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818479+0000", 2024-09-11T06:50:49.143 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0368000000000002e-05 2024-09-11T06:50:49.143 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.143 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.143 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.143 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818487+0000", 2024-09-11T06:50:49.143 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2339999999999994e-06 2024-09-11T06:50:49.143 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.143 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.143 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.143 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818491+0000", 2024-09-11T06:50:49.143 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4329999999999997e-06 2024-09-11T06:50:49.143 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.143 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.144 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.144 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828080+0000", 2024-09-11T06:50:49.144 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0095890449999999992 2024-09-11T06:50:49.144 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.144 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.144 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:49.144 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828130+0000", 2024-09-11T06:50:49.144 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9239000000000001e-05 2024-09-11T06:50:49.144 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.144 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.144 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:49.144 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828208+0000", 2024-09-11T06:50:49.144 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.852e-05 2024-09-11T06:50:49.144 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.145 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.145 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:49.145 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831625+0000", 2024-09-11T06:50:49.145 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034169410000000002 2024-09-11T06:50:49.145 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.145 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.145 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:49.145 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834411+0000", 2024-09-11T06:50:49.145 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002786111 2024-09-11T06:50:49.145 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.145 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.145 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:49.145 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834457+0000", 2024-09-11T06:50:49.145 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6041000000000002e-05 2024-09-11T06:50:49.146 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.146 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.146 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.146 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834516+0000", 2024-09-11T06:50:49.146 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8740999999999999e-05 2024-09-11T06:50:49.146 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.146 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.146 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.146 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.146 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.146 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:49.146 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:49.146 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 102.25372674899999, 2024-09-11T06:50:49.146 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016588772000000002, 2024-09-11T06:50:49.147 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.147 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.147 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:49.147 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:49.147 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:49.147 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:49.147 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 141 2024-09-11T06:50:49.147 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.147 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.147 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.147 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.147 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:49.147 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.147 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.147 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.148 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.148 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:49.148 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.148 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.148 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.148 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.148 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818500+0000", 2024-09-11T06:50:49.148 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0800000000000005e-07 2024-09-11T06:50:49.148 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.148 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.148 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.148 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818505+0000", 2024-09-11T06:50:49.148 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2429999999999996e-06 2024-09-11T06:50:49.148 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.149 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.149 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.149 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818510+0000", 2024-09-11T06:50:49.149 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9339999999999997e-06 2024-09-11T06:50:49.149 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.149 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.149 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.149 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818514+0000", 2024-09-11T06:50:49.149 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7270000000000001e-06 2024-09-11T06:50:49.149 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.149 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.149 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.149 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828711+0000", 2024-09-11T06:50:49.149 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010197031 2024-09-11T06:50:49.149 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.150 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.150 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:49.150 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828853+0000", 2024-09-11T06:50:49.150 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014201200000000001 2024-09-11T06:50:49.150 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.150 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.150 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:49.150 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.829115+0000", 2024-09-11T06:50:49.150 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026257500000000003 2024-09-11T06:50:49.150 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.150 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.150 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:49.150 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831629+0000", 2024-09-11T06:50:49.150 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025138169999999998 2024-09-11T06:50:49.151 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.151 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.151 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:49.151 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834991+0000", 2024-09-11T06:50:49.151 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033624459999999998 2024-09-11T06:50:49.151 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.151 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.151 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:49.151 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835032+0000", 2024-09-11T06:50:49.151 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1001000000000003e-05 2024-09-11T06:50:49.151 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.151 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.151 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.151 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835088+0000", 2024-09-11T06:50:49.151 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5478e-05 2024-09-11T06:50:49.152 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.152 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.152 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.152 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.152 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.152 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5235.0:1 2.5 2:b2cc3336:::rbd_id.testrbdnbd21607:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-11T06:50:49.152 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:50:49.152 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 14.512924727, 2024-09-11T06:50:49.152 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.68353462499999995, 2024-09-11T06:50:49.152 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.152 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.152 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:49.152 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:49.153 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5235", 2024-09-11T06:50:49.153 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2008977164", 2024-09-11T06:50:49.153 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:50:49.153 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.153 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.153 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.153 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.153 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:50:49.153 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.153 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.153 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.153 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.153 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:50:49.153 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.153 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.154 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.154 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.154 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559303+0000", 2024-09-11T06:50:49.154 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1830000000000001e-06 2024-09-11T06:50:49.154 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.154 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.154 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.154 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559314+0000", 2024-09-11T06:50:49.154 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0396e-05 2024-09-11T06:50:49.154 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.154 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.154 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.154 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559335+0000", 2024-09-11T06:50:49.154 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1183e-05 2024-09-11T06:50:49.154 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.155 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.155 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.155 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559354+0000", 2024-09-11T06:50:49.155 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9225999999999999e-05 2024-09-11T06:50:49.155 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.155 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.155 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.155 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559522+0000", 2024-09-11T06:50:49.155 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000168021 2024-09-11T06:50:49.155 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.155 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.155 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:50:49.155 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559543+0000", 2024-09-11T06:50:49.155 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0616999999999999e-05 2024-09-11T06:50:49.156 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.156 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.156 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.156 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.238584+0000", 2024-09-11T06:50:49.156 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.67904107499999999 2024-09-11T06:50:49.156 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.156 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.156 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for missing object", 2024-09-11T06:50:49.156 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.238933+0000", 2024-09-11T06:50:49.156 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034879600000000001 2024-09-11T06:50:49.156 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.156 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.156 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.156 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.241968+0000", 2024-09-11T06:50:49.157 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030350830000000001 2024-09-11T06:50:49.157 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.157 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.157 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:49.157 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.242281+0000", 2024-09-11T06:50:49.157 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00031299499999999999 2024-09-11T06:50:49.157 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.157 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.157 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.157 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.242836+0000", 2024-09-11T06:50:49.157 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00055504999999999997 2024-09-11T06:50:49.157 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.157 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.157 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.158 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.158 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.158 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5235.0:18 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 0~1024 [fadvise_sequential]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T06:50:49.158 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:50:49.158 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 13.810432287999999, 2024-09-11T06:50:49.158 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.048993760999999997, 2024-09-11T06:50:49.158 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.158 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.158 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:49.158 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:49.158 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5235", 2024-09-11T06:50:49.158 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2008977164", 2024-09-11T06:50:49.158 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 18 2024-09-11T06:50:49.158 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.158 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.159 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.159 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.159 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:50:49.159 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.159 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.159 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.159 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.159 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:50:49.159 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.159 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.159 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.159 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.159 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261797+0000", 2024-09-11T06:50:49.159 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2059999999999999e-06 2024-09-11T06:50:49.160 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.160 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.160 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.160 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261812+0000", 2024-09-11T06:50:49.160 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5159e-05 2024-09-11T06:50:49.160 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.160 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.160 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.160 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261852+0000", 2024-09-11T06:50:49.160 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.9906e-05 2024-09-11T06:50:49.160 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.160 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.160 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.160 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261865+0000", 2024-09-11T06:50:49.161 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.312e-05 2024-09-11T06:50:49.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.161 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.161 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.161 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.263166+0000", 2024-09-11T06:50:49.161 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001301134 2024-09-11T06:50:49.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.161 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.161 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for missing object", 2024-09-11T06:50:49.161 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.263529+0000", 2024-09-11T06:50:49.161 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036279900000000002 2024-09-11T06:50:49.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.161 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.161 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.161 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.300567+0000", 2024-09-11T06:50:49.162 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.037038326000000003 2024-09-11T06:50:49.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.162 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.162 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:49.162 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.300800+0000", 2024-09-11T06:50:49.162 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023323300000000001 2024-09-11T06:50:49.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.162 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.162 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.162 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.306709+0000", 2024-09-11T06:50:49.162 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0059087690000000003 2024-09-11T06:50:49.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.162 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.162 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:49.163 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.306850+0000", 2024-09-11T06:50:49.163 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014106200000000001 2024-09-11T06:50:49.163 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.163 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.163 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:49.163 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.307121+0000", 2024-09-11T06:50:49.163 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027083599999999999 2024-09-11T06:50:49.163 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.163 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.163 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:49.163 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.308809+0000", 2024-09-11T06:50:49.163 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0016880759999999999 2024-09-11T06:50:49.163 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.163 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.163 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:49.164 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310382+0000", 2024-09-11T06:50:49.164 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0015732840000000001 2024-09-11T06:50:49.164 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.164 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.164 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:49.164 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310501+0000", 2024-09-11T06:50:49.164 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000118276 2024-09-11T06:50:49.164 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.164 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.164 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.164 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310787+0000", 2024-09-11T06:50:49.164 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00028657500000000001 2024-09-11T06:50:49.164 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.164 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.165 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.165 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.165 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:70 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 393216~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:50:49.165 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596118+0000", 2024-09-11T06:50:49.165 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.47610788599999998, 2024-09-11T06:50:49.165 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012279145, 2024-09-11T06:50:49.165 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.165 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.165 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:49.165 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:49.165 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:50:49.165 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:50:49.165 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 70 2024-09-11T06:50:49.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.166 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.166 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.166 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.166 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596118+0000", 2024-09-11T06:50:49.166 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.166 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.166 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.166 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596118+0000", 2024-09-11T06:50:49.166 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.166 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.166 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.167 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596119+0000", 2024-09-11T06:50:49.167 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0299999999999997e-07 2024-09-11T06:50:49.167 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.167 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.167 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.167 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596238+0000", 2024-09-11T06:50:49.167 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000118798 2024-09-11T06:50:49.167 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.167 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.167 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.167 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596250+0000", 2024-09-11T06:50:49.167 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2775999999999999e-05 2024-09-11T06:50:49.167 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.167 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.167 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.168 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596259+0000", 2024-09-11T06:50:49.168 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3100000000000001e-06 2024-09-11T06:50:49.168 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.168 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.168 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.168 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.603389+0000", 2024-09-11T06:50:49.168 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0071300759999999999 2024-09-11T06:50:49.168 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.168 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.168 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:49.168 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.603540+0000", 2024-09-11T06:50:49.168 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015111700000000001 2024-09-11T06:50:49.168 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.168 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.169 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:49.169 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.603766+0000", 2024-09-11T06:50:49.169 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022570899999999999 2024-09-11T06:50:49.169 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.169 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.169 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:49.169 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.606676+0000", 2024-09-11T06:50:49.169 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0029106380000000001 2024-09-11T06:50:49.169 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.169 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.169 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:49.169 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608173+0000", 2024-09-11T06:50:49.169 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014967629999999999 2024-09-11T06:50:49.169 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.169 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.170 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:49.170 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608277+0000", 2024-09-11T06:50:49.170 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000104398 2024-09-11T06:50:49.170 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.170 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.170 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.170 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608397+0000", 2024-09-11T06:50:49.170 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000119657 2024-09-11T06:50:49.170 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.170 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.170 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.170 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.170 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.170 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:71 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 524288~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:50:49.171 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596272+0000", 2024-09-11T06:50:49.171 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.47595363299999999, 2024-09-11T06:50:49.171 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013390681999999999, 2024-09-11T06:50:49.171 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.171 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.171 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:49.171 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:49.171 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:50:49.171 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:50:49.171 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 71 2024-09-11T06:50:49.171 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.171 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.171 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.171 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.172 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596272+0000", 2024-09-11T06:50:49.172 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.172 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.172 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.172 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.172 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596272+0000", 2024-09-11T06:50:49.172 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.172 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.172 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.172 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.172 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596273+0000", 2024-09-11T06:50:49.172 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.9999999999999996e-07 2024-09-11T06:50:49.172 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.172 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.173 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.173 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596395+0000", 2024-09-11T06:50:49.173 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012204799999999999 2024-09-11T06:50:49.173 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.173 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.173 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.173 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596411+0000", 2024-09-11T06:50:49.173 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5746000000000001e-05 2024-09-11T06:50:49.173 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.173 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.174 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.174 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596419+0000", 2024-09-11T06:50:49.174 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2600000000000005e-06 2024-09-11T06:50:49.174 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.174 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.174 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.174 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.605311+0000", 2024-09-11T06:50:49.174 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0088915179999999993 2024-09-11T06:50:49.174 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.174 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.174 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:49.174 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.605436+0000", 2024-09-11T06:50:49.174 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000125203 2024-09-11T06:50:49.174 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.175 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.175 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:49.175 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.605625+0000", 2024-09-11T06:50:49.175 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018961299999999999 2024-09-11T06:50:49.175 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.175 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.175 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:49.175 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609477+0000", 2024-09-11T06:50:49.175 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0038513549999999999 2024-09-11T06:50:49.175 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.175 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.175 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:49.175 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609499+0000", 2024-09-11T06:50:49.175 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2215e-05 2024-09-11T06:50:49.175 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.176 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.176 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:49.176 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609589+0000", 2024-09-11T06:50:49.176 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0029000000000001e-05 2024-09-11T06:50:49.176 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.176 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.176 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.176 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609663+0000", 2024-09-11T06:50:49.176 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3894999999999996e-05 2024-09-11T06:50:49.176 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.176 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.176 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.176 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.176 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.177 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:72 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 655360~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:50:49.177 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596432+0000", 2024-09-11T06:50:49.177 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.47579414599999997, 2024-09-11T06:50:49.177 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01414727, 2024-09-11T06:50:49.177 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.177 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.177 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:49.177 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:49.177 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:50:49.177 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:50:49.177 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 72 2024-09-11T06:50:49.177 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.177 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.177 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.178 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.178 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596432+0000", 2024-09-11T06:50:49.178 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.178 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.178 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.178 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.178 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596432+0000", 2024-09-11T06:50:49.178 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.178 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.178 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.178 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.178 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596433+0000", 2024-09-11T06:50:49.178 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1999999999999998e-07 2024-09-11T06:50:49.178 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.178 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.179 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.179 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596556+0000", 2024-09-11T06:50:49.179 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012379000000000001 2024-09-11T06:50:49.179 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.179 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.179 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.179 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596571+0000", 2024-09-11T06:50:49.179 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4749999999999999e-05 2024-09-11T06:50:49.179 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.179 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.179 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.179 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596579+0000", 2024-09-11T06:50:49.179 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2500000000000006e-06 2024-09-11T06:50:49.179 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.606784+0000", 2024-09-11T06:50:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010204757 2024-09-11T06:50:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.606950+0000", 2024-09-11T06:50:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000166229 2024-09-11T06:50:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.607176+0000", 2024-09-11T06:50:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022549899999999999 2024-09-11T06:50:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610363+0000", 2024-09-11T06:50:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031874690000000001 2024-09-11T06:50:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610490+0000", 2024-09-11T06:50:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012639899999999999 2024-09-11T06:50:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610529+0000", 2024-09-11T06:50:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.8936000000000001e-05 2024-09-11T06:50:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610579+0000", 2024-09-11T06:50:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0371000000000003e-05 2024-09-11T06:50:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:73 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 786432~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:50:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596592+0000", 2024-09-11T06:50:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.47563389299999997, 2024-09-11T06:50:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014914779, 2024-09-11T06:50:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:50:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:50:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 73 2024-09-11T06:50:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596592+0000", 2024-09-11T06:50:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596592+0000", 2024-09-11T06:50:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596593+0000", 2024-09-11T06:50:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.0699999999999996e-07 2024-09-11T06:50:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596786+0000", 2024-09-11T06:50:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019278200000000001 2024-09-11T06:50:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596801+0000", 2024-09-11T06:50:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5690000000000001e-05 2024-09-11T06:50:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596815+0000", 2024-09-11T06:50:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3753e-05 2024-09-11T06:50:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608442+0000", 2024-09-11T06:50:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011627385 2024-09-11T06:50:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608570+0000", 2024-09-11T06:50:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012740399999999999 2024-09-11T06:50:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608754+0000", 2024-09-11T06:50:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018392899999999999 2024-09-11T06:50:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610370+0000", 2024-09-11T06:50:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001615998 2024-09-11T06:50:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611420+0000", 2024-09-11T06:50:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010502790000000001 2024-09-11T06:50:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611458+0000", 2024-09-11T06:50:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7749000000000001e-05 2024-09-11T06:50:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611507+0000", 2024-09-11T06:50:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9002999999999998e-05 2024-09-11T06:50:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:74 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 917504~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:50:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596849+0000", 2024-09-11T06:50:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.475377255, 2024-09-11T06:50:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015202963999999999, 2024-09-11T06:50:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:50:49.189 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:50:49.189 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 74 2024-09-11T06:50:49.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.189 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.189 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.189 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.189 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596849+0000", 2024-09-11T06:50:49.189 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.189 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.189 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.189 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596849+0000", 2024-09-11T06:50:49.189 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.190 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.190 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.190 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596850+0000", 2024-09-11T06:50:49.190 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3170000000000001e-06 2024-09-11T06:50:49.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.190 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.190 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.190 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.597118+0000", 2024-09-11T06:50:49.190 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026772999999999999 2024-09-11T06:50:49.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.190 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.190 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.190 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.597136+0000", 2024-09-11T06:50:49.190 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8467000000000001e-05 2024-09-11T06:50:49.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.191 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.191 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.191 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.597148+0000", 2024-09-11T06:50:49.191 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1486e-05 2024-09-11T06:50:49.191 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.191 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.191 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.191 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609714+0000", 2024-09-11T06:50:49.191 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012566124 2024-09-11T06:50:49.191 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.191 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.191 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:49.191 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609792+0000", 2024-09-11T06:50:49.191 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.8579999999999996e-05 2024-09-11T06:50:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609899+0000", 2024-09-11T06:50:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00010716499999999999 2024-09-11T06:50:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610865+0000", 2024-09-11T06:50:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00096540299999999999 2024-09-11T06:50:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611970+0000", 2024-09-11T06:50:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001105118 2024-09-11T06:50:49.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.193 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.193 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:49.193 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.612005+0000", 2024-09-11T06:50:49.193 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4789000000000002e-05 2024-09-11T06:50:49.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.193 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.193 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.193 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.612052+0000", 2024-09-11T06:50:49.193 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6785e-05 2024-09-11T06:50:49.193 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.193 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.193 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.193 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.194 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.194 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:49.195 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:50:49.195 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:49.191+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:49.296 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:49.291+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:49.325 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T06:50:49.366 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:49.367 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:49.367 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:50:49.367 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:49.381 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:50:49.428 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:49.427+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:49.496 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:49.491+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:49.498 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:49.498 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:49.498 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:50:49.498 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:50:49.498 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:49.512 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:50:49.597 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:49.595+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:49.688 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:49.688 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:49.688 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:49.688 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:50:49.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.689 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4558.0:1 3.3 3:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-11T06:50:49.689 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:50:49.689 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 32.594514721000003, 2024-09-11T06:50:49.689 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0085520600000000002, 2024-09-11T06:50:49.689 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.689 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.689 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:49.689 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:49.689 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4558", 2024-09-11T06:50:49.689 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1478388898", 2024-09-11T06:50:49.689 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:50:49.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.689 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.690 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.690 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:50:49.690 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.690 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.690 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:50:49.690 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.690 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.690 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090533+0000", 2024-09-11T06:50:49.691 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2629999999999999e-06 2024-09-11T06:50:49.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.691 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.691 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090546+0000", 2024-09-11T06:50:49.691 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2918999999999999e-05 2024-09-11T06:50:49.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.691 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.691 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090572+0000", 2024-09-11T06:50:49.691 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6412000000000001e-05 2024-09-11T06:50:49.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.691 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.692 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090594+0000", 2024-09-11T06:50:49.692 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2476e-05 2024-09-11T06:50:49.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.692 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.692 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090938+0000", 2024-09-11T06:50:49.692 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034325099999999998 2024-09-11T06:50:49.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.692 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:49.692 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.091149+0000", 2024-09-11T06:50:49.692 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021151300000000001 2024-09-11T06:50:49.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.693 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:49.693 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.091377+0000", 2024-09-11T06:50:49.693 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022785399999999999 2024-09-11T06:50:49.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.693 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:49.693 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.092912+0000", 2024-09-11T06:50:49.693 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0015346089999999999 2024-09-11T06:50:49.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.693 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:49.693 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098919+0000", 2024-09-11T06:50:49.693 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0060076799999999996 2024-09-11T06:50:49.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.694 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:49.694 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.099009+0000", 2024-09-11T06:50:49.694 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0056000000000006e-05 2024-09-11T06:50:49.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.694 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.694 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.099081+0000", 2024-09-11T06:50:49.694 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2026999999999993e-05 2024-09-11T06:50:49.694 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.694 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.694 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.695 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:50:49.695 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:50:49.695 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 26.553025641000001, 2024-09-11T06:50:49.695 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0071924390000000001, 2024-09-11T06:50:49.695 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.695 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.695 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:49.695 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.695 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.695 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.695 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:50:49.695 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.695 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.695 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.696 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.696 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:50:49.696 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.696 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.696 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.696 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132020+0000", 2024-09-11T06:50:49.696 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.846e-06 2024-09-11T06:50:49.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.696 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.696 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.696 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135688+0000", 2024-09-11T06:50:49.696 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036677350000000001 2024-09-11T06:50:49.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.696 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.697 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.697 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135747+0000", 2024-09-11T06:50:49.697 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.9314999999999997e-05 2024-09-11T06:50:49.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.697 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.697 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.697 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135797+0000", 2024-09-11T06:50:49.697 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9984000000000001e-05 2024-09-11T06:50:49.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.697 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.697 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.697 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136143+0000", 2024-09-11T06:50:49.697 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034594299999999998 2024-09-11T06:50:49.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.698 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.698 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:49.698 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136179+0000", 2024-09-11T06:50:49.698 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5636000000000002e-05 2024-09-11T06:50:49.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.698 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.698 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.698 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.139211+0000", 2024-09-11T06:50:49.698 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030319800000000001 2024-09-11T06:50:49.698 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.698 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.698 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.698 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.699 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:50:49.699 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:50:49.699 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 26.550962386999998, 2024-09-11T06:50:49.699 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064954139999999997, 2024-09-11T06:50:49.699 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.699 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.699 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:49.699 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.699 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.699 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:50:49.699 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.699 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.700 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:50:49.700 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.700 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.700 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.700 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134084+0000", 2024-09-11T06:50:49.700 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.894e-06 2024-09-11T06:50:49.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.700 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.700 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.700 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136513+0000", 2024-09-11T06:50:49.700 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024290969999999999 2024-09-11T06:50:49.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.700 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.701 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.701 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136554+0000", 2024-09-11T06:50:49.701 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.0932e-05 2024-09-11T06:50:49.701 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.701 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.701 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.701 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136584+0000", 2024-09-11T06:50:49.701 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0056000000000002e-05 2024-09-11T06:50:49.701 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.701 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.701 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.701 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136805+0000", 2024-09-11T06:50:49.701 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022086100000000001 2024-09-11T06:50:49.701 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.701 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.702 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:49.702 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136864+0000", 2024-09-11T06:50:49.702 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.9907000000000001e-05 2024-09-11T06:50:49.702 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.702 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.702 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.702 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.140577+0000", 2024-09-11T06:50:49.702 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0037126669999999998 2024-09-11T06:50:49.702 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.702 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.702 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.702 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.702 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.702 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:50:49.702 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:50:49.703 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 26.542671587000001, 2024-09-11T06:50:49.703 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035763180000000002, 2024-09-11T06:50:49.703 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.703 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.703 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:49.703 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.703 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.703 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.703 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:50:49.703 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.703 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.703 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.703 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:50:49.703 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.704 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.704 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.704 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.704 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142374+0000", 2024-09-11T06:50:49.704 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5460000000000001e-06 2024-09-11T06:50:49.704 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.704 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.704 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.704 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143902+0000", 2024-09-11T06:50:49.704 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00152808 2024-09-11T06:50:49.704 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.704 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.704 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.704 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143946+0000", 2024-09-11T06:50:49.704 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3874e-05 2024-09-11T06:50:49.704 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.705 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.705 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.705 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143978+0000", 2024-09-11T06:50:49.705 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2425999999999998e-05 2024-09-11T06:50:49.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.705 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.705 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.705 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.144177+0000", 2024-09-11T06:50:49.705 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019837500000000001 2024-09-11T06:50:49.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.705 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.705 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:49.705 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.144198+0000", 2024-09-11T06:50:49.706 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1639000000000001e-05 2024-09-11T06:50:49.706 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.706 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.706 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.706 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.145949+0000", 2024-09-11T06:50:49.706 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0017503779999999999 2024-09-11T06:50:49.706 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.706 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.706 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.706 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.706 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.706 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:50:49.706 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:50:49.706 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 26.538355885000001, 2024-09-11T06:50:49.707 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0057693220000000003, 2024-09-11T06:50:49.707 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.707 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.707 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:49.707 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.707 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.707 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.707 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:50:49.707 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.707 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.707 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.707 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.707 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:50:49.707 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.707 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.708 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.708 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.708 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146690+0000", 2024-09-11T06:50:49.708 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.607e-06 2024-09-11T06:50:49.708 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.708 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.708 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.708 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148693+0000", 2024-09-11T06:50:49.708 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0020029739999999998 2024-09-11T06:50:49.708 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.708 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.708 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.708 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148735+0000", 2024-09-11T06:50:49.708 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1958e-05 2024-09-11T06:50:49.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.709 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.709 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.709 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148764+0000", 2024-09-11T06:50:49.709 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8999000000000001e-05 2024-09-11T06:50:49.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.709 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.709 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.709 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149069+0000", 2024-09-11T06:50:49.709 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000305032 2024-09-11T06:50:49.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.709 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.709 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:49.710 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149204+0000", 2024-09-11T06:50:49.710 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013495 2024-09-11T06:50:49.710 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.710 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.710 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.710 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.152458+0000", 2024-09-11T06:50:49.710 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0032538020000000001 2024-09-11T06:50:49.710 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.710 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.710 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.710 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.710 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.710 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:50:49.711 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:50:49.711 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 26.537457344, 2024-09-11T06:50:49.711 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0042425329999999997, 2024-09-11T06:50:49.711 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.711 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.711 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:49.711 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.711 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.711 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.711 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:50:49.711 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.711 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.711 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.711 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.712 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:50:49.712 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.712 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.712 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.712 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147588+0000", 2024-09-11T06:50:49.712 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.277e-06 2024-09-11T06:50:49.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.712 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.712 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.712 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149051+0000", 2024-09-11T06:50:49.712 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001462669 2024-09-11T06:50:49.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.712 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.713 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.713 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149090+0000", 2024-09-11T06:50:49.713 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.8967999999999997e-05 2024-09-11T06:50:49.713 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.713 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.713 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.713 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149120+0000", 2024-09-11T06:50:49.713 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0766000000000003e-05 2024-09-11T06:50:49.713 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.713 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.713 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.713 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149395+0000", 2024-09-11T06:50:49.713 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027484900000000002 2024-09-11T06:50:49.713 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.714 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.714 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:49.714 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149439+0000", 2024-09-11T06:50:49.714 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3940999999999998e-05 2024-09-11T06:50:49.714 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.714 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.714 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.714 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.151829+0000", 2024-09-11T06:50:49.714 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002390063 2024-09-11T06:50:49.714 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.714 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.714 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.714 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.714 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.714 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:50:49.715 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:50:49.715 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 26.531316712999999, 2024-09-11T06:50:49.715 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0046568219999999997, 2024-09-11T06:50:49.715 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.715 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.715 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:49.715 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.715 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.715 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.715 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:50:49.715 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.715 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.715 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.716 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.716 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:50:49.716 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.716 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.716 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.716 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153729+0000", 2024-09-11T06:50:49.716 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5629999999999999e-06 2024-09-11T06:50:49.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.716 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.716 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.716 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155179+0000", 2024-09-11T06:50:49.716 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001450469 2024-09-11T06:50:49.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.717 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.717 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.717 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155221+0000", 2024-09-11T06:50:49.717 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1448e-05 2024-09-11T06:50:49.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.717 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.717 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.717 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155249+0000", 2024-09-11T06:50:49.717 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8433e-05 2024-09-11T06:50:49.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.717 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.717 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.717 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155420+0000", 2024-09-11T06:50:49.717 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017070599999999999 2024-09-11T06:50:49.718 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.718 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.718 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:49.718 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155445+0000", 2024-09-11T06:50:49.718 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4813000000000001e-05 2024-09-11T06:50:49.718 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.718 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.718 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.718 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.158384+0000", 2024-09-11T06:50:49.718 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0029393900000000001 2024-09-11T06:50:49.718 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.718 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.718 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.718 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.719 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.719 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:50:49.719 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:50:49.719 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 26.529289964, 2024-09-11T06:50:49.719 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0046826020000000001, 2024-09-11T06:50:49.719 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.719 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.719 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:49.719 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.719 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.719 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.719 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:50:49.719 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.719 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.720 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.720 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.720 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:50:49.720 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.720 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.720 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.720 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.720 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155756+0000", 2024-09-11T06:50:49.720 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.384e-06 2024-09-11T06:50:49.720 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.720 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.720 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.720 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157205+0000", 2024-09-11T06:50:49.720 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014496260000000001 2024-09-11T06:50:49.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.721 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.721 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.721 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157252+0000", 2024-09-11T06:50:49.721 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6368000000000001e-05 2024-09-11T06:50:49.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.721 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.721 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.721 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157286+0000", 2024-09-11T06:50:49.721 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4944999999999998e-05 2024-09-11T06:50:49.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.721 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.721 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.721 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157540+0000", 2024-09-11T06:50:49.722 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00025307999999999998 2024-09-11T06:50:49.722 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.722 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.722 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:49.722 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157576+0000", 2024-09-11T06:50:49.722 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.6878999999999997e-05 2024-09-11T06:50:49.722 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.722 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.722 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.722 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.160437+0000", 2024-09-11T06:50:49.722 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00286032 2024-09-11T06:50:49.722 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.722 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.722 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.723 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.723 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.723 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:50:49.723 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.280679+0000", 2024-09-11T06:50:49.723 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 14.404364975, 2024-09-11T06:50:49.723 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0040934309999999998, 2024-09-11T06:50:49.723 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.723 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.723 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:49.723 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.723 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.723 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.723 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280679+0000", 2024-09-11T06:50:49.724 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.724 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.724 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.724 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.724 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280679+0000", 2024-09-11T06:50:49.724 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.724 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.724 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.724 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.724 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280681+0000", 2024-09-11T06:50:49.724 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2529999999999999e-06 2024-09-11T06:50:49.724 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.724 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.725 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.725 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280713+0000", 2024-09-11T06:50:49.725 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.1936000000000001e-05 2024-09-11T06:50:49.725 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.725 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.725 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.725 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280758+0000", 2024-09-11T06:50:49.725 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5124999999999999e-05 2024-09-11T06:50:49.725 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.725 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.725 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.725 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280779+0000", 2024-09-11T06:50:49.725 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0936000000000001e-05 2024-09-11T06:50:49.725 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.726 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.726 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.726 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280932+0000", 2024-09-11T06:50:49.726 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015220899999999999 2024-09-11T06:50:49.726 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.726 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.726 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.726 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.284773+0000", 2024-09-11T06:50:49.726 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0038409719999999998 2024-09-11T06:50:49.726 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.726 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.726 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.726 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:50:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.292943+0000", 2024-09-11T06:50:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 14.392100781, 2024-09-11T06:50:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0028686010000000001, 2024-09-11T06:50:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.292943+0000", 2024-09-11T06:50:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.292943+0000", 2024-09-11T06:50:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.292945+0000", 2024-09-11T06:50:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7799999999999999e-06 2024-09-11T06:50:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.292968+0000", 2024-09-11T06:50:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2586e-05 2024-09-11T06:50:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.292998+0000", 2024-09-11T06:50:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9893e-05 2024-09-11T06:50:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293012+0000", 2024-09-11T06:50:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4504999999999999e-05 2024-09-11T06:50:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293155+0000", 2024-09-11T06:50:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000142993 2024-09-11T06:50:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.295812+0000", 2024-09-11T06:50:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0026568439999999998 2024-09-11T06:50:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:50:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.293438+0000", 2024-09-11T06:50:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 14.391606239, 2024-09-11T06:50:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0061479350000000002, 2024-09-11T06:50:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293438+0000", 2024-09-11T06:50:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293438+0000", 2024-09-11T06:50:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293439+0000", 2024-09-11T06:50:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:50:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293459+0000", 2024-09-11T06:50:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9922e-05 2024-09-11T06:50:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293482+0000", 2024-09-11T06:50:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2969e-05 2024-09-11T06:50:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293494+0000", 2024-09-11T06:50:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.131e-05 2024-09-11T06:50:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.295901+0000", 2024-09-11T06:50:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024077949999999999 2024-09-11T06:50:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.299586+0000", 2024-09-11T06:50:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036845020000000001 2024-09-11T06:50:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:50:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.309418+0000", 2024-09-11T06:50:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 14.375626208, 2024-09-11T06:50:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031185879999999998, 2024-09-11T06:50:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.309418+0000", 2024-09-11T06:50:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.309418+0000", 2024-09-11T06:50:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.309419+0000", 2024-09-11T06:50:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4699999999999999e-06 2024-09-11T06:50:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.309441+0000", 2024-09-11T06:50:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1668999999999999e-05 2024-09-11T06:50:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.309463+0000", 2024-09-11T06:50:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2106000000000001e-05 2024-09-11T06:50:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.309477+0000", 2024-09-11T06:50:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3482999999999999e-05 2024-09-11T06:50:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310475+0000", 2024-09-11T06:50:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00099831099999999999 2024-09-11T06:50:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.312536+0000", 2024-09-11T06:50:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0020615490000000002 2024-09-11T06:50:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:50:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.315262+0000", 2024-09-11T06:50:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 14.369782401, 2024-09-11T06:50:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0028909550000000002, 2024-09-11T06:50:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.315262+0000", 2024-09-11T06:50:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.315262+0000", 2024-09-11T06:50:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.315264+0000", 2024-09-11T06:50:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1629999999999999e-06 2024-09-11T06:50:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.315291+0000", 2024-09-11T06:50:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7489000000000001e-05 2024-09-11T06:50:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.315329+0000", 2024-09-11T06:50:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7175999999999998e-05 2024-09-11T06:50:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.315348+0000", 2024-09-11T06:50:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9596e-05 2024-09-11T06:50:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.315478+0000", 2024-09-11T06:50:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129975 2024-09-11T06:50:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.318153+0000", 2024-09-11T06:50:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0026745559999999998 2024-09-11T06:50:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:50:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.324823+0000", 2024-09-11T06:50:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 14.360221340000001, 2024-09-11T06:50:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033143130000000002, 2024-09-11T06:50:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.324823+0000", 2024-09-11T06:50:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.324823+0000", 2024-09-11T06:50:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.324825+0000", 2024-09-11T06:50:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.88e-06 2024-09-11T06:50:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.324851+0000", 2024-09-11T06:50:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6126e-05 2024-09-11T06:50:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.324882+0000", 2024-09-11T06:50:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0778999999999998e-05 2024-09-11T06:50:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.324902+0000", 2024-09-11T06:50:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0046000000000001e-05 2024-09-11T06:50:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.325015+0000", 2024-09-11T06:50:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000113739 2024-09-11T06:50:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.328137+0000", 2024-09-11T06:50:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031217430000000002 2024-09-11T06:50:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:50:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.327802+0000", 2024-09-11T06:50:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 14.357242325, 2024-09-11T06:50:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027834639999999998, 2024-09-11T06:50:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.327802+0000", 2024-09-11T06:50:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.327802+0000", 2024-09-11T06:50:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.327804+0000", 2024-09-11T06:50:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7370000000000001e-06 2024-09-11T06:50:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.327828+0000", 2024-09-11T06:50:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4831999999999999e-05 2024-09-11T06:50:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.327854+0000", 2024-09-11T06:50:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5525999999999999e-05 2024-09-11T06:50:49.748 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.748 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.748 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.748 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.327870+0000", 2024-09-11T06:50:49.748 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6330000000000001e-05 2024-09-11T06:50:49.748 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.748 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.748 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.748 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.328230+0000", 2024-09-11T06:50:49.748 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036026800000000001 2024-09-11T06:50:49.748 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.748 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.748 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.748 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.330585+0000", 2024-09-11T06:50:49.749 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0023547709999999999 2024-09-11T06:50:49.749 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.749 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.749 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.749 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.749 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.749 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.7 e70/69 cost 1048576)", 2024-09-11T06:50:49.749 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.343042+0000", 2024-09-11T06:50:49.749 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 14.342001721999999, 2024-09-11T06:50:49.749 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031815010000000002, 2024-09-11T06:50:49.749 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.749 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.749 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:49.749 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.750 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.750 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.750 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.343042+0000", 2024-09-11T06:50:49.750 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.750 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.750 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.750 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.750 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.343042+0000", 2024-09-11T06:50:49.750 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.750 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.750 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.750 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.750 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.343044+0000", 2024-09-11T06:50:49.750 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.716e-06 2024-09-11T06:50:49.751 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.751 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.751 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.751 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.343071+0000", 2024-09-11T06:50:49.751 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.654e-05 2024-09-11T06:50:49.751 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.751 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.751 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.751 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.343098+0000", 2024-09-11T06:50:49.751 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7529e-05 2024-09-11T06:50:49.751 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.751 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.751 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.752 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.343115+0000", 2024-09-11T06:50:49.752 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6629e-05 2024-09-11T06:50:49.752 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.752 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.752 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.752 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.343229+0000", 2024-09-11T06:50:49.752 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011443900000000001 2024-09-11T06:50:49.752 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.752 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.752 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.752 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.346224+0000", 2024-09-11T06:50:49.752 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0029946479999999999 2024-09-11T06:50:49.752 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.752 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.753 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.753 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.753 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.753 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.7 e70/69 cost 1048576)", 2024-09-11T06:50:49.753 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.352608+0000", 2024-09-11T06:50:49.753 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 14.332436438, 2024-09-11T06:50:49.753 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0032189060000000001, 2024-09-11T06:50:49.753 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.753 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.753 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:49.753 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.753 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.753 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.753 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.352608+0000", 2024-09-11T06:50:49.754 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.754 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.754 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.754 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.754 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.352608+0000", 2024-09-11T06:50:49.754 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.754 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.754 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.754 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.754 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.352610+0000", 2024-09-11T06:50:49.754 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.627e-06 2024-09-11T06:50:49.754 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.754 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.754 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.755 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.352641+0000", 2024-09-11T06:50:49.755 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0442000000000001e-05 2024-09-11T06:50:49.755 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.755 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.755 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.755 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.352673+0000", 2024-09-11T06:50:49.755 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2493000000000002e-05 2024-09-11T06:50:49.755 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.755 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.755 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.755 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.352693+0000", 2024-09-11T06:50:49.755 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9911999999999998e-05 2024-09-11T06:50:49.755 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.756 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.756 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.756 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.352785+0000", 2024-09-11T06:50:49.756 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.1539999999999994e-05 2024-09-11T06:50:49.756 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.756 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.756 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.756 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.355827+0000", 2024-09-11T06:50:49.756 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003041892 2024-09-11T06:50:49.756 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.756 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.756 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.756 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.757 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.757 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.7 e70/69 cost 1048576)", 2024-09-11T06:50:49.757 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.379470+0000", 2024-09-11T06:50:49.757 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 14.305573645000001, 2024-09-11T06:50:49.757 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0029267730000000001, 2024-09-11T06:50:49.757 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.757 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.757 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:49.757 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.757 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.757 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.757 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.379470+0000", 2024-09-11T06:50:49.757 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.758 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.758 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.758 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.758 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.379470+0000", 2024-09-11T06:50:49.758 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.758 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.758 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.758 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.758 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.379472+0000", 2024-09-11T06:50:49.758 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6729999999999999e-06 2024-09-11T06:50:49.758 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.758 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.758 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.758 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.379494+0000", 2024-09-11T06:50:49.759 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1823000000000001e-05 2024-09-11T06:50:49.759 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.759 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.759 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.759 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.379521+0000", 2024-09-11T06:50:49.759 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7161999999999998e-05 2024-09-11T06:50:49.759 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.759 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.759 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.759 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.379538+0000", 2024-09-11T06:50:49.759 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7212999999999999e-05 2024-09-11T06:50:49.759 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.759 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.760 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.760 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.379662+0000", 2024-09-11T06:50:49.760 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000123776 2024-09-11T06:50:49.760 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.760 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.760 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.760 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.382397+0000", 2024-09-11T06:50:49.760 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027351260000000001 2024-09-11T06:50:49.760 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.760 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.760 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.760 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.760 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.760 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.7 e70/69 cost 1048576)", 2024-09-11T06:50:49.761 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.466322+0000", 2024-09-11T06:50:49.761 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 14.218721959, 2024-09-11T06:50:49.761 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002802531, 2024-09-11T06:50:49.761 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.761 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.761 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:49.761 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.761 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.761 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.761 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.466322+0000", 2024-09-11T06:50:49.761 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.761 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.761 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.761 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.761 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.466322+0000", 2024-09-11T06:50:49.762 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.762 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.762 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.762 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.762 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.466324+0000", 2024-09-11T06:50:49.762 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7770000000000001e-06 2024-09-11T06:50:49.762 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.762 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.762 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.762 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.466345+0000", 2024-09-11T06:50:49.762 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0716e-05 2024-09-11T06:50:49.762 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.762 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.762 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.763 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.466372+0000", 2024-09-11T06:50:49.763 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7583e-05 2024-09-11T06:50:49.763 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.763 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.763 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.763 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.466386+0000", 2024-09-11T06:50:49.763 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3532e-05 2024-09-11T06:50:49.763 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.763 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.763 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.763 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.466545+0000", 2024-09-11T06:50:49.763 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000159333 2024-09-11T06:50:49.763 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.763 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.764 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.764 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.469125+0000", 2024-09-11T06:50:49.764 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025795900000000001 2024-09-11T06:50:49.764 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.764 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.764 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.764 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.764 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.764 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.7 e70/69 cost 1048576)", 2024-09-11T06:50:49.764 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.497220+0000", 2024-09-11T06:50:49.764 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 14.187824304999999, 2024-09-11T06:50:49.765 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027665979999999999, 2024-09-11T06:50:49.765 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:49.765 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:49.765 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:49.765 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:49.765 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.765 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:49.765 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.497220+0000", 2024-09-11T06:50:49.765 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.765 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.765 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.765 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:49.765 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.497220+0000", 2024-09-11T06:50:49.766 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:49.766 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.766 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.766 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:49.766 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.497222+0000", 2024-09-11T06:50:49.766 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.883e-06 2024-09-11T06:50:49.766 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.766 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.766 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:49.766 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.497250+0000", 2024-09-11T06:50:49.766 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8293000000000002e-05 2024-09-11T06:50:49.766 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.766 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.767 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:49.767 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.497286+0000", 2024-09-11T06:50:49.767 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5651999999999997e-05 2024-09-11T06:50:49.767 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.767 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.767 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:49.767 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.497303+0000", 2024-09-11T06:50:49.767 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7767000000000001e-05 2024-09-11T06:50:49.767 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.767 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.767 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:49.767 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.497424+0000", 2024-09-11T06:50:49.767 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000120432 2024-09-11T06:50:49.767 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:49.767 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:49.768 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:49.768 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.499986+0000", 2024-09-11T06:50:49.768 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002562571 2024-09-11T06:50:49.768 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.768 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.768 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.768 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:49.768 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:49.768 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:49.770 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:50:49.770 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:49.767+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:49.871 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:49.867+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:49.938 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:49.939 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:49.939 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:50:49.939 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:49.953 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:50:50.000 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:49.999+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:50.071 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:50.071 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:50.071 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:50:50.071 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:50:50.071 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:50.072 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:50.071+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:50.086 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:50:50.173 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:50.170+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:50.252 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:50.253 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:50.253 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:50.253 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:50:50.253 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.253 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:50:50.253 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:50:50.253 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 99.021004558000001, 2024-09-11T06:50:50.253 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010553425999999999, 2024-09-11T06:50:50.253 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:50.253 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:50.253 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:50.253 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:50.254 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:50.254 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:50.254 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:50:50.254 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.254 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:50.254 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.254 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:50.254 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:50:50.254 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.254 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.254 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.254 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:50.254 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:50:50.254 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.255 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.255 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.255 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:50.255 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230274+0000", 2024-09-11T06:50:50.255 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.533e-06 2024-09-11T06:50:50.255 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.255 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.255 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:50.255 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230285+0000", 2024-09-11T06:50:50.255 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0540000000000001e-05 2024-09-11T06:50:50.255 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.255 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.255 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:50.255 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230309+0000", 2024-09-11T06:50:50.256 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4223000000000002e-05 2024-09-11T06:50:50.256 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.256 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.256 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:50.256 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230327+0000", 2024-09-11T06:50:50.256 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:50:50.256 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.256 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.256 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:50.256 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230478+0000", 2024-09-11T06:50:50.256 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000151459 2024-09-11T06:50:50.256 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.256 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.256 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:50.257 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230490+0000", 2024-09-11T06:50:50.257 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.147e-05 2024-09-11T06:50:50.257 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.257 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.257 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:50.257 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231803+0000", 2024-09-11T06:50:50.257 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001313158 2024-09-11T06:50:50.257 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.257 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.257 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:50:50.257 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231817+0000", 2024-09-11T06:50:50.257 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4197e-05 2024-09-11T06:50:50.257 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.257 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.258 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:50.258 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.235945+0000", 2024-09-11T06:50:50.258 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0041282480000000002 2024-09-11T06:50:50.258 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.258 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.258 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:50.258 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236312+0000", 2024-09-11T06:50:50.258 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036652700000000001 2024-09-11T06:50:50.258 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.258 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.258 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:50:50.258 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236689+0000", 2024-09-11T06:50:50.258 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000377117 2024-09-11T06:50:50.258 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.259 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.259 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:50.259 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238460+0000", 2024-09-11T06:50:50.259 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001770918 2024-09-11T06:50:50.259 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.259 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.259 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:50.259 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240627+0000", 2024-09-11T06:50:50.259 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021670779999999998 2024-09-11T06:50:50.259 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.259 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.259 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:50.259 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240751+0000", 2024-09-11T06:50:50.259 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012384900000000001 2024-09-11T06:50:50.260 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.260 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.260 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:50.260 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240821+0000", 2024-09-11T06:50:50.260 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0516e-05 2024-09-11T06:50:50.260 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.260 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:50.260 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.260 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.260 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.260 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-11T06:50:50.260 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:50:50.260 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 98.018833715, 2024-09-11T06:50:50.260 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069904770000000001, 2024-09-11T06:50:50.261 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:50.261 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:50.261 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:50.261 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:50.261 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:50.261 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:50.261 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 5 2024-09-11T06:50:50.261 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.261 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:50.261 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.261 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:50.261 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:50:50.261 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.261 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.262 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.262 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:50.262 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:50:50.262 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.262 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.262 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.262 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:50.262 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232442+0000", 2024-09-11T06:50:50.262 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.044e-06 2024-09-11T06:50:50.262 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.262 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.262 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:50.262 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232460+0000", 2024-09-11T06:50:50.262 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7863000000000002e-05 2024-09-11T06:50:50.262 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.263 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.263 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:50.263 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232492+0000", 2024-09-11T06:50:50.263 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2812999999999999e-05 2024-09-11T06:50:50.263 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.263 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.263 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:50.263 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232510+0000", 2024-09-11T06:50:50.263 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7552999999999999e-05 2024-09-11T06:50:50.263 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.263 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.263 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:50.263 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232640+0000", 2024-09-11T06:50:50.263 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013029900000000001 2024-09-11T06:50:50.263 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.264 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.264 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:50.264 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232650+0000", 2024-09-11T06:50:50.264 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0023999999999999e-05 2024-09-11T06:50:50.264 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.264 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.264 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:50.264 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235641+0000", 2024-09-11T06:50:50.264 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00299089 2024-09-11T06:50:50.264 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.264 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.264 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:50.264 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235980+0000", 2024-09-11T06:50:50.264 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000338907 2024-09-11T06:50:50.265 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.265 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.265 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:50.265 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.236351+0000", 2024-09-11T06:50:50.265 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037087699999999998 2024-09-11T06:50:50.265 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.265 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.265 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:50.265 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238074+0000", 2024-09-11T06:50:50.265 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001722859 2024-09-11T06:50:50.265 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.265 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.265 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:50.265 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239160+0000", 2024-09-11T06:50:50.266 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001086074 2024-09-11T06:50:50.266 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.266 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.266 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:50.266 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239299+0000", 2024-09-11T06:50:50.266 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013933500000000001 2024-09-11T06:50:50.266 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.266 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.266 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:50.266 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239429+0000", 2024-09-11T06:50:50.266 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129939 2024-09-11T06:50:50.266 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.266 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:50.267 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.267 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.267 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.267 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-11T06:50:50.267 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:50:50.267 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 97.004745763000003, 2024-09-11T06:50:50.267 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069253250000000004, 2024-09-11T06:50:50.267 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:50.267 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:50.267 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:50.267 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:50.267 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:50.267 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:50.267 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 7 2024-09-11T06:50:50.268 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.268 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:50.268 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.268 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:50.268 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:50:50.268 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.268 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.268 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.268 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:50.268 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:50:50.268 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.268 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.268 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.268 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:50.269 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246530+0000", 2024-09-11T06:50:50.269 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9469999999999999e-06 2024-09-11T06:50:50.269 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.269 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.269 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:50.269 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246545+0000", 2024-09-11T06:50:50.269 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5973000000000001e-05 2024-09-11T06:50:50.269 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.269 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.269 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:50.269 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246578+0000", 2024-09-11T06:50:50.269 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2759999999999998e-05 2024-09-11T06:50:50.269 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.270 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.270 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:50.270 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246595+0000", 2024-09-11T06:50:50.270 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6939000000000002e-05 2024-09-11T06:50:50.270 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.270 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.270 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:50.270 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246770+0000", 2024-09-11T06:50:50.270 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000174723 2024-09-11T06:50:50.270 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.270 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.270 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:50.270 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246789+0000", 2024-09-11T06:50:50.271 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9159000000000001e-05 2024-09-11T06:50:50.271 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.271 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.271 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:50.271 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250337+0000", 2024-09-11T06:50:50.271 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035480759999999998 2024-09-11T06:50:50.271 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.271 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.271 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:50.271 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250549+0000", 2024-09-11T06:50:50.271 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021192199999999999 2024-09-11T06:50:50.271 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.271 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.271 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:50.271 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250750+0000", 2024-09-11T06:50:50.272 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020104199999999999 2024-09-11T06:50:50.272 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.272 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.272 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:50.272 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.252455+0000", 2024-09-11T06:50:50.272 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001705249 2024-09-11T06:50:50.272 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.272 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.272 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:50.272 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253305+0000", 2024-09-11T06:50:50.272 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00084941200000000002 2024-09-11T06:50:50.272 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.272 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.273 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:50.273 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253388+0000", 2024-09-11T06:50:50.273 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3642999999999999e-05 2024-09-11T06:50:50.273 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.273 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.273 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:50.273 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253452+0000", 2024-09-11T06:50:50.273 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3479999999999995e-05 2024-09-11T06:50:50.273 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.273 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:50.273 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.273 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.273 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.274 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(client.4558.0:5 3.2 e57/55 3:5cea7035:::rbd_info:head v 57'1, mlcod=0'0)", 2024-09-11T06:50:50.274 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:50:50.274 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 32.156641401000002, 2024-09-11T06:50:50.274 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011016419999999999, 2024-09-11T06:50:50.274 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:50.274 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:50.274 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:50.274 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:50.274 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.274 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:50.274 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:50:50.274 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.274 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.274 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.275 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:50.275 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:50:50.275 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.275 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.275 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.275 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:50.275 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:50:50.275 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4000000000000002e-07 2024-09-11T06:50:50.275 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.275 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.275 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:50.275 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094640+0000", 2024-09-11T06:50:50.275 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6230000000000003e-06 2024-09-11T06:50:50.275 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.276 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.276 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:50.276 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094669+0000", 2024-09-11T06:50:50.276 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8462e-05 2024-09-11T06:50:50.276 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.276 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.276 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:50.276 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094684+0000", 2024-09-11T06:50:50.276 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5536000000000002e-05 2024-09-11T06:50:50.276 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.276 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.276 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:50.276 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094798+0000", 2024-09-11T06:50:50.276 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011439599999999999 2024-09-11T06:50:50.277 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.277 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.277 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:50.277 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094819+0000", 2024-09-11T06:50:50.277 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0296e-05 2024-09-11T06:50:50.277 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.277 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.277 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:50.277 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.095687+0000", 2024-09-11T06:50:50.277 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086851799999999998 2024-09-11T06:50:50.277 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.277 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.277 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:50.277 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.095733+0000", 2024-09-11T06:50:50.278 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5271000000000001e-05 2024-09-11T06:50:50.278 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.278 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:50.278 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.278 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.278 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.278 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:160 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 4096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:50.278 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:50:50.278 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 29.217526394, 2024-09-11T06:50:50.278 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001122215, 2024-09-11T06:50:50.278 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:50.278 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:50.278 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:50.278 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:50.279 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:50.279 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:50.279 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 160 2024-09-11T06:50:50.279 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.279 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:50.279 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.279 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:50.279 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:50:50.279 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.279 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.279 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.279 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:50.279 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:50:50.279 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.280 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.280 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.280 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:50.280 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:50:50.280 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0999999999999999e-07 2024-09-11T06:50:50.280 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.280 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.280 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:50.280 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033749+0000", 2024-09-11T06:50:50.280 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8930000000000001e-06 2024-09-11T06:50:50.280 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.280 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.280 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:50.281 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033758+0000", 2024-09-11T06:50:50.281 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.242e-06 2024-09-11T06:50:50.281 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.281 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.281 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:50.281 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033764+0000", 2024-09-11T06:50:50.281 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0510000000000002e-06 2024-09-11T06:50:50.281 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.281 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.281 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:50.281 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.034207+0000", 2024-09-11T06:50:50.281 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00044320600000000003 2024-09-11T06:50:50.281 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.281 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.282 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:50.282 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.034280+0000", 2024-09-11T06:50:50.282 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3094000000000002e-05 2024-09-11T06:50:50.282 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.282 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.282 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:50.282 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.034868+0000", 2024-09-11T06:50:50.282 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00058821899999999996 2024-09-11T06:50:50.282 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.282 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:50.282 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.282 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.282 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.282 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:162 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:50.283 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.039519+0000", 2024-09-11T06:50:50.283 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 29.211753276, 2024-09-11T06:50:50.283 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001298485, 2024-09-11T06:50:50.283 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:50.283 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:50.283 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:50.283 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:50.283 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:50.283 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:50.283 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 162 2024-09-11T06:50:50.283 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.283 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:50.283 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.283 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:50.283 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039519+0000", 2024-09-11T06:50:50.284 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.284 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.284 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.284 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:50.284 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039519+0000", 2024-09-11T06:50:50.284 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.284 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.284 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.284 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:50.284 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039520+0000", 2024-09-11T06:50:50.284 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9299999999999998e-07 2024-09-11T06:50:50.284 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.284 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.284 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:50.285 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039523+0000", 2024-09-11T06:50:50.285 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.247e-06 2024-09-11T06:50:50.285 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.285 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.285 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:50.285 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039529+0000", 2024-09-11T06:50:50.285 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8300000000000001e-06 2024-09-11T06:50:50.285 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.285 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.285 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:50.285 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039533+0000", 2024-09-11T06:50:50.285 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.2379999999999999e-06 2024-09-11T06:50:50.285 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.285 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.286 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:50.286 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.040236+0000", 2024-09-11T06:50:50.286 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00070326500000000001 2024-09-11T06:50:50.286 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.286 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.286 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:50.286 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.040299+0000", 2024-09-11T06:50:50.286 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3077999999999995e-05 2024-09-11T06:50:50.286 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.286 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.286 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:50.286 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.040818+0000", 2024-09-11T06:50:50.286 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00051833399999999996 2024-09-11T06:50:50.286 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.287 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:50.287 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.287 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.287 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.287 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:164 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 528384~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:50.287 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.041536+0000", 2024-09-11T06:50:50.287 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 29.209736553999999, 2024-09-11T06:50:50.287 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00119962, 2024-09-11T06:50:50.287 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:50.287 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:50.287 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:50.287 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:50.287 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:50.287 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:50.288 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 164 2024-09-11T06:50:50.288 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.288 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:50.288 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.288 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:50.288 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041536+0000", 2024-09-11T06:50:50.288 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.288 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.288 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.288 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:50.288 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041536+0000", 2024-09-11T06:50:50.288 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.288 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.289 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.289 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:50.289 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041537+0000", 2024-09-11T06:50:50.289 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3e-06 2024-09-11T06:50:50.289 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.289 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.289 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:50.289 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041543+0000", 2024-09-11T06:50:50.289 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2360000000000001e-06 2024-09-11T06:50:50.289 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.289 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.289 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:50.289 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041558+0000", 2024-09-11T06:50:50.289 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4909000000000001e-05 2024-09-11T06:50:50.289 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.290 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.290 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:50.290 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041569+0000", 2024-09-11T06:50:50.290 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0946e-05 2024-09-11T06:50:50.290 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.290 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.290 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:50.290 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.042138+0000", 2024-09-11T06:50:50.290 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000568996 2024-09-11T06:50:50.290 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.290 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.290 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:50.290 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.042190+0000", 2024-09-11T06:50:50.290 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1609e-05 2024-09-11T06:50:50.291 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.291 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.291 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:50.291 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.042735+0000", 2024-09-11T06:50:50.291 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00054562399999999998 2024-09-11T06:50:50.291 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.291 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:50.291 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.291 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.291 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.291 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:166 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 790528~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:50.291 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.043564+0000", 2024-09-11T06:50:50.291 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 29.207708156999999, 2024-09-11T06:50:50.291 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011042000000000001, 2024-09-11T06:50:50.291 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:50.292 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:50.292 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:50.292 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:50.292 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:50.292 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:50.292 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 166 2024-09-11T06:50:50.292 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.292 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:50.292 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.292 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:50.292 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043564+0000", 2024-09-11T06:50:50.292 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.292 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.292 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.293 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:50.293 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043564+0000", 2024-09-11T06:50:50.293 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.293 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.293 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.293 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:50.293 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043565+0000", 2024-09-11T06:50:50.293 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0449999999999999e-06 2024-09-11T06:50:50.293 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.293 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.294 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:50.294 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043571+0000", 2024-09-11T06:50:50.294 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.6849999999999998e-06 2024-09-11T06:50:50.294 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.294 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.294 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:50.294 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043586+0000", 2024-09-11T06:50:50.294 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4902000000000001e-05 2024-09-11T06:50:50.294 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.294 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.294 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:50.294 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043593+0000", 2024-09-11T06:50:50.294 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9240000000000001e-06 2024-09-11T06:50:50.294 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.295 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.295 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:50.295 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.044073+0000", 2024-09-11T06:50:50.295 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000479813 2024-09-11T06:50:50.295 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.295 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.295 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:50.295 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.044123+0000", 2024-09-11T06:50:50.295 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0908000000000001e-05 2024-09-11T06:50:50.295 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.295 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.295 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:50.295 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.044668+0000", 2024-09-11T06:50:50.296 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000544923 2024-09-11T06:50:50.296 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.296 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:50.296 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.296 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.296 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.296 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:178 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 2363392~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:50.296 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.061599+0000", 2024-09-11T06:50:50.296 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 29.189673442, 2024-09-11T06:50:50.296 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012194529999999999, 2024-09-11T06:50:50.296 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:50.296 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:50.296 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:50.297 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:50.297 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:50.297 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:50.297 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 178 2024-09-11T06:50:50.297 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.297 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:50.297 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.297 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:50.297 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061599+0000", 2024-09-11T06:50:50.297 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.297 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.297 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.297 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:50.297 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061599+0000", 2024-09-11T06:50:50.298 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.298 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.298 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.298 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:50.298 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061600+0000", 2024-09-11T06:50:50.298 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3600000000000003e-07 2024-09-11T06:50:50.298 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.298 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.298 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:50.298 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061603+0000", 2024-09-11T06:50:50.298 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.6550000000000002e-06 2024-09-11T06:50:50.298 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.298 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.298 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:50.298 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061618+0000", 2024-09-11T06:50:50.299 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4377e-05 2024-09-11T06:50:50.299 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.299 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.299 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:50.299 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061626+0000", 2024-09-11T06:50:50.299 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.0679999999999997e-06 2024-09-11T06:50:50.299 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.299 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.299 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:50.299 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.062098+0000", 2024-09-11T06:50:50.299 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00047251299999999998 2024-09-11T06:50:50.299 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.299 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.299 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:50.300 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.062253+0000", 2024-09-11T06:50:50.300 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015533100000000001 2024-09-11T06:50:50.300 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.300 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.300 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:50.300 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.062818+0000", 2024-09-11T06:50:50.300 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00056487300000000002 2024-09-11T06:50:50.300 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.300 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:50.300 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.300 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.300 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.300 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:184 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 3149824~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:50.301 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:50:50.301 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 29.185063739, 2024-09-11T06:50:50.301 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010685969999999999, 2024-09-11T06:50:50.301 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:50.301 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:50.301 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:50.301 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:50.301 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:50.301 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:50.301 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 184 2024-09-11T06:50:50.301 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.301 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:50.301 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.301 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:50.302 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:50:50.302 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.302 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.302 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.302 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:50.302 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:50:50.302 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.302 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.302 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.302 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:50.302 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:50:50.302 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.6700000000000003e-07 2024-09-11T06:50:50.302 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.303 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.303 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:50.303 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066212+0000", 2024-09-11T06:50:50.303 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.1669999999999999e-06 2024-09-11T06:50:50.303 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.303 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.303 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:50.303 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066219+0000", 2024-09-11T06:50:50.303 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5699999999999998e-06 2024-09-11T06:50:50.303 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.303 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.303 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:50.303 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066223+0000", 2024-09-11T06:50:50.303 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3540000000000002e-06 2024-09-11T06:50:50.303 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.304 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.304 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:50.304 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066713+0000", 2024-09-11T06:50:50.304 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00048985399999999996 2024-09-11T06:50:50.304 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.304 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.304 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:50.304 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066813+0000", 2024-09-11T06:50:50.304 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00010003 2024-09-11T06:50:50.304 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.304 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.304 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:50.304 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.067277+0000", 2024-09-11T06:50:50.304 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000463955 2024-09-11T06:50:50.305 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.305 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:50.305 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.305 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.305 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.305 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:186 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 3411968~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:50.305 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.071198+0000", 2024-09-11T06:50:50.305 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 29.180074394999998, 2024-09-11T06:50:50.305 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012515779999999999, 2024-09-11T06:50:50.305 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:50.305 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:50.305 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:50.305 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:50.305 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:50.306 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:50.306 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 186 2024-09-11T06:50:50.306 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.306 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:50.306 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.306 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:50.306 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071198+0000", 2024-09-11T06:50:50.306 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.306 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.306 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.306 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:50.307 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071198+0000", 2024-09-11T06:50:50.308 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.308 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.308 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.308 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:50.308 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071199+0000", 2024-09-11T06:50:50.308 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0299999999999999e-07 2024-09-11T06:50:50.308 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.308 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.308 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:50.308 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071201+0000", 2024-09-11T06:50:50.308 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8540000000000001e-06 2024-09-11T06:50:50.308 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.308 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.308 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:50.309 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071209+0000", 2024-09-11T06:50:50.309 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3629999999999996e-06 2024-09-11T06:50:50.309 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.309 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.309 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:50.309 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071214+0000", 2024-09-11T06:50:50.309 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2109999999999997e-06 2024-09-11T06:50:50.309 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.309 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.309 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:50.309 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071915+0000", 2024-09-11T06:50:50.309 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00070101799999999998 2024-09-11T06:50:50.309 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.309 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.310 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:50.310 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.072011+0000", 2024-09-11T06:50:50.310 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.5755999999999996e-05 2024-09-11T06:50:50.310 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.310 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.310 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:50.310 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.072450+0000", 2024-09-11T06:50:50.310 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000438773 2024-09-11T06:50:50.310 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.310 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:50.310 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.310 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.310 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.310 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:190 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 3936256~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:50.311 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.074875+0000", 2024-09-11T06:50:50.311 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 29.176397479999999, 2024-09-11T06:50:50.311 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0013119620000000001, 2024-09-11T06:50:50.311 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:50.311 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:50.311 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:50.311 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:50.311 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:50.311 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:50.311 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 190 2024-09-11T06:50:50.311 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.311 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:50.312 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.312 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:50.312 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074875+0000", 2024-09-11T06:50:50.312 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.312 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.312 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.312 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:50.312 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074875+0000", 2024-09-11T06:50:50.312 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.312 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.312 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.312 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:50.312 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074876+0000", 2024-09-11T06:50:50.313 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7400000000000003e-07 2024-09-11T06:50:50.313 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.313 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.313 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:50.313 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074887+0000", 2024-09-11T06:50:50.313 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1817999999999999e-05 2024-09-11T06:50:50.313 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.313 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.313 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:50.313 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074896+0000", 2024-09-11T06:50:50.313 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9120000000000004e-06 2024-09-11T06:50:50.313 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.313 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.313 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:50.314 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074900+0000", 2024-09-11T06:50:50.314 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.2220000000000004e-06 2024-09-11T06:50:50.314 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.314 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.314 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:50.314 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.075198+0000", 2024-09-11T06:50:50.314 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00029750399999999998 2024-09-11T06:50:50.314 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.314 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.314 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:50.314 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.075430+0000", 2024-09-11T06:50:50.314 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023200400000000001 2024-09-11T06:50:50.314 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.314 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.315 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:50.315 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.076187+0000", 2024-09-11T06:50:50.315 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00075682799999999997 2024-09-11T06:50:50.315 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.315 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:50.315 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.315 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.315 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.315 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:359 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 135168~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:50.315 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.298832+0000", 2024-09-11T06:50:50.315 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 28.952439846000001, 2024-09-11T06:50:50.315 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001078575, 2024-09-11T06:50:50.315 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:50.315 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:50.316 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:50.316 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:50.316 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:50.316 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:50.316 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 359 2024-09-11T06:50:50.316 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.316 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:50.316 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.316 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:50.316 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298832+0000", 2024-09-11T06:50:50.316 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.316 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.316 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.317 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:50.317 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298832+0000", 2024-09-11T06:50:50.317 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.317 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.317 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.317 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:50.317 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298834+0000", 2024-09-11T06:50:50.317 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.646e-06 2024-09-11T06:50:50.317 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.317 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.317 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:50.317 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298845+0000", 2024-09-11T06:50:50.317 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.081e-05 2024-09-11T06:50:50.317 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.318 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.318 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:50.318 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298872+0000", 2024-09-11T06:50:50.318 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7382e-05 2024-09-11T06:50:50.318 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.318 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.318 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:50.318 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298884+0000", 2024-09-11T06:50:50.318 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1956e-05 2024-09-11T06:50:50.318 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.318 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.318 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:50.318 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298959+0000", 2024-09-11T06:50:50.318 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5214000000000002e-05 2024-09-11T06:50:50.319 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.319 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.319 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:50.319 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.299090+0000", 2024-09-11T06:50:50.319 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000130247 2024-09-11T06:50:50.319 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.319 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.319 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:50.319 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.299911+0000", 2024-09-11T06:50:50.319 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00082131999999999995 2024-09-11T06:50:50.319 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.319 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:50.319 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.319 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.319 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.320 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:360 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:50.320 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.298914+0000", 2024-09-11T06:50:50.320 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 28.952358176000001, 2024-09-11T06:50:50.320 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0023571690000000001, 2024-09-11T06:50:50.320 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:50.320 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:50.320 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:50.320 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:50.320 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:50.320 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:50.320 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 360 2024-09-11T06:50:50.320 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.320 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:50.321 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.321 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:50.321 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298914+0000", 2024-09-11T06:50:50.321 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.321 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.321 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.321 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:50.321 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298914+0000", 2024-09-11T06:50:50.321 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.321 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.321 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.321 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:50.321 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298915+0000", 2024-09-11T06:50:50.321 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9300000000000006e-07 2024-09-11T06:50:50.322 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.322 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.322 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:50.322 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298921+0000", 2024-09-11T06:50:50.322 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.6030000000000004e-06 2024-09-11T06:50:50.322 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.322 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.322 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:50.322 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298932+0000", 2024-09-11T06:50:50.322 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.082e-05 2024-09-11T06:50:50.322 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.322 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.322 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:50.322 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298939+0000", 2024-09-11T06:50:50.323 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7919999999999993e-06 2024-09-11T06:50:50.323 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.323 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.323 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:50.323 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.300011+0000", 2024-09-11T06:50:50.323 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001071179 2024-09-11T06:50:50.323 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.323 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.323 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:50.323 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.300245+0000", 2024-09-11T06:50:50.323 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000234434 2024-09-11T06:50:50.323 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.323 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.323 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:50.323 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.301271+0000", 2024-09-11T06:50:50.324 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010263480000000001 2024-09-11T06:50:50.324 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.324 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:50.324 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.324 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.324 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.324 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:362 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 528384~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:50.324 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.302376+0000", 2024-09-11T06:50:50.324 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 28.948895853, 2024-09-11T06:50:50.324 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014132909999999999, 2024-09-11T06:50:50.324 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:50.324 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:50.324 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:50.325 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:50.325 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:50.325 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:50.325 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 362 2024-09-11T06:50:50.325 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.325 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:50.325 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.325 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:50.325 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302376+0000", 2024-09-11T06:50:50.325 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.325 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.325 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.325 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:50.326 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302376+0000", 2024-09-11T06:50:50.326 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.326 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.326 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.326 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:50.326 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302378+0000", 2024-09-11T06:50:50.326 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1629999999999999e-06 2024-09-11T06:50:50.326 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.326 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.326 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:50.326 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302383+0000", 2024-09-11T06:50:50.326 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.463e-06 2024-09-11T06:50:50.326 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.326 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.327 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:50.327 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302401+0000", 2024-09-11T06:50:50.327 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8219e-05 2024-09-11T06:50:50.327 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.327 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.327 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:50.327 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302411+0000", 2024-09-11T06:50:50.327 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9000000000000001e-06 2024-09-11T06:50:50.327 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.327 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.327 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:50.327 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302524+0000", 2024-09-11T06:50:50.327 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000113199 2024-09-11T06:50:50.327 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.328 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.328 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:50.328 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302745+0000", 2024-09-11T06:50:50.328 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022057399999999999 2024-09-11T06:50:50.328 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.328 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.328 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:50.328 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.303790+0000", 2024-09-11T06:50:50.328 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010447729999999999 2024-09-11T06:50:50.328 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.328 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:50.328 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.328 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.329 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.329 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:363 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 659456~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:50.329 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.302938+0000", 2024-09-11T06:50:50.329 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 28.948334259999999, 2024-09-11T06:50:50.329 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00185012, 2024-09-11T06:50:50.329 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:50.329 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:50.329 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:50.329 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:50.329 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:50.329 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:50.329 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 363 2024-09-11T06:50:50.329 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.329 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:50.330 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.330 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:50.330 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302938+0000", 2024-09-11T06:50:50.330 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.330 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.330 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.330 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:50.330 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302938+0000", 2024-09-11T06:50:50.330 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.330 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.330 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.330 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:50.330 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302939+0000", 2024-09-11T06:50:50.330 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2300000000000001e-06 2024-09-11T06:50:50.331 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.331 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.331 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:50.331 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302944+0000", 2024-09-11T06:50:50.331 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.493e-06 2024-09-11T06:50:50.331 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.331 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.331 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:50.331 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302960+0000", 2024-09-11T06:50:50.331 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.632e-05 2024-09-11T06:50:50.331 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.331 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.331 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:50.331 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302972+0000", 2024-09-11T06:50:50.332 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1885000000000001e-05 2024-09-11T06:50:50.332 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.332 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.332 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:50.332 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.303835+0000", 2024-09-11T06:50:50.332 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000863141 2024-09-11T06:50:50.332 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.332 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.332 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:50.332 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.303965+0000", 2024-09-11T06:50:50.332 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013032199999999999 2024-09-11T06:50:50.332 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.332 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.332 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:50.333 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.304788+0000", 2024-09-11T06:50:50.333 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00082272900000000002 2024-09-11T06:50:50.333 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.333 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:50.333 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.333 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.333 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.333 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:367 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 1183744~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:50.333 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.312392+0000", 2024-09-11T06:50:50.333 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 28.938880619999999, 2024-09-11T06:50:50.333 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011343460000000001, 2024-09-11T06:50:50.333 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:50.333 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:50.333 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:50.334 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:50.334 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:50.334 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:50.334 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 367 2024-09-11T06:50:50.334 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.334 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:50.334 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.334 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:50.334 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312392+0000", 2024-09-11T06:50:50.334 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.334 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.334 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.334 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:50.334 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312392+0000", 2024-09-11T06:50:50.334 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.335 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.335 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.335 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:50.335 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312393+0000", 2024-09-11T06:50:50.335 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6199999999999999e-06 2024-09-11T06:50:50.335 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.335 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.335 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:50.335 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312400+0000", 2024-09-11T06:50:50.335 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0700000000000001e-06 2024-09-11T06:50:50.335 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.335 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.335 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:50.335 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312420+0000", 2024-09-11T06:50:50.336 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9375999999999999e-05 2024-09-11T06:50:50.336 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.336 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.336 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:50.336 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312430+0000", 2024-09-11T06:50:50.336 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0209e-05 2024-09-11T06:50:50.336 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.336 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.336 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:50.336 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312549+0000", 2024-09-11T06:50:50.336 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000118779 2024-09-11T06:50:50.336 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.336 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.336 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:50.337 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312767+0000", 2024-09-11T06:50:50.337 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021786400000000001 2024-09-11T06:50:50.337 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.337 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.337 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:50.337 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313526+0000", 2024-09-11T06:50:50.337 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00075942800000000003 2024-09-11T06:50:50.337 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.337 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:50.337 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.337 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.337 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.337 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:368 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 1314816~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:50.337 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.312454+0000", 2024-09-11T06:50:50.337 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 28.938818095999999, 2024-09-11T06:50:50.338 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014704550000000001, 2024-09-11T06:50:50.338 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:50.338 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:50.338 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:50.338 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:50.338 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:50.338 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:50.338 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 368 2024-09-11T06:50:50.338 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.338 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:50.338 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.338 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:50.338 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312454+0000", 2024-09-11T06:50:50.338 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.339 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.339 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.339 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:50.339 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312454+0000", 2024-09-11T06:50:50.339 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.339 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.339 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.339 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:50.339 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312455+0000", 2024-09-11T06:50:50.339 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4399999999999999e-07 2024-09-11T06:50:50.339 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.339 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.339 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:50.339 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312474+0000", 2024-09-11T06:50:50.340 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8899e-05 2024-09-11T06:50:50.340 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.340 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.340 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:50.340 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312485+0000", 2024-09-11T06:50:50.340 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0793e-05 2024-09-11T06:50:50.340 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.340 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.340 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:50.340 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312492+0000", 2024-09-11T06:50:50.340 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.593e-06 2024-09-11T06:50:50.340 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.340 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.340 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:50.340 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313576+0000", 2024-09-11T06:50:50.341 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010837970000000001 2024-09-11T06:50:50.341 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.341 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.341 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:50.341 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313620+0000", 2024-09-11T06:50:50.341 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3640000000000002e-05 2024-09-11T06:50:50.341 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.341 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.341 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:50.341 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313925+0000", 2024-09-11T06:50:50.341 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00030498899999999998 2024-09-11T06:50:50.341 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.341 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:50.341 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.342 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.342 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.342 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:370 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 1576960~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:50.342 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.314817+0000", 2024-09-11T06:50:50.342 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 28.936455386999999, 2024-09-11T06:50:50.342 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011606190000000001, 2024-09-11T06:50:50.342 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:50.342 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:50.342 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:50.342 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:50.342 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:50.342 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:50.342 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 370 2024-09-11T06:50:50.342 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.343 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:50.343 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.343 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:50.343 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314817+0000", 2024-09-11T06:50:50.343 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.343 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.343 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.343 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:50.343 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314817+0000", 2024-09-11T06:50:50.343 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.343 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.343 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.343 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:50.343 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314818+0000", 2024-09-11T06:50:50.344 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.463e-06 2024-09-11T06:50:50.344 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.344 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.344 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:50.344 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314826+0000", 2024-09-11T06:50:50.344 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3860000000000001e-06 2024-09-11T06:50:50.344 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.344 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.344 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:50.344 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314842+0000", 2024-09-11T06:50:50.344 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6220000000000001e-05 2024-09-11T06:50:50.344 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.344 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.344 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:50.345 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314854+0000", 2024-09-11T06:50:50.345 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1562e-05 2024-09-11T06:50:50.345 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.345 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.345 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:50.345 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.315212+0000", 2024-09-11T06:50:50.345 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000358012 2024-09-11T06:50:50.345 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.345 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.345 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:50.345 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.315433+0000", 2024-09-11T06:50:50.345 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022103800000000001 2024-09-11T06:50:50.345 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.346 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.346 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:50.346 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.315978+0000", 2024-09-11T06:50:50.346 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00054493800000000002 2024-09-11T06:50:50.346 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.346 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:50.346 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.346 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.346 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.346 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:57 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:50:50.346 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.568529+0000", 2024-09-11T06:50:50.346 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 1.682742843, 2024-09-11T06:50:50.346 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004228229, 2024-09-11T06:50:50.346 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:50.346 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:50.347 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:50.347 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:50.347 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:50:50.347 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:50:50.347 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 57 2024-09-11T06:50:50.347 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.347 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:50.347 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.347 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:50.347 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568529+0000", 2024-09-11T06:50:50.347 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.347 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.347 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.347 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:50.348 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568529+0000", 2024-09-11T06:50:50.348 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:50.348 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.348 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.348 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:50.348 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568532+0000", 2024-09-11T06:50:50.348 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0000000000000001e-06 2024-09-11T06:50:50.348 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.348 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.348 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:50.348 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568547+0000", 2024-09-11T06:50:50.348 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4219999999999999e-05 2024-09-11T06:50:50.348 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.348 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.349 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:50.349 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568580+0000", 2024-09-11T06:50:50.349 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.3328999999999999e-05 2024-09-11T06:50:50.349 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.349 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.349 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:50.349 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568596+0000", 2024-09-11T06:50:50.349 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5990000000000001e-05 2024-09-11T06:50:50.349 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.349 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.349 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:50.349 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568728+0000", 2024-09-11T06:50:50.349 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013175400000000001 2024-09-11T06:50:50.349 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.350 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.350 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:50.350 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568738+0000", 2024-09-11T06:50:50.350 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0196e-05 2024-09-11T06:50:50.350 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.350 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.350 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:50.350 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.572464+0000", 2024-09-11T06:50:50.350 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0037264160000000002 2024-09-11T06:50:50.350 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.350 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.350 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:50.350 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.572584+0000", 2024-09-11T06:50:50.350 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000119438 2024-09-11T06:50:50.351 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:50.351 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:50.351 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:50.351 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.572758+0000", 2024-09-11T06:50:50.351 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017388599999999999 2024-09-11T06:50:50.351 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.351 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:50.351 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.351 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:50.351 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:50.351 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:50.351 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:50:50.353 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:50:50.354 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:50.350+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:50.454 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:50.451+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:50.469 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:50.469 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:50.469 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:50:50.470 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:50.483 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:50:50.501 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:50:50.502 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:50:50.518 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:50:50.518 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:50:50.534 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:50:50.534 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:50:50.545 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is unset 2024-09-11T06:50:50.555 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:50.551+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:50.557 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:50:50.557 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:50:50.577 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:50:50.578 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:50:50.598 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:50:50.599 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:50:50.604 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:50.605 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:50.620 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:50:50.662 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:50.658+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:50.757 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:50.755+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:50.776 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:50.777 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:50.791 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:50:50.857 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:50.854+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:50.913 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:50.913 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:50.927 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:50:50.958 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:50.955+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:51.059 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:51.054+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:51.093 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:51.093 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:51.109 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:50:51.160 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:51.159+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:51.261 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:51.259+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:51.274 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:51.275 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:51.289 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:50:51.320 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 1, 3, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T06:50:51.321 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T06:50:51.321 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-11T06:50:51.321 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T06:50:51.321 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T06:50:51.321 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-11T06:50:51.361 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:51.358+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:51.408 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:51.408 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:51.421 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:50:51.452 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:51.452 INFO:teuthology.orchestra.run.smithi089.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-11T06:50:51.452 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:51.465 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:51.462+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:51.564 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:51.558+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:51.594 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:51.595 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:51.609 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:50:51.701 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:51.698+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:51.766 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:51.762+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:51.789 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:51.789 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:51.803 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:50:51.867 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:51.862+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:51.936 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:51.936 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:51.950 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:50:51.996 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:51.995+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:52.068 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:52.067+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:52.073 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:52.073 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:52.088 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:50:52.169 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:52.166+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:52.254 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:52.254 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:52.268 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:50:52.269 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:52.267+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:52.371 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:52.367+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:52.394 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:52.394 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:52.408 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:50:52.496 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:52.495+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:52.570 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T06:50:52.572 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:52.566+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:52.574 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:52.574 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:52.588 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:50:52.616 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:50:52.672 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:52.666+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:52.757 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:52.757 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:52.765 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:50:52.766 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:50:52.772 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:50:52.774 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:52.770+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:52.782 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:50:52.782 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:50:52.798 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:50:52.799 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:50:52.817 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:50:52.817 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:50:52.835 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:50:52.835 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:50:52.853 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:50:52.854 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:50:52.891 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:52.892 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:52.892 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:50:52.892 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:52.906 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:50:52.913 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:52.910+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:52.977 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:52.975+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:53.030 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:53.030 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:53.030 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:50:53.030 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:50:53.030 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:53.044 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:50:53.088 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:53.087+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:53.179 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:53.174+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:53.212 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:53.212 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:53.212 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:53.212 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:50:53.213 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.213 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:50:53.213 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:50:53.213 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 101.979836986, 2024-09-11T06:50:53.213 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010553425999999999, 2024-09-11T06:50:53.213 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:53.213 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:53.213 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:53.213 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:53.213 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:53.213 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:53.213 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:50:53.213 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.214 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:53.214 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.214 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:53.214 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:50:53.214 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.214 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.214 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.214 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:53.214 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:50:53.214 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.214 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.214 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.214 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:53.214 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230274+0000", 2024-09-11T06:50:53.215 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.533e-06 2024-09-11T06:50:53.215 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.215 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.215 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:53.215 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230285+0000", 2024-09-11T06:50:53.215 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0540000000000001e-05 2024-09-11T06:50:53.215 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.215 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.215 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:53.215 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230309+0000", 2024-09-11T06:50:53.215 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4223000000000002e-05 2024-09-11T06:50:53.215 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.215 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.215 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.216 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230327+0000", 2024-09-11T06:50:53.216 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:50:53.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.216 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.216 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:53.216 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230478+0000", 2024-09-11T06:50:53.216 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000151459 2024-09-11T06:50:53.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.216 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.216 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:53.216 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230490+0000", 2024-09-11T06:50:53.216 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.147e-05 2024-09-11T06:50:53.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.216 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.217 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:53.217 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231803+0000", 2024-09-11T06:50:53.217 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001313158 2024-09-11T06:50:53.217 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.217 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.217 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:50:53.217 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231817+0000", 2024-09-11T06:50:53.217 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4197e-05 2024-09-11T06:50:53.217 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.217 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.217 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:53.217 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.235945+0000", 2024-09-11T06:50:53.217 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0041282480000000002 2024-09-11T06:50:53.217 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.218 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.218 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:53.218 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236312+0000", 2024-09-11T06:50:53.218 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036652700000000001 2024-09-11T06:50:53.218 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.218 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.218 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:50:53.218 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236689+0000", 2024-09-11T06:50:53.218 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000377117 2024-09-11T06:50:53.218 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.218 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.218 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:53.219 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238460+0000", 2024-09-11T06:50:53.219 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001770918 2024-09-11T06:50:53.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.219 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.219 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:53.219 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240627+0000", 2024-09-11T06:50:53.219 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021670779999999998 2024-09-11T06:50:53.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.219 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.219 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:53.219 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240751+0000", 2024-09-11T06:50:53.219 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012384900000000001 2024-09-11T06:50:53.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.220 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.220 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:53.220 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240821+0000", 2024-09-11T06:50:53.220 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0516e-05 2024-09-11T06:50:53.220 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.220 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:53.220 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.220 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.220 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.220 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-11T06:50:53.221 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:50:53.221 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 100.97766614299999, 2024-09-11T06:50:53.221 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069904770000000001, 2024-09-11T06:50:53.221 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:53.221 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:53.221 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:53.221 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:53.221 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:53.221 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:53.221 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 5 2024-09-11T06:50:53.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.222 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:53.222 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.222 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:53.222 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:50:53.222 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.223 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:53.223 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:50:53.223 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.224 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:53.224 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232442+0000", 2024-09-11T06:50:53.224 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.044e-06 2024-09-11T06:50:53.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.224 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.224 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:53.224 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232460+0000", 2024-09-11T06:50:53.224 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7863000000000002e-05 2024-09-11T06:50:53.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.224 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.224 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:53.224 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232492+0000", 2024-09-11T06:50:53.225 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2812999999999999e-05 2024-09-11T06:50:53.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.225 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.225 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232510+0000", 2024-09-11T06:50:53.225 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7552999999999999e-05 2024-09-11T06:50:53.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.225 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:53.225 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232640+0000", 2024-09-11T06:50:53.225 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013029900000000001 2024-09-11T06:50:53.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.225 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:53.226 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232650+0000", 2024-09-11T06:50:53.226 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0023999999999999e-05 2024-09-11T06:50:53.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.226 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.226 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:53.226 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235641+0000", 2024-09-11T06:50:53.226 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00299089 2024-09-11T06:50:53.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.226 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.226 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:53.226 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235980+0000", 2024-09-11T06:50:53.226 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000338907 2024-09-11T06:50:53.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.227 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.227 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:53.227 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.236351+0000", 2024-09-11T06:50:53.227 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037087699999999998 2024-09-11T06:50:53.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.227 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.227 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:53.227 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238074+0000", 2024-09-11T06:50:53.227 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001722859 2024-09-11T06:50:53.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.227 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.227 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:53.228 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239160+0000", 2024-09-11T06:50:53.228 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001086074 2024-09-11T06:50:53.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.228 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:53.228 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239299+0000", 2024-09-11T06:50:53.228 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013933500000000001 2024-09-11T06:50:53.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.228 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:53.228 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239429+0000", 2024-09-11T06:50:53.228 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129939 2024-09-11T06:50:53.228 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.229 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:53.229 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.229 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.229 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-11T06:50:53.229 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:50:53.229 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 99.963578190999996, 2024-09-11T06:50:53.229 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069253250000000004, 2024-09-11T06:50:53.229 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:53.229 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:53.229 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:53.230 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:53.230 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:53.230 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:53.230 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 7 2024-09-11T06:50:53.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.230 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:53.230 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.230 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:53.230 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:50:53.230 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.230 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.230 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:53.230 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:50:53.231 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.231 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.231 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.231 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:53.231 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246530+0000", 2024-09-11T06:50:53.231 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9469999999999999e-06 2024-09-11T06:50:53.231 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.231 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.231 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:53.231 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246545+0000", 2024-09-11T06:50:53.231 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5973000000000001e-05 2024-09-11T06:50:53.231 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.231 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.232 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:53.232 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246578+0000", 2024-09-11T06:50:53.232 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2759999999999998e-05 2024-09-11T06:50:53.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.232 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.232 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.232 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246595+0000", 2024-09-11T06:50:53.232 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6939000000000002e-05 2024-09-11T06:50:53.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.232 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.232 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:53.232 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246770+0000", 2024-09-11T06:50:53.232 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000174723 2024-09-11T06:50:53.233 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.233 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:53.233 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246789+0000", 2024-09-11T06:50:53.233 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9159000000000001e-05 2024-09-11T06:50:53.233 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.233 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:53.233 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250337+0000", 2024-09-11T06:50:53.233 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035480759999999998 2024-09-11T06:50:53.233 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.233 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:53.233 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250549+0000", 2024-09-11T06:50:53.234 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021192199999999999 2024-09-11T06:50:53.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.234 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.234 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:53.234 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250750+0000", 2024-09-11T06:50:53.234 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020104199999999999 2024-09-11T06:50:53.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.234 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.234 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:53.235 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.252455+0000", 2024-09-11T06:50:53.235 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001705249 2024-09-11T06:50:53.235 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.235 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.235 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:53.235 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253305+0000", 2024-09-11T06:50:53.235 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00084941200000000002 2024-09-11T06:50:53.235 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.235 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.235 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:53.235 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253388+0000", 2024-09-11T06:50:53.235 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3642999999999999e-05 2024-09-11T06:50:53.236 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.236 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.236 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:53.236 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253452+0000", 2024-09-11T06:50:53.236 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3479999999999995e-05 2024-09-11T06:50:53.236 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.236 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:53.236 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.236 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.236 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.236 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(client.4558.0:5 3.2 e57/55 3:5cea7035:::rbd_info:head v 57'1, mlcod=0'0)", 2024-09-11T06:50:53.236 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:50:53.236 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 35.115473829000003, 2024-09-11T06:50:53.236 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011016419999999999, 2024-09-11T06:50:53.237 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:53.237 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:53.237 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:53.237 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:53.237 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.237 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:53.237 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:50:53.237 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.237 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.237 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.237 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:53.237 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:50:53.237 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.237 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.238 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.238 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:53.238 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:50:53.238 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4000000000000002e-07 2024-09-11T06:50:53.238 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.238 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.238 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:53.238 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094640+0000", 2024-09-11T06:50:53.238 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6230000000000003e-06 2024-09-11T06:50:53.238 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.238 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.238 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:53.238 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094669+0000", 2024-09-11T06:50:53.238 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8462e-05 2024-09-11T06:50:53.238 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.239 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.239 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.239 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094684+0000", 2024-09-11T06:50:53.239 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5536000000000002e-05 2024-09-11T06:50:53.239 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.239 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.239 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:53.239 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094798+0000", 2024-09-11T06:50:53.239 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011439599999999999 2024-09-11T06:50:53.239 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.239 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.239 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:53.239 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094819+0000", 2024-09-11T06:50:53.239 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0296e-05 2024-09-11T06:50:53.239 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.239 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.240 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:53.240 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.095687+0000", 2024-09-11T06:50:53.240 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086851799999999998 2024-09-11T06:50:53.240 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.240 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.240 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:53.240 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.095733+0000", 2024-09-11T06:50:53.240 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5271000000000001e-05 2024-09-11T06:50:53.240 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.240 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:53.240 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.240 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.241 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.241 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:160 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 4096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:53.241 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:50:53.241 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 32.176358821999997, 2024-09-11T06:50:53.241 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001122215, 2024-09-11T06:50:53.241 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:53.241 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:53.241 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:53.241 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:53.241 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:53.241 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:53.241 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 160 2024-09-11T06:50:53.241 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.241 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:53.242 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.242 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:53.242 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:50:53.242 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.242 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.242 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.242 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:53.242 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:50:53.242 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.242 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.242 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.242 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:53.242 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:50:53.242 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0999999999999999e-07 2024-09-11T06:50:53.243 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.243 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.243 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:53.243 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033749+0000", 2024-09-11T06:50:53.243 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8930000000000001e-06 2024-09-11T06:50:53.243 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.243 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.243 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:53.243 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033758+0000", 2024-09-11T06:50:53.243 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.242e-06 2024-09-11T06:50:53.243 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.243 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.243 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.244 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033764+0000", 2024-09-11T06:50:53.244 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0510000000000002e-06 2024-09-11T06:50:53.244 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.244 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.244 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:53.244 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.034207+0000", 2024-09-11T06:50:53.244 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00044320600000000003 2024-09-11T06:50:53.244 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.244 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.245 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:53.245 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.034280+0000", 2024-09-11T06:50:53.245 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3094000000000002e-05 2024-09-11T06:50:53.245 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.245 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.245 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:53.245 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.034868+0000", 2024-09-11T06:50:53.245 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00058821899999999996 2024-09-11T06:50:53.245 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.245 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:53.245 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.245 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.245 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.245 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:162 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:53.246 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.039519+0000", 2024-09-11T06:50:53.246 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 32.170585703999997, 2024-09-11T06:50:53.246 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001298485, 2024-09-11T06:50:53.246 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:53.246 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:53.246 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:53.246 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:53.246 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:53.246 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:53.246 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 162 2024-09-11T06:50:53.247 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.247 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:53.247 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.247 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:53.247 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039519+0000", 2024-09-11T06:50:53.247 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.247 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.247 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.247 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:53.247 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039519+0000", 2024-09-11T06:50:53.247 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.247 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.247 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.248 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:53.248 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039520+0000", 2024-09-11T06:50:53.248 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9299999999999998e-07 2024-09-11T06:50:53.248 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.248 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.248 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:53.248 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039523+0000", 2024-09-11T06:50:53.248 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.247e-06 2024-09-11T06:50:53.248 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.248 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.248 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:53.248 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039529+0000", 2024-09-11T06:50:53.249 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8300000000000001e-06 2024-09-11T06:50:53.249 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.249 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.249 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.249 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039533+0000", 2024-09-11T06:50:53.249 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.2379999999999999e-06 2024-09-11T06:50:53.249 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.249 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.249 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:53.249 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.040236+0000", 2024-09-11T06:50:53.249 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00070326500000000001 2024-09-11T06:50:53.249 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.250 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.250 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:53.250 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.040299+0000", 2024-09-11T06:50:53.250 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3077999999999995e-05 2024-09-11T06:50:53.250 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.250 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.250 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:53.250 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.040818+0000", 2024-09-11T06:50:53.250 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00051833399999999996 2024-09-11T06:50:53.250 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.250 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:53.251 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.251 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.251 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.251 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:164 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 528384~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:53.251 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.041536+0000", 2024-09-11T06:50:53.251 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 32.168568981999996, 2024-09-11T06:50:53.251 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00119962, 2024-09-11T06:50:53.251 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:53.251 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:53.251 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:53.251 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:53.251 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:53.251 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:53.252 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 164 2024-09-11T06:50:53.252 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.252 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:53.252 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.252 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:53.252 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041536+0000", 2024-09-11T06:50:53.252 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.252 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.252 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.252 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:53.253 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041536+0000", 2024-09-11T06:50:53.253 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.253 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.253 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.253 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:53.253 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041537+0000", 2024-09-11T06:50:53.253 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3e-06 2024-09-11T06:50:53.253 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.253 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.253 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:53.253 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041543+0000", 2024-09-11T06:50:53.253 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2360000000000001e-06 2024-09-11T06:50:53.254 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.254 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.254 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:53.254 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041558+0000", 2024-09-11T06:50:53.254 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4909000000000001e-05 2024-09-11T06:50:53.254 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.254 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.254 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.254 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041569+0000", 2024-09-11T06:50:53.254 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0946e-05 2024-09-11T06:50:53.254 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.254 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.254 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:53.254 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.042138+0000", 2024-09-11T06:50:53.254 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000568996 2024-09-11T06:50:53.255 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.255 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.255 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:53.255 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.042190+0000", 2024-09-11T06:50:53.255 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1609e-05 2024-09-11T06:50:53.255 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.255 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.255 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:53.255 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.042735+0000", 2024-09-11T06:50:53.255 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00054562399999999998 2024-09-11T06:50:53.255 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.255 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:53.255 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.255 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.256 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.256 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:166 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 790528~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:53.256 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.043564+0000", 2024-09-11T06:50:53.256 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 32.166540585, 2024-09-11T06:50:53.256 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011042000000000001, 2024-09-11T06:50:53.256 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:53.256 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:53.256 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:53.256 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:53.256 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:53.256 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:53.256 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 166 2024-09-11T06:50:53.256 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.256 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:53.257 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.257 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:53.257 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043564+0000", 2024-09-11T06:50:53.257 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.257 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.257 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.257 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:53.257 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043564+0000", 2024-09-11T06:50:53.257 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.257 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.257 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.257 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:53.257 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043565+0000", 2024-09-11T06:50:53.257 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0449999999999999e-06 2024-09-11T06:50:53.258 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.258 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.258 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:53.258 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043571+0000", 2024-09-11T06:50:53.258 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.6849999999999998e-06 2024-09-11T06:50:53.258 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.258 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.258 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:53.258 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043586+0000", 2024-09-11T06:50:53.258 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4902000000000001e-05 2024-09-11T06:50:53.258 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.258 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.258 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.258 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043593+0000", 2024-09-11T06:50:53.259 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9240000000000001e-06 2024-09-11T06:50:53.259 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.259 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.259 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:53.259 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.044073+0000", 2024-09-11T06:50:53.259 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000479813 2024-09-11T06:50:53.259 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.259 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.259 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:53.259 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.044123+0000", 2024-09-11T06:50:53.259 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0908000000000001e-05 2024-09-11T06:50:53.260 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.260 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.260 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:53.260 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.044668+0000", 2024-09-11T06:50:53.260 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000544923 2024-09-11T06:50:53.260 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.260 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:53.260 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.260 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.260 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.260 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:178 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 2363392~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:53.260 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.061599+0000", 2024-09-11T06:50:53.260 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 32.148505870000001, 2024-09-11T06:50:53.261 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012194529999999999, 2024-09-11T06:50:53.261 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:53.261 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:53.261 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:53.261 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:53.261 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:53.261 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:53.261 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 178 2024-09-11T06:50:53.261 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.261 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:53.261 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.261 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:53.261 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061599+0000", 2024-09-11T06:50:53.261 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.262 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.262 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.262 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:53.262 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061599+0000", 2024-09-11T06:50:53.262 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.262 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.262 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.262 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:53.262 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061600+0000", 2024-09-11T06:50:53.262 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3600000000000003e-07 2024-09-11T06:50:53.262 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.262 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.263 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:53.263 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061603+0000", 2024-09-11T06:50:53.263 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.6550000000000002e-06 2024-09-11T06:50:53.263 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.263 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.263 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:53.263 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061618+0000", 2024-09-11T06:50:53.263 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4377e-05 2024-09-11T06:50:53.263 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.263 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.263 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.263 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061626+0000", 2024-09-11T06:50:53.263 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.0679999999999997e-06 2024-09-11T06:50:53.263 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.264 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.264 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:53.264 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.062098+0000", 2024-09-11T06:50:53.264 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00047251299999999998 2024-09-11T06:50:53.264 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.264 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.264 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:53.264 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.062253+0000", 2024-09-11T06:50:53.264 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015533100000000001 2024-09-11T06:50:53.264 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.264 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.264 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:53.265 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.062818+0000", 2024-09-11T06:50:53.265 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00056487300000000002 2024-09-11T06:50:53.265 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.265 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:53.265 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.265 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.265 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.265 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:184 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 3149824~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:53.265 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:50:53.265 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 32.143896167000001, 2024-09-11T06:50:53.265 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010685969999999999, 2024-09-11T06:50:53.265 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:53.265 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:53.265 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:53.266 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:53.266 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:53.266 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:53.266 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 184 2024-09-11T06:50:53.266 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.266 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:53.266 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.266 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:53.266 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:50:53.266 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.266 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.266 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.266 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:53.267 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:50:53.267 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.267 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.267 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.267 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:53.267 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:50:53.267 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.6700000000000003e-07 2024-09-11T06:50:53.267 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.267 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.267 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:53.267 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066212+0000", 2024-09-11T06:50:53.267 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.1669999999999999e-06 2024-09-11T06:50:53.267 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.267 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.268 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:53.268 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066219+0000", 2024-09-11T06:50:53.268 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5699999999999998e-06 2024-09-11T06:50:53.268 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.268 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.268 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.268 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066223+0000", 2024-09-11T06:50:53.268 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3540000000000002e-06 2024-09-11T06:50:53.268 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.268 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.268 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:53.268 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066713+0000", 2024-09-11T06:50:53.268 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00048985399999999996 2024-09-11T06:50:53.269 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.269 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.269 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:53.269 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066813+0000", 2024-09-11T06:50:53.269 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00010003 2024-09-11T06:50:53.269 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.269 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.269 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:53.269 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.067277+0000", 2024-09-11T06:50:53.269 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000463955 2024-09-11T06:50:53.269 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.269 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:53.269 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.269 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.270 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.270 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:186 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 3411968~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:53.270 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.071198+0000", 2024-09-11T06:50:53.270 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 32.138906822999999, 2024-09-11T06:50:53.270 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012515779999999999, 2024-09-11T06:50:53.270 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:53.270 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:53.270 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:53.270 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:53.270 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:53.270 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:53.270 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 186 2024-09-11T06:50:53.270 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.270 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:53.271 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.271 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:53.271 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071198+0000", 2024-09-11T06:50:53.271 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.271 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.271 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.271 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:53.271 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071198+0000", 2024-09-11T06:50:53.271 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.271 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.271 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.271 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:53.271 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071199+0000", 2024-09-11T06:50:53.271 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0299999999999999e-07 2024-09-11T06:50:53.272 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.272 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.272 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:53.272 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071201+0000", 2024-09-11T06:50:53.272 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8540000000000001e-06 2024-09-11T06:50:53.272 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.272 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.272 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:53.272 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071209+0000", 2024-09-11T06:50:53.272 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3629999999999996e-06 2024-09-11T06:50:53.272 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.272 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.272 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.272 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071214+0000", 2024-09-11T06:50:53.273 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2109999999999997e-06 2024-09-11T06:50:53.273 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.273 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.273 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:53.273 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071915+0000", 2024-09-11T06:50:53.273 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00070101799999999998 2024-09-11T06:50:53.273 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.273 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.273 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:53.273 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.072011+0000", 2024-09-11T06:50:53.273 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.5755999999999996e-05 2024-09-11T06:50:53.273 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.274 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.274 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:53.274 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.072450+0000", 2024-09-11T06:50:53.274 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000438773 2024-09-11T06:50:53.274 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.274 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:53.274 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.274 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.274 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.274 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:190 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 3936256~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:53.274 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.074875+0000", 2024-09-11T06:50:53.274 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 32.135229907999999, 2024-09-11T06:50:53.274 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0013119620000000001, 2024-09-11T06:50:53.274 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:53.275 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:53.275 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:53.275 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:53.275 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:53.275 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:53.275 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 190 2024-09-11T06:50:53.275 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.275 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:53.275 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.275 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:53.275 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074875+0000", 2024-09-11T06:50:53.275 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.275 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.275 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.276 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:53.276 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074875+0000", 2024-09-11T06:50:53.276 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.276 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.276 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.276 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:53.276 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074876+0000", 2024-09-11T06:50:53.276 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7400000000000003e-07 2024-09-11T06:50:53.276 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.276 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.276 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:53.276 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074887+0000", 2024-09-11T06:50:53.276 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1817999999999999e-05 2024-09-11T06:50:53.277 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.277 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.277 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:53.277 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074896+0000", 2024-09-11T06:50:53.277 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9120000000000004e-06 2024-09-11T06:50:53.277 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.277 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.277 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.277 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074900+0000", 2024-09-11T06:50:53.277 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.2220000000000004e-06 2024-09-11T06:50:53.277 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.277 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.277 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:53.277 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.075198+0000", 2024-09-11T06:50:53.278 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00029750399999999998 2024-09-11T06:50:53.278 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.278 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.278 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:53.278 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.075430+0000", 2024-09-11T06:50:53.278 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023200400000000001 2024-09-11T06:50:53.278 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.278 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.278 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:53.278 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.076187+0000", 2024-09-11T06:50:53.278 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00075682799999999997 2024-09-11T06:50:53.278 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.278 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:53.278 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.279 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.279 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.279 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:359 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 135168~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:53.279 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.298832+0000", 2024-09-11T06:50:53.279 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 31.911272274000002, 2024-09-11T06:50:53.279 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001078575, 2024-09-11T06:50:53.279 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:53.279 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:53.279 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:53.279 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:53.279 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:53.280 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:53.280 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 359 2024-09-11T06:50:53.280 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.280 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:53.280 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.280 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:53.280 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298832+0000", 2024-09-11T06:50:53.280 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.280 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.280 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.281 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:53.281 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298832+0000", 2024-09-11T06:50:53.281 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.281 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.281 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.281 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:53.281 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298834+0000", 2024-09-11T06:50:53.281 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.646e-06 2024-09-11T06:50:53.281 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.281 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.281 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:53.281 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298845+0000", 2024-09-11T06:50:53.281 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.081e-05 2024-09-11T06:50:53.282 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.282 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.282 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:53.282 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298872+0000", 2024-09-11T06:50:53.282 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7382e-05 2024-09-11T06:50:53.282 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.282 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.282 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.282 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298884+0000", 2024-09-11T06:50:53.282 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1956e-05 2024-09-11T06:50:53.283 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.283 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.283 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:53.283 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298959+0000", 2024-09-11T06:50:53.283 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5214000000000002e-05 2024-09-11T06:50:53.283 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.283 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.283 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:53.283 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.299090+0000", 2024-09-11T06:50:53.283 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000130247 2024-09-11T06:50:53.283 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.283 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.283 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:53.283 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.299911+0000", 2024-09-11T06:50:53.284 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00082131999999999995 2024-09-11T06:50:53.284 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.284 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:53.284 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.284 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.284 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.284 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:360 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:53.284 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.298914+0000", 2024-09-11T06:50:53.285 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 31.911190604000002, 2024-09-11T06:50:53.285 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0023571690000000001, 2024-09-11T06:50:53.285 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:53.285 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:53.285 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:53.285 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:53.285 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:53.285 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:53.285 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 360 2024-09-11T06:50:53.285 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.285 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:53.285 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.285 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:53.285 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298914+0000", 2024-09-11T06:50:53.286 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.286 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.286 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.286 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:53.286 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298914+0000", 2024-09-11T06:50:53.286 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.286 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.286 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.286 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:53.286 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298915+0000", 2024-09-11T06:50:53.287 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9300000000000006e-07 2024-09-11T06:50:53.287 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.287 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.287 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:53.287 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298921+0000", 2024-09-11T06:50:53.287 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.6030000000000004e-06 2024-09-11T06:50:53.287 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.287 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.287 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:53.287 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298932+0000", 2024-09-11T06:50:53.287 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.082e-05 2024-09-11T06:50:53.287 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.287 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.288 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.288 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298939+0000", 2024-09-11T06:50:53.288 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7919999999999993e-06 2024-09-11T06:50:53.288 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.288 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.288 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:53.288 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.300011+0000", 2024-09-11T06:50:53.288 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001071179 2024-09-11T06:50:53.288 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.288 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.288 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:53.289 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.300245+0000", 2024-09-11T06:50:53.289 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000234434 2024-09-11T06:50:53.289 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.289 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.289 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:53.289 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.301271+0000", 2024-09-11T06:50:53.289 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010263480000000001 2024-09-11T06:50:53.289 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.289 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:53.289 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.289 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.289 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.290 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:362 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 528384~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:53.290 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.302376+0000", 2024-09-11T06:50:53.290 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 31.907728281000001, 2024-09-11T06:50:53.290 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014132909999999999, 2024-09-11T06:50:53.290 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:53.290 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:53.290 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:53.290 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:53.290 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:53.290 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:53.290 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 362 2024-09-11T06:50:53.290 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.290 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:53.291 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.291 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:53.291 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302376+0000", 2024-09-11T06:50:53.291 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.291 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.291 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.291 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:53.291 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302376+0000", 2024-09-11T06:50:53.291 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.291 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.291 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.291 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:53.291 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302378+0000", 2024-09-11T06:50:53.292 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1629999999999999e-06 2024-09-11T06:50:53.292 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.292 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.292 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:53.292 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302383+0000", 2024-09-11T06:50:53.292 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.463e-06 2024-09-11T06:50:53.292 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.292 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.292 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:53.292 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302401+0000", 2024-09-11T06:50:53.292 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8219e-05 2024-09-11T06:50:53.292 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.292 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.293 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.293 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302411+0000", 2024-09-11T06:50:53.293 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9000000000000001e-06 2024-09-11T06:50:53.293 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.293 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.293 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:53.293 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302524+0000", 2024-09-11T06:50:53.293 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000113199 2024-09-11T06:50:53.293 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.293 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.293 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:53.293 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302745+0000", 2024-09-11T06:50:53.293 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022057399999999999 2024-09-11T06:50:53.293 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.294 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.294 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:53.294 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.303790+0000", 2024-09-11T06:50:53.294 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010447729999999999 2024-09-11T06:50:53.294 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.294 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:53.294 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.294 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.294 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.294 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:363 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 659456~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:53.294 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.302938+0000", 2024-09-11T06:50:53.295 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 31.907166688, 2024-09-11T06:50:53.295 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00185012, 2024-09-11T06:50:53.295 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:53.295 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:53.295 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:53.295 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:53.295 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:53.295 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:53.295 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 363 2024-09-11T06:50:53.295 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.295 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:53.295 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.296 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:53.296 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302938+0000", 2024-09-11T06:50:53.296 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.296 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.296 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.296 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:53.296 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302938+0000", 2024-09-11T06:50:53.296 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.296 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.296 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.296 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:53.296 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302939+0000", 2024-09-11T06:50:53.296 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2300000000000001e-06 2024-09-11T06:50:53.297 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.297 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.297 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:53.297 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302944+0000", 2024-09-11T06:50:53.297 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.493e-06 2024-09-11T06:50:53.297 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.297 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.297 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:53.297 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302960+0000", 2024-09-11T06:50:53.297 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.632e-05 2024-09-11T06:50:53.297 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.297 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.297 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.298 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302972+0000", 2024-09-11T06:50:53.298 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1885000000000001e-05 2024-09-11T06:50:53.298 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.298 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.298 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:53.298 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.303835+0000", 2024-09-11T06:50:53.298 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000863141 2024-09-11T06:50:53.298 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.298 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.298 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:53.298 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.303965+0000", 2024-09-11T06:50:53.298 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013032199999999999 2024-09-11T06:50:53.298 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.299 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.299 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:53.299 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.304788+0000", 2024-09-11T06:50:53.299 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00082272900000000002 2024-09-11T06:50:53.299 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.299 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:53.299 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.299 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.299 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.299 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:367 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 1183744~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:53.299 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.312392+0000", 2024-09-11T06:50:53.299 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 31.897713048, 2024-09-11T06:50:53.300 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011343460000000001, 2024-09-11T06:50:53.300 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:53.300 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:53.300 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:53.300 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:53.300 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:53.300 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:53.300 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 367 2024-09-11T06:50:53.300 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.300 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:53.300 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.300 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:53.301 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312392+0000", 2024-09-11T06:50:53.301 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.301 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.301 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.301 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:53.301 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312392+0000", 2024-09-11T06:50:53.301 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.301 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.301 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.301 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:53.301 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312393+0000", 2024-09-11T06:50:53.301 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6199999999999999e-06 2024-09-11T06:50:53.301 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.301 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.302 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:53.302 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312400+0000", 2024-09-11T06:50:53.302 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0700000000000001e-06 2024-09-11T06:50:53.302 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.302 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.302 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:53.302 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312420+0000", 2024-09-11T06:50:53.302 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9375999999999999e-05 2024-09-11T06:50:53.302 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.302 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.302 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.302 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312430+0000", 2024-09-11T06:50:53.302 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0209e-05 2024-09-11T06:50:53.303 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.303 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.303 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:53.303 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312549+0000", 2024-09-11T06:50:53.303 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000118779 2024-09-11T06:50:53.303 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.303 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.303 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:53.303 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312767+0000", 2024-09-11T06:50:53.303 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021786400000000001 2024-09-11T06:50:53.303 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.303 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.303 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:53.303 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313526+0000", 2024-09-11T06:50:53.303 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00075942800000000003 2024-09-11T06:50:53.304 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.304 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:53.304 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.304 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.304 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.304 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:368 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 1314816~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:53.304 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.312454+0000", 2024-09-11T06:50:53.304 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 31.897650523999999, 2024-09-11T06:50:53.304 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014704550000000001, 2024-09-11T06:50:53.304 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:53.304 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:53.304 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:53.304 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:53.305 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:53.305 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:53.305 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 368 2024-09-11T06:50:53.305 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.305 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:53.305 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.305 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:53.305 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312454+0000", 2024-09-11T06:50:53.305 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.305 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.305 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.305 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:53.305 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312454+0000", 2024-09-11T06:50:53.306 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.306 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.306 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.306 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:53.306 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312455+0000", 2024-09-11T06:50:53.306 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4399999999999999e-07 2024-09-11T06:50:53.306 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.306 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.306 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:53.306 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312474+0000", 2024-09-11T06:50:53.306 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8899e-05 2024-09-11T06:50:53.306 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.306 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.306 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:53.306 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312485+0000", 2024-09-11T06:50:53.307 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0793e-05 2024-09-11T06:50:53.307 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.307 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.307 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.307 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312492+0000", 2024-09-11T06:50:53.307 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.593e-06 2024-09-11T06:50:53.307 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.307 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.307 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:53.307 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313576+0000", 2024-09-11T06:50:53.307 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010837970000000001 2024-09-11T06:50:53.307 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.307 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.308 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:53.308 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313620+0000", 2024-09-11T06:50:53.308 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3640000000000002e-05 2024-09-11T06:50:53.308 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.308 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.308 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:53.308 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313925+0000", 2024-09-11T06:50:53.308 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00030498899999999998 2024-09-11T06:50:53.308 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.308 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:53.308 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.308 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.308 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.308 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:370 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 1576960~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:53.309 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.314817+0000", 2024-09-11T06:50:53.309 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 31.895287815, 2024-09-11T06:50:53.309 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011606190000000001, 2024-09-11T06:50:53.309 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:53.309 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:53.309 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:53.309 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:53.309 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:53.309 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:53.309 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 370 2024-09-11T06:50:53.309 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.309 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:53.309 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.309 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:53.310 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314817+0000", 2024-09-11T06:50:53.310 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.310 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.310 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.310 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:53.310 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314817+0000", 2024-09-11T06:50:53.310 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.310 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.310 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.310 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:53.310 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314818+0000", 2024-09-11T06:50:53.310 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.463e-06 2024-09-11T06:50:53.310 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.311 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.311 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:53.311 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314826+0000", 2024-09-11T06:50:53.311 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3860000000000001e-06 2024-09-11T06:50:53.311 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.311 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.311 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:53.311 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314842+0000", 2024-09-11T06:50:53.311 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6220000000000001e-05 2024-09-11T06:50:53.311 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.311 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.311 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.311 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314854+0000", 2024-09-11T06:50:53.311 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1562e-05 2024-09-11T06:50:53.312 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.312 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.312 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:53.312 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.315212+0000", 2024-09-11T06:50:53.312 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000358012 2024-09-11T06:50:53.312 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.312 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.312 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:53.312 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.315433+0000", 2024-09-11T06:50:53.312 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022103800000000001 2024-09-11T06:50:53.312 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.312 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.312 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:53.313 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.315978+0000", 2024-09-11T06:50:53.313 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00054493800000000002 2024-09-11T06:50:53.313 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.313 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:53.313 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.313 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.313 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.313 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:57 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:50:53.313 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.568529+0000", 2024-09-11T06:50:53.313 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.6415752709999998, 2024-09-11T06:50:53.313 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004228229, 2024-09-11T06:50:53.313 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:53.313 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:53.313 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:53.314 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:53.314 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:50:53.314 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:50:53.314 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 57 2024-09-11T06:50:53.314 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.314 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:53.314 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.314 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:53.314 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568529+0000", 2024-09-11T06:50:53.314 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.314 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.314 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.314 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:53.314 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568529+0000", 2024-09-11T06:50:53.315 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:53.315 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.315 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.315 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:53.315 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568532+0000", 2024-09-11T06:50:53.315 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0000000000000001e-06 2024-09-11T06:50:53.315 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.315 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.315 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:53.315 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568547+0000", 2024-09-11T06:50:53.315 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4219999999999999e-05 2024-09-11T06:50:53.315 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.316 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.316 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:53.316 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568580+0000", 2024-09-11T06:50:53.316 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.3328999999999999e-05 2024-09-11T06:50:53.316 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.316 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.316 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.316 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568596+0000", 2024-09-11T06:50:53.316 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5990000000000001e-05 2024-09-11T06:50:53.316 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.317 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.317 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:53.317 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568728+0000", 2024-09-11T06:50:53.317 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013175400000000001 2024-09-11T06:50:53.317 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.317 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.317 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:53.317 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568738+0000", 2024-09-11T06:50:53.317 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0196e-05 2024-09-11T06:50:53.317 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.317 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.317 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:53.317 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.572464+0000", 2024-09-11T06:50:53.317 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0037264160000000002 2024-09-11T06:50:53.318 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.318 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.318 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:53.318 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.572584+0000", 2024-09-11T06:50:53.318 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000119438 2024-09-11T06:50:53.318 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:53.318 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:53.318 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:53.318 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.572758+0000", 2024-09-11T06:50:53.318 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017388599999999999 2024-09-11T06:50:53.318 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.318 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:53.318 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.319 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:53.319 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:53.319 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:53.320 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:53.319+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:53.321 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:50:53.421 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:53.419+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:53.450 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:53.451 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:53.451 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:50:53.451 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:53.466 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:50:53.522 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:53.518+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:53.589 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:53.589 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:53.589 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:50:53.589 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:50:53.589 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:53.603 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:50:53.645 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:53.642+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:53.723 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:53.718+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:53.773 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:53.773 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:50:53.773 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:50:53.773 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:50:53.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.773 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:54 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'32, mlcod=13'29)", 2024-09-11T06:50:53.773 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:50:53.773 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 107.558903957, 2024-09-11T06:50:53.773 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034271670000000001, 2024-09-11T06:50:53.774 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:53.774 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:53.774 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:53.774 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:53.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.774 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:53.774 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:50:53.774 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.774 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:53.774 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:50:53.774 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.775 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:53.775 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210361+0000", 2024-09-11T06:50:53.775 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1260000000000001e-06 2024-09-11T06:50:53.775 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.775 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:53.775 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210370+0000", 2024-09-11T06:50:53.775 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8200000000000003e-06 2024-09-11T06:50:53.775 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.775 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:53.775 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210390+0000", 2024-09-11T06:50:53.775 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0126e-05 2024-09-11T06:50:53.775 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210404+0000", 2024-09-11T06:50:53.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4589999999999999e-05 2024-09-11T06:50:53.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:53.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210538+0000", 2024-09-11T06:50:53.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000133719 2024-09-11T06:50:53.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:53.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210593+0000", 2024-09-11T06:50:53.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5016999999999999e-05 2024-09-11T06:50:53.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.777 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:53.777 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213695+0000", 2024-09-11T06:50:53.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003101866 2024-09-11T06:50:53.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.777 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:53.777 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213787+0000", 2024-09-11T06:50:53.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1903000000000004e-05 2024-09-11T06:50:53.777 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.777 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:53.777 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.778 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:55 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'33, mlcod=13'29)", 2024-09-11T06:50:53.778 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:50:53.778 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 107.557884719, 2024-09-11T06:50:53.778 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.005086091, 2024-09-11T06:50:53.778 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:53.778 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:53.778 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:53.778 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:53.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.778 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:53.778 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:50:53.779 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.779 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.779 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.779 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:53.779 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:50:53.779 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.779 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.779 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.779 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:53.779 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211380+0000", 2024-09-11T06:50:53.779 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.097e-06 2024-09-11T06:50:53.779 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.779 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.780 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:53.780 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211537+0000", 2024-09-11T06:50:53.780 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000157369 2024-09-11T06:50:53.780 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.780 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.780 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:53.780 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211559+0000", 2024-09-11T06:50:53.780 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000001e-05 2024-09-11T06:50:53.780 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.780 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.780 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.780 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211573+0000", 2024-09-11T06:50:53.780 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4946000000000001e-05 2024-09-11T06:50:53.780 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:53.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211676+0000", 2024-09-11T06:50:53.781 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0001026 2024-09-11T06:50:53.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:53.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211709+0000", 2024-09-11T06:50:53.781 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3071999999999998e-05 2024-09-11T06:50:53.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:53.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216379+0000", 2024-09-11T06:50:53.782 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0046693780000000001 2024-09-11T06:50:53.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.782 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:53.782 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216465+0000", 2024-09-11T06:50:53.782 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6469000000000002e-05 2024-09-11T06:50:53.782 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.782 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:53.782 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.782 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:56 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'34, mlcod=13'29)", 2024-09-11T06:50:53.783 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:50:53.783 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 107.55645086, 2024-09-11T06:50:53.783 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0060310629999999997, 2024-09-11T06:50:53.783 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:53.783 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:53.783 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:53.783 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:53.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.783 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:53.783 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:50:53.783 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.784 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:53.784 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:50:53.784 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.784 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.784 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:53.784 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212814+0000", 2024-09-11T06:50:53.784 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0869999999999999e-06 2024-09-11T06:50:53.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.784 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.784 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:53.784 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212822+0000", 2024-09-11T06:50:53.784 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0870000000000007e-06 2024-09-11T06:50:53.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.785 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.785 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:53.785 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212840+0000", 2024-09-11T06:50:53.785 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:50:53.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.785 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.785 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.785 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212854+0000", 2024-09-11T06:50:53.785 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4003e-05 2024-09-11T06:50:53.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.785 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.785 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:53.785 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213026+0000", 2024-09-11T06:50:53.786 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017207499999999999 2024-09-11T06:50:53.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.786 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:53.786 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213063+0000", 2024-09-11T06:50:53.786 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7187000000000002e-05 2024-09-11T06:50:53.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.786 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:53.786 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218762+0000", 2024-09-11T06:50:53.786 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0056993290000000004 2024-09-11T06:50:53.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.786 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:53.787 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218844+0000", 2024-09-11T06:50:53.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1701999999999998e-05 2024-09-11T06:50:53.787 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.787 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:53.787 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.787 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:60 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'38, mlcod=13'36)", 2024-09-11T06:50:53.787 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:50:53.787 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 107.54250951100001, 2024-09-11T06:50:53.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039254270000000004, 2024-09-11T06:50:53.787 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:53.787 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:53.788 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:53.788 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:53.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.788 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:53.788 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:50:53.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.788 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:53.788 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:50:53.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.788 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:53.789 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226756+0000", 2024-09-11T06:50:53.789 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-11T06:50:53.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.789 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.789 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:53.789 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226768+0000", 2024-09-11T06:50:53.789 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1919999999999999e-05 2024-09-11T06:50:53.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.789 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.789 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:53.789 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226797+0000", 2024-09-11T06:50:53.789 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9573e-05 2024-09-11T06:50:53.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.790 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.790 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.792 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226818+0000", 2024-09-11T06:50:53.792 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0432999999999999e-05 2024-09-11T06:50:53.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.792 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:53.793 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226909+0000", 2024-09-11T06:50:53.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0920000000000004e-05 2024-09-11T06:50:53.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.793 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.793 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:53.793 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226936+0000", 2024-09-11T06:50:53.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7472999999999999e-05 2024-09-11T06:50:53.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.793 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.793 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:53.793 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230596+0000", 2024-09-11T06:50:53.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0036602520000000001 2024-09-11T06:50:53.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.794 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:53.794 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230680+0000", 2024-09-11T06:50:53.794 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3106e-05 2024-09-11T06:50:53.794 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.794 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:53.794 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.794 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:107 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'69, mlcod=13'64)", 2024-09-11T06:50:53.794 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:50:53.794 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 107.326062054, 2024-09-11T06:50:53.794 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040728830000000002, 2024-09-11T06:50:53.795 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:53.795 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:53.795 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:53.795 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:53.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.795 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:53.795 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:50:53.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.795 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:53.795 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:50:53.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:53.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443203+0000", 2024-09-11T06:50:53.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1480000000000001e-06 2024-09-11T06:50:53.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:53.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443222+0000", 2024-09-11T06:50:53.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9040000000000001e-05 2024-09-11T06:50:53.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:53.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443246+0000", 2024-09-11T06:50:53.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4606000000000001e-05 2024-09-11T06:50:53.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443277+0000", 2024-09-11T06:50:53.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0207999999999999e-05 2024-09-11T06:50:53.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:53.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443340+0000", 2024-09-11T06:50:53.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3602000000000005e-05 2024-09-11T06:50:53.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:53.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443363+0000", 2024-09-11T06:50:53.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2331e-05 2024-09-11T06:50:53.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:53.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447216+0000", 2024-09-11T06:50:53.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0038539690000000001 2024-09-11T06:50:53.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:53.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447274+0000", 2024-09-11T06:50:53.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7979000000000002e-05 2024-09-11T06:50:53.798 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.799 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:53.799 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.799 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:111 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'73, mlcod=13'67)", 2024-09-11T06:50:53.799 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:50:53.799 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 107.319550424, 2024-09-11T06:50:53.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034372360000000002, 2024-09-11T06:50:53.799 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:53.799 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:53.799 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:53.799 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:53.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.799 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:53.800 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:50:53.800 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.800 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.800 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:53.800 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:50:53.800 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.800 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.800 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:53.800 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449714+0000", 2024-09-11T06:50:53.800 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5000000000000001e-07 2024-09-11T06:50:53.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.801 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:53.801 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449721+0000", 2024-09-11T06:50:53.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8530000000000003e-06 2024-09-11T06:50:53.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.801 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:53.801 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449736+0000", 2024-09-11T06:50:53.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5114e-05 2024-09-11T06:50:53.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.801 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.801 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449743+0000", 2024-09-11T06:50:53.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0210000000000002e-06 2024-09-11T06:50:53.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:53.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449804+0000", 2024-09-11T06:50:53.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0479999999999997e-05 2024-09-11T06:50:53.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:53.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449826+0000", 2024-09-11T06:50:53.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1922000000000001e-05 2024-09-11T06:50:53.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:53.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453104+0000", 2024-09-11T06:50:53.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003278652 2024-09-11T06:50:53.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.803 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:53.803 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453150+0000", 2024-09-11T06:50:53.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6244e-05 2024-09-11T06:50:53.803 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.803 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:53.803 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.803 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:114 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'76, mlcod=13'71)", 2024-09-11T06:50:53.803 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:50:53.804 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 107.314872353, 2024-09-11T06:50:53.804 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0042576660000000002, 2024-09-11T06:50:53.804 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:53.804 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:53.804 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:53.804 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:53.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.804 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:53.804 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:50:53.804 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.804 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:53.805 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:50:53.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.805 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:53.805 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454392+0000", 2024-09-11T06:50:53.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5999999999999991e-07 2024-09-11T06:50:53.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.805 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:53.805 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454399+0000", 2024-09-11T06:50:53.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.878e-06 2024-09-11T06:50:53.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:53.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454415+0000", 2024-09-11T06:50:53.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5659999999999999e-05 2024-09-11T06:50:53.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454422+0000", 2024-09-11T06:50:53.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:50:53.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:53.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454474+0000", 2024-09-11T06:50:53.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1860999999999997e-05 2024-09-11T06:50:53.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.807 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:53.807 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454497+0000", 2024-09-11T06:50:53.807 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3686e-05 2024-09-11T06:50:53.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.807 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:53.807 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458552+0000", 2024-09-11T06:50:53.807 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040544550000000002 2024-09-11T06:50:53.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.807 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:53.807 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458649+0000", 2024-09-11T06:50:53.808 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7005999999999999e-05 2024-09-11T06:50:53.808 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.808 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:53.808 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.808 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.808 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:115 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'77, mlcod=13'72)", 2024-09-11T06:50:53.808 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:53.808 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 107.31353174100001, 2024-09-11T06:50:53.808 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003355584, 2024-09-11T06:50:53.808 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:53.808 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:53.808 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:53.809 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:53.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.809 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:53.809 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:53.809 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.809 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:53.809 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:53.809 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.809 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:53.809 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:50:53.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5999999999999999e-07 2024-09-11T06:50:53.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:53.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455803+0000", 2024-09-11T06:50:53.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0310999999999998e-05 2024-09-11T06:50:53.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:53.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455815+0000", 2024-09-11T06:50:53.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2787e-05 2024-09-11T06:50:53.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.811 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455823+0000", 2024-09-11T06:50:53.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5959999999999998e-06 2024-09-11T06:50:53.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.811 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:53.811 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455875+0000", 2024-09-11T06:50:53.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2054e-05 2024-09-11T06:50:53.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.811 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:53.811 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455896+0000", 2024-09-11T06:50:53.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0554999999999999e-05 2024-09-11T06:50:53.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.812 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:53.812 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459000+0000", 2024-09-11T06:50:53.812 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003104842 2024-09-11T06:50:53.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.812 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:53.812 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459087+0000", 2024-09-11T06:50:53.812 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6978999999999995e-05 2024-09-11T06:50:53.812 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.812 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:53.812 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.813 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:120 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'82, mlcod=13'78)", 2024-09-11T06:50:53.813 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:50:53.813 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 107.30285445600001, 2024-09-11T06:50:53.813 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0064612549999999999, 2024-09-11T06:50:53.813 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:53.813 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:53.813 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:53.813 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:53.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.813 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:53.813 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:50:53.813 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.814 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:53.814 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:50:53.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.814 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:53.814 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466410+0000", 2024-09-11T06:50:53.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0389999999999999e-06 2024-09-11T06:50:53.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:53.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466496+0000", 2024-09-11T06:50:53.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5519000000000006e-05 2024-09-11T06:50:53.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:53.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466510+0000", 2024-09-11T06:50:53.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3842999999999999e-05 2024-09-11T06:50:53.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466517+0000", 2024-09-11T06:50:53.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5229999999999998e-06 2024-09-11T06:50:53.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.816 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:53.816 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466595+0000", 2024-09-11T06:50:53.816 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8019e-05 2024-09-11T06:50:53.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.816 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:53.816 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466612+0000", 2024-09-11T06:50:53.816 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7013000000000001e-05 2024-09-11T06:50:53.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.816 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:53.816 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472787+0000", 2024-09-11T06:50:53.817 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0061750269999999996 2024-09-11T06:50:53.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.817 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.817 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:53.817 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472870+0000", 2024-09-11T06:50:53.817 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3271999999999998e-05 2024-09-11T06:50:53.817 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.817 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:53.817 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.817 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.817 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:121 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'83, mlcod=13'78)", 2024-09-11T06:50:53.817 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:50:53.817 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 107.30162966, 2024-09-11T06:50:53.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055613069999999997, 2024-09-11T06:50:53.818 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:53.818 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:53.818 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:53.818 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:53.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.818 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:53.818 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:50:53.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.818 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:53.818 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:50:53.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:53.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467635+0000", 2024-09-11T06:50:53.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6899999999999996e-07 2024-09-11T06:50:53.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:53.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467727+0000", 2024-09-11T06:50:53.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1800999999999997e-05 2024-09-11T06:50:53.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:53.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467740+0000", 2024-09-11T06:50:53.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3747999999999999e-05 2024-09-11T06:50:53.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467753+0000", 2024-09-11T06:50:53.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2807e-05 2024-09-11T06:50:53.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:53.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467812+0000", 2024-09-11T06:50:53.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8381999999999998e-05 2024-09-11T06:50:53.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.821 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:53.821 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467828+0000", 2024-09-11T06:50:53.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6728e-05 2024-09-11T06:50:53.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.821 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:53.821 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473112+0000", 2024-09-11T06:50:53.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052833619999999998 2024-09-11T06:50:53.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.821 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:53.821 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473195+0000", 2024-09-11T06:50:53.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3609999999999994e-05 2024-09-11T06:50:53.822 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.822 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:53.822 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.822 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.822 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:131 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'89, mlcod=13'85)", 2024-09-11T06:50:53.822 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:50:53.822 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 106.950223945, 2024-09-11T06:50:53.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052492260000000001, 2024-09-11T06:50:53.822 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:53.822 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:53.822 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:53.823 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:53.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.823 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:53.823 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:50:53.823 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.823 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:53.823 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:50:53.823 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.823 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:53.823 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819041+0000", 2024-09-11T06:50:53.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4360000000000001e-06 2024-09-11T06:50:53.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.824 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:53.824 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819100+0000", 2024-09-11T06:50:53.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9225999999999998e-05 2024-09-11T06:50:53.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.824 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:53.824 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819128+0000", 2024-09-11T06:50:53.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.779e-05 2024-09-11T06:50:53.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819142+0000", 2024-09-11T06:50:53.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3573e-05 2024-09-11T06:50:53.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:53.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819263+0000", 2024-09-11T06:50:53.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000121086 2024-09-11T06:50:53.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:53.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819292+0000", 2024-09-11T06:50:53.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9363e-05 2024-09-11T06:50:53.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.826 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:53.826 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824185+0000", 2024-09-11T06:50:53.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0048924620000000002 2024-09-11T06:50:53.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.826 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:53.826 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824289+0000", 2024-09-11T06:50:53.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010429000000000001 2024-09-11T06:50:53.826 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.826 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:53.826 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.827 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.827 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:132 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'90, mlcod=13'85)", 2024-09-11T06:50:53.827 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:50:53.827 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 106.949449255, 2024-09-11T06:50:53.827 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.004532342, 2024-09-11T06:50:53.827 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:53.827 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:53.827 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:53.827 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:53.827 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.827 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:53.827 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:50:53.827 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:53.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:50:53.828 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:53.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819816+0000", 2024-09-11T06:50:53.828 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.61e-06 2024-09-11T06:50:53.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:53.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819827+0000", 2024-09-11T06:50:53.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1e-05 2024-09-11T06:50:53.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:53.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819851+0000", 2024-09-11T06:50:53.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4267e-05 2024-09-11T06:50:53.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819864+0000", 2024-09-11T06:50:53.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2767e-05 2024-09-11T06:50:53.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.830 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:53.830 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820071+0000", 2024-09-11T06:50:53.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000207061 2024-09-11T06:50:53.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.830 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:53.830 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820103+0000", 2024-09-11T06:50:53.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2326999999999997e-05 2024-09-11T06:50:53.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.830 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:53.830 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824302+0000", 2024-09-11T06:50:53.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0041986000000000002 2024-09-11T06:50:53.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:53.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824347+0000", 2024-09-11T06:50:53.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4709999999999997e-05 2024-09-11T06:50:53.831 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.831 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:53.831 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.831 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:133 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'91, mlcod=13'86)", 2024-09-11T06:50:53.831 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:50:53.831 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 106.948817596, 2024-09-11T06:50:53.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039421459999999997, 2024-09-11T06:50:53.831 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:53.832 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:53.832 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:53.832 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:53.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:53.832 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:50:53.832 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:53.832 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:50:53.832 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.833 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:53.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820448+0000", 2024-09-11T06:50:53.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.84e-06 2024-09-11T06:50:53.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.833 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:53.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820463+0000", 2024-09-11T06:50:53.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5356e-05 2024-09-11T06:50:53.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.833 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:53.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820499+0000", 2024-09-11T06:50:53.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.625e-05 2024-09-11T06:50:53.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.834 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.834 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820546+0000", 2024-09-11T06:50:53.834 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7017000000000001e-05 2024-09-11T06:50:53.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.834 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:53.834 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820882+0000", 2024-09-11T06:50:53.834 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033512 2024-09-11T06:50:53.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.834 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:53.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820941+0000", 2024-09-11T06:50:53.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8922999999999998e-05 2024-09-11T06:50:53.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:53.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824351+0000", 2024-09-11T06:50:53.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034101940000000001 2024-09-11T06:50:53.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:53.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824388+0000", 2024-09-11T06:50:53.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7446e-05 2024-09-11T06:50:53.835 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.835 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:53.836 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.836 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:139 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'97, mlcod=13'86)", 2024-09-11T06:50:53.836 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:50:53.836 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 106.94340761799999, 2024-09-11T06:50:53.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003732221, 2024-09-11T06:50:53.836 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:53.836 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:53.836 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:53.836 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:53.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:53.837 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:50:53.837 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.837 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.837 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:53.837 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:50:53.837 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.837 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.837 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:53.837 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825858+0000", 2024-09-11T06:50:53.837 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5829999999999999e-06 2024-09-11T06:50:53.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.837 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:53.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825874+0000", 2024-09-11T06:50:53.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5967000000000001e-05 2024-09-11T06:50:53.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:53.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825909+0000", 2024-09-11T06:50:53.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5692999999999999e-05 2024-09-11T06:50:53.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825923+0000", 2024-09-11T06:50:53.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4080000000000001e-05 2024-09-11T06:50:53.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.839 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:53.839 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826010+0000", 2024-09-11T06:50:53.839 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6248999999999994e-05 2024-09-11T06:50:53.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.839 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:53.839 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826038+0000", 2024-09-11T06:50:53.839 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8010000000000001e-05 2024-09-11T06:50:53.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.839 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:53.839 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829499+0000", 2024-09-11T06:50:53.840 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003461783 2024-09-11T06:50:53.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.840 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:53.840 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829588+0000", 2024-09-11T06:50:53.840 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8856000000000004e-05 2024-09-11T06:50:53.840 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.840 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:53.840 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.840 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:140 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'98, mlcod=13'93)", 2024-09-11T06:50:53.840 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:50:53.841 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 106.941556889, 2024-09-11T06:50:53.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0057995929999999996, 2024-09-11T06:50:53.841 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:53.841 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:53.841 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:53.841 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:53.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:53.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:50:53.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:53.842 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:50:53.842 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.842 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:53.842 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827708+0000", 2024-09-11T06:50:53.842 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.547e-06 2024-09-11T06:50:53.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.842 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:53.842 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827765+0000", 2024-09-11T06:50:53.842 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6266e-05 2024-09-11T06:50:53.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.842 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:53.843 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827791+0000", 2024-09-11T06:50:53.843 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6098999999999999e-05 2024-09-11T06:50:53.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.843 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.843 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.843 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827806+0000", 2024-09-11T06:50:53.843 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5537e-05 2024-09-11T06:50:53.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.843 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.843 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:53.843 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827928+0000", 2024-09-11T06:50:53.843 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000122012 2024-09-11T06:50:53.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.844 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:53.844 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827961+0000", 2024-09-11T06:50:53.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2663999999999998e-05 2024-09-11T06:50:53.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.844 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:53.844 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833409+0000", 2024-09-11T06:50:53.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054479350000000001 2024-09-11T06:50:53.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.844 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:53.844 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833506+0000", 2024-09-11T06:50:53.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7533000000000003e-05 2024-09-11T06:50:53.845 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.845 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:53.845 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.845 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:141 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'99, mlcod=13'93)", 2024-09-11T06:50:53.845 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:50:53.845 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 106.94066350600001, 2024-09-11T06:50:53.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055394709999999998, 2024-09-11T06:50:53.845 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:53.845 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:53.845 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:53.845 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:53.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:53.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:50:53.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:53.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:50:53.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:53.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828602+0000", 2024-09-11T06:50:53.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.46e-06 2024-09-11T06:50:53.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.847 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:53.847 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828618+0000", 2024-09-11T06:50:53.847 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6592999999999999e-05 2024-09-11T06:50:53.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.847 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:53.847 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828646+0000", 2024-09-11T06:50:53.847 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7552999999999999e-05 2024-09-11T06:50:53.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.847 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.847 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828663+0000", 2024-09-11T06:50:53.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.719e-05 2024-09-11T06:50:53.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.848 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:53.848 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829023+0000", 2024-09-11T06:50:53.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036054799999999998 2024-09-11T06:50:53.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.848 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:53.848 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829062+0000", 2024-09-11T06:50:53.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8865999999999997e-05 2024-09-11T06:50:53.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.848 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:53.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834078+0000", 2024-09-11T06:50:53.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050157480000000004 2024-09-11T06:50:53.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.849 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:53.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834140+0000", 2024-09-11T06:50:53.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1513000000000005e-05 2024-09-11T06:50:53.849 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.849 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:53.849 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.849 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:145 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'103, mlcod=13'98)", 2024-09-11T06:50:53.850 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:50:53.850 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 106.93071100900001, 2024-09-11T06:50:53.850 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054455409999999999, 2024-09-11T06:50:53.850 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:53.850 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:53.850 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:53.850 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:53.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.850 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:53.850 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:50:53.850 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:53.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:50:53.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:53.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838554+0000", 2024-09-11T06:50:53.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3430000000000001e-06 2024-09-11T06:50:53.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:53.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838640+0000", 2024-09-11T06:50:53.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5569999999999996e-05 2024-09-11T06:50:53.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.852 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.852 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:53.852 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838660+0000", 2024-09-11T06:50:53.852 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0576000000000001e-05 2024-09-11T06:50:53.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.852 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.852 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.852 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838674+0000", 2024-09-11T06:50:53.852 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3529999999999999e-05 2024-09-11T06:50:53.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.852 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.852 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:53.852 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838838+0000", 2024-09-11T06:50:53.853 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016468799999999999 2024-09-11T06:50:53.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.853 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.853 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:53.853 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838885+0000", 2024-09-11T06:50:53.853 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6884e-05 2024-09-11T06:50:53.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.853 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.853 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:53.853 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843901+0000", 2024-09-11T06:50:53.853 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050162310000000003 2024-09-11T06:50:53.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.853 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.853 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:53.854 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843998+0000", 2024-09-11T06:50:53.854 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.6718999999999993e-05 2024-09-11T06:50:53.854 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.854 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:53.854 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.854 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4558.0:1 3.3 e56/55 3:c4c92e5a:::rbd_trash:head v 56'1, mlcod=0'0)", 2024-09-11T06:50:53.854 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:50:53.854 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 36.677907947000001, 2024-09-11T06:50:53.854 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0053791810000000002, 2024-09-11T06:50:53.854 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:53.854 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:53.854 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:53.855 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:53.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.855 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:53.855 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:50:53.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.855 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:53.855 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:50:53.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.856 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:53.856 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091357+0000", 2024-09-11T06:50:53.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.54e-07 2024-09-11T06:50:53.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.856 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:53.856 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091376+0000", 2024-09-11T06:50:53.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9769e-05 2024-09-11T06:50:53.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.856 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:53.856 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091396+0000", 2024-09-11T06:50:53.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0152999999999998e-05 2024-09-11T06:50:53.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.857 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.857 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091416+0000", 2024-09-11T06:50:53.857 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0006000000000001e-05 2024-09-11T06:50:53.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.857 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:53.857 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091506+0000", 2024-09-11T06:50:53.857 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9325000000000003e-05 2024-09-11T06:50:53.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.857 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:53.857 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091523+0000", 2024-09-11T06:50:53.858 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.77e-05 2024-09-11T06:50:53.858 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.858 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:53.858 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.096685+0000", 2024-09-11T06:50:53.858 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0051618569999999997 2024-09-11T06:50:53.858 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.858 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:53.858 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.096735+0000", 2024-09-11T06:50:53.858 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9517e-05 2024-09-11T06:50:53.858 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.858 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:53.859 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.859 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.859 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.5544.0:67 2.5 e82/69 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head v 82'77, mlcod=81'74)", 2024-09-11T06:50:53.859 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.595302+0000", 2024-09-11T06:50:53.859 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.1739615309999998, 2024-09-11T06:50:53.859 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054511999999999998, 2024-09-11T06:50:53.859 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:53.859 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:53.859 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:53.859 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:53.859 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.859 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:53.859 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595302+0000", 2024-09-11T06:50:53.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.860 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:53.860 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595302+0000", 2024-09-11T06:50:53.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.860 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:53.860 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595303+0000", 2024-09-11T06:50:53.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2699999999999999e-07 2024-09-11T06:50:53.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.861 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:53.861 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595522+0000", 2024-09-11T06:50:53.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000218946 2024-09-11T06:50:53.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.861 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:53.861 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595544+0000", 2024-09-11T06:50:53.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2558999999999999e-05 2024-09-11T06:50:53.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.861 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.861 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595557+0000", 2024-09-11T06:50:53.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2653000000000001e-05 2024-09-11T06:50:53.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.862 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:53.862 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595662+0000", 2024-09-11T06:50:53.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000104862 2024-09-11T06:50:53.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.862 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:53.862 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595699+0000", 2024-09-11T06:50:53.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7376000000000003e-05 2024-09-11T06:50:53.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.862 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:53.862 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.600688+0000", 2024-09-11T06:50:53.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0049885279999999999 2024-09-11T06:50:53.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:53.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.600753+0000", 2024-09-11T06:50:53.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5549000000000005e-05 2024-09-11T06:50:53.863 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.863 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:53.863 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.863 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.5544.0:90 2.5 e82/69 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head v 82'86, mlcod=82'84)", 2024-09-11T06:50:53.863 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.725934+0000", 2024-09-11T06:50:53.863 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.0433297699999997, 2024-09-11T06:50:53.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0044823220000000004, 2024-09-11T06:50:53.864 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:53.864 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:53.864 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:53.864 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:53.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.864 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:53.864 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.725934+0000", 2024-09-11T06:50:53.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.864 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:53.864 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.725934+0000", 2024-09-11T06:50:53.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:53.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:53.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.725936+0000", 2024-09-11T06:50:53.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9269999999999999e-06 2024-09-11T06:50:53.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:53.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.727188+0000", 2024-09-11T06:50:53.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012522779999999999 2024-09-11T06:50:53.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.866 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:53.866 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.727217+0000", 2024-09-11T06:50:53.866 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9373000000000002e-05 2024-09-11T06:50:53.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.866 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:53.866 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.727231+0000", 2024-09-11T06:50:53.866 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3499999999999999e-05 2024-09-11T06:50:53.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.866 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:53.866 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.727335+0000", 2024-09-11T06:50:53.866 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000104064 2024-09-11T06:50:53.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.867 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:53.867 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.727374+0000", 2024-09-11T06:50:53.867 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8967000000000001e-05 2024-09-11T06:50:53.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.867 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:53.867 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.730326+0000", 2024-09-11T06:50:53.867 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0029516519999999999 2024-09-11T06:50:53.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:53.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:53.867 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:53.867 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.730416+0000", 2024-09-11T06:50:53.868 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0561000000000002e-05 2024-09-11T06:50:53.868 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.868 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:53.868 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.868 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:53.868 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:53.868 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:53.870 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:50:53.871 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is unset 2024-09-11T06:50:53.871 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:53.866+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:53.970 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:53.971+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:54.040 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:54.040 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:54.041 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:50:54.041 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:54.055 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:50:54.101 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:54.098+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:54.172 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:54.172 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:54.172 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:50:54.172 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:50:54.172 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:54.172 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:54.166+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:54.186 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:50:54.273 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:54.271+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:54.355 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:54.355 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:50:54.355 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:50:54.356 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:50:54.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.356 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4558.0:5 3.2 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-11T06:50:54.356 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:50:54.356 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 36.262443300000001, 2024-09-11T06:50:54.356 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0062353959999999998, 2024-09-11T06:50:54.356 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.356 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.356 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:54.356 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.356 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4558", 2024-09-11T06:50:54.356 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1478388898", 2024-09-11T06:50:54.356 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 5 2024-09-11T06:50:54.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.357 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.357 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.357 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:50:54.357 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.357 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.357 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:50:54.357 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.357 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.358 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088422+0000", 2024-09-11T06:50:54.358 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7859999999999998e-06 2024-09-11T06:50:54.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.358 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.358 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088440+0000", 2024-09-11T06:50:54.358 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8185999999999998e-05 2024-09-11T06:50:54.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.358 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.358 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088479+0000", 2024-09-11T06:50:54.358 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8248999999999999e-05 2024-09-11T06:50:54.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.359 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088497+0000", 2024-09-11T06:50:54.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8889000000000002e-05 2024-09-11T06:50:54.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.359 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088658+0000", 2024-09-11T06:50:54.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000160271 2024-09-11T06:50:54.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.359 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op must wait for map", 2024-09-11T06:50:54.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088668+0000", 2024-09-11T06:50:54.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0409e-05 2024-09-11T06:50:54.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.360 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.360 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092347+0000", 2024-09-11T06:50:54.360 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00367915 2024-09-11T06:50:54.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.360 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.360 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092541+0000", 2024-09-11T06:50:54.360 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000193833 2024-09-11T06:50:54.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.360 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-11T06:50:54.360 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092732+0000", 2024-09-11T06:50:54.361 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000190772 2024-09-11T06:50:54.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.361 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:54.361 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094136+0000", 2024-09-11T06:50:54.361 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014040809999999999 2024-09-11T06:50:54.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.361 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:54.361 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094506+0000", 2024-09-11T06:50:54.361 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000369959 2024-09-11T06:50:54.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.362 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:54.362 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094590+0000", 2024-09-11T06:50:54.362 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4033e-05 2024-09-11T06:50:54.362 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.362 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.362 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.362 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094654+0000", 2024-09-11T06:50:54.362 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3777999999999998e-05 2024-09-11T06:50:54.362 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.362 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.362 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.362 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.362 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.362 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:261 2.6 2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:54.363 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.154411+0000", 2024-09-11T06:50:54.363 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.196450501999998, 2024-09-11T06:50:54.363 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002245024, 2024-09-11T06:50:54.363 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.363 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.363 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:54.363 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.363 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:54.363 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:54.363 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 261 2024-09-11T06:50:54.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.363 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.363 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.363 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.364 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154411+0000", 2024-09-11T06:50:54.364 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.364 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.364 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154411+0000", 2024-09-11T06:50:54.364 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.364 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.364 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154412+0000", 2024-09-11T06:50:54.364 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:50:54.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.365 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.365 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154416+0000", 2024-09-11T06:50:54.365 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3930000000000001e-06 2024-09-11T06:50:54.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.365 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.365 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154428+0000", 2024-09-11T06:50:54.365 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2099999999999999e-05 2024-09-11T06:50:54.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.365 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.365 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154438+0000", 2024-09-11T06:50:54.366 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2590000000000006e-06 2024-09-11T06:50:54.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.366 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.366 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.155643+0000", 2024-09-11T06:50:54.366 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012048689999999999 2024-09-11T06:50:54.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.366 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.366 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.155838+0000", 2024-09-11T06:50:54.366 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019531299999999999 2024-09-11T06:50:54.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.367 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.367 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.156656+0000", 2024-09-11T06:50:54.367 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00081824700000000003 2024-09-11T06:50:54.367 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.367 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.367 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.367 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.367 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:587 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-11T06:50:54.367 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:50:54.367 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 32.121610644999997, 2024-09-11T06:50:54.367 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0023436289999999999, 2024-09-11T06:50:54.367 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.368 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.368 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:54.368 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.368 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:50:54.368 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:54.368 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 587 2024-09-11T06:50:54.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.368 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.368 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.368 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.368 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:50:54.368 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.369 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.369 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:50:54.369 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.369 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.369 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229252+0000", 2024-09-11T06:50:54.369 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3799999999999999e-06 2024-09-11T06:50:54.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.369 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.369 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229259+0000", 2024-09-11T06:50:54.370 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4760000000000003e-06 2024-09-11T06:50:54.370 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.370 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.370 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229271+0000", 2024-09-11T06:50:54.370 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2313000000000001e-05 2024-09-11T06:50:54.370 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.370 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.370 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229279+0000", 2024-09-11T06:50:54.370 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1610000000000003e-06 2024-09-11T06:50:54.370 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.371 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.371 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229344+0000", 2024-09-11T06:50:54.371 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4375000000000006e-05 2024-09-11T06:50:54.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.371 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.371 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229404+0000", 2024-09-11T06:50:54.371 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0517999999999999e-05 2024-09-11T06:50:54.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.371 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:54.371 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229516+0000", 2024-09-11T06:50:54.371 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011129900000000001 2024-09-11T06:50:54.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.372 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:54.372 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231502+0000", 2024-09-11T06:50:54.372 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001986238 2024-09-11T06:50:54.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.372 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:54.372 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231537+0000", 2024-09-11T06:50:54.372 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4968000000000001e-05 2024-09-11T06:50:54.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.372 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:54.372 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231569+0000", 2024-09-11T06:50:54.372 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2459000000000002e-05 2024-09-11T06:50:54.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231595+0000", 2024-09-11T06:50:54.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5442000000000001e-05 2024-09-11T06:50:54.373 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.373 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.373 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.373 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:15 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:54.373 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.709418+0000", 2024-09-11T06:50:54.374 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.641443492000001, 2024-09-11T06:50:54.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027504280000000001, 2024-09-11T06:50:54.374 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.374 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.374 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:54.374 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.374 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:54.374 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:54.374 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 15 2024-09-11T06:50:54.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.374 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.374 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709418+0000", 2024-09-11T06:50:54.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.375 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.375 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709418+0000", 2024-09-11T06:50:54.375 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.375 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.375 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709420+0000", 2024-09-11T06:50:54.375 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2299999999999998e-06 2024-09-11T06:50:54.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.375 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.376 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709432+0000", 2024-09-11T06:50:54.376 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1897e-05 2024-09-11T06:50:54.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.376 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.376 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709464+0000", 2024-09-11T06:50:54.376 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1819000000000001e-05 2024-09-11T06:50:54.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.376 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.376 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709479+0000", 2024-09-11T06:50:54.376 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4853e-05 2024-09-11T06:50:54.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.377 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.377 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.377 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709624+0000", 2024-09-11T06:50:54.377 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144912 2024-09-11T06:50:54.377 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.377 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.377 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.377 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709773+0000", 2024-09-11T06:50:54.377 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000149529 2024-09-11T06:50:54.377 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.377 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.377 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:54.377 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.710055+0000", 2024-09-11T06:50:54.377 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000281692 2024-09-11T06:50:54.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.378 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:54.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.711412+0000", 2024-09-11T06:50:54.378 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001356931 2024-09-11T06:50:54.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.378 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:54.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.711890+0000", 2024-09-11T06:50:54.378 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00047768700000000002 2024-09-11T06:50:54.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.378 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:54.379 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.712050+0000", 2024-09-11T06:50:54.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016041499999999999 2024-09-11T06:50:54.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.379 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.379 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.712169+0000", 2024-09-11T06:50:54.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011846299999999999 2024-09-11T06:50:54.379 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.379 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.379 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.379 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:18 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:54.380 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.713502+0000", 2024-09-11T06:50:54.380 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.637360065999999, 2024-09-11T06:50:54.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032323069999999998, 2024-09-11T06:50:54.380 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.380 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.380 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:54.380 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.380 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:54.380 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:54.380 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18 2024-09-11T06:50:54.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.380 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.381 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.381 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713502+0000", 2024-09-11T06:50:54.381 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.381 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.381 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713502+0000", 2024-09-11T06:50:54.381 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.381 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.381 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713503+0000", 2024-09-11T06:50:54.381 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8199999999999999e-06 2024-09-11T06:50:54.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.382 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.382 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713513+0000", 2024-09-11T06:50:54.382 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8539999999999992e-06 2024-09-11T06:50:54.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.382 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.382 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713540+0000", 2024-09-11T06:50:54.382 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6594999999999999e-05 2024-09-11T06:50:54.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.382 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.382 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713553+0000", 2024-09-11T06:50:54.382 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2843999999999999e-05 2024-09-11T06:50:54.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.383 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.383 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713686+0000", 2024-09-11T06:50:54.383 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013294899999999999 2024-09-11T06:50:54.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.383 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.383 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713937+0000", 2024-09-11T06:50:54.383 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025110600000000002 2024-09-11T06:50:54.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.383 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:54.383 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.714412+0000", 2024-09-11T06:50:54.384 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00047562 2024-09-11T06:50:54.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.384 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:54.384 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.715867+0000", 2024-09-11T06:50:54.384 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001455004 2024-09-11T06:50:54.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.384 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:54.384 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.716569+0000", 2024-09-11T06:50:54.384 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070159400000000005 2024-09-11T06:50:54.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.385 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:54.385 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.716660+0000", 2024-09-11T06:50:54.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1403000000000006e-05 2024-09-11T06:50:54.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.385 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.385 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.716734+0000", 2024-09-11T06:50:54.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3517999999999996e-05 2024-09-11T06:50:54.385 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.385 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.385 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.385 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:23 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:54.386 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.723392+0000", 2024-09-11T06:50:54.386 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.627469470000001, 2024-09-11T06:50:54.386 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027098489999999999, 2024-09-11T06:50:54.386 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.386 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.386 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:54.386 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.386 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:54.386 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:54.386 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23 2024-09-11T06:50:54.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.386 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.386 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.387 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723392+0000", 2024-09-11T06:50:54.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.387 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.387 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723392+0000", 2024-09-11T06:50:54.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.387 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.387 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723394+0000", 2024-09-11T06:50:54.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0669999999999999e-06 2024-09-11T06:50:54.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.388 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.388 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723406+0000", 2024-09-11T06:50:54.388 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1367e-05 2024-09-11T06:50:54.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.388 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.388 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723436+0000", 2024-09-11T06:50:54.388 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0874999999999998e-05 2024-09-11T06:50:54.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.388 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.388 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723451+0000", 2024-09-11T06:50:54.388 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4192999999999999e-05 2024-09-11T06:50:54.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.389 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.389 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723577+0000", 2024-09-11T06:50:54.389 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000126267 2024-09-11T06:50:54.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.389 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.389 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723718+0000", 2024-09-11T06:50:54.389 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000141206 2024-09-11T06:50:54.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.389 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:54.389 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723982+0000", 2024-09-11T06:50:54.390 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026338499999999999 2024-09-11T06:50:54.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.390 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:54.390 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.725377+0000", 2024-09-11T06:50:54.390 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013956999999999999 2024-09-11T06:50:54.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.390 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:54.390 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.725908+0000", 2024-09-11T06:50:54.390 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00053036200000000004 2024-09-11T06:50:54.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.390 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:54.391 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.725999+0000", 2024-09-11T06:50:54.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1016999999999999e-05 2024-09-11T06:50:54.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.391 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.391 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726102+0000", 2024-09-11T06:50:54.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010341 2024-09-11T06:50:54.391 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.391 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.391 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.391 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:15 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:54.391 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.868921+0000", 2024-09-11T06:50:54.392 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.481940429000002, 2024-09-11T06:50:54.392 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0026879, 2024-09-11T06:50:54.392 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.392 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.392 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:54.392 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.392 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:54.392 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:54.392 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 15 2024-09-11T06:50:54.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.392 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.392 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.393 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868921+0000", 2024-09-11T06:50:54.393 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.393 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.393 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868921+0000", 2024-09-11T06:50:54.393 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.393 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.393 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868923+0000", 2024-09-11T06:50:54.393 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8959999999999999e-06 2024-09-11T06:50:54.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.394 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.394 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868934+0000", 2024-09-11T06:50:54.394 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1083e-05 2024-09-11T06:50:54.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.394 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.394 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868965+0000", 2024-09-11T06:50:54.394 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0833000000000001e-05 2024-09-11T06:50:54.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.394 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.394 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868990+0000", 2024-09-11T06:50:54.394 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4545999999999998e-05 2024-09-11T06:50:54.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.869124+0000", 2024-09-11T06:50:54.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013489599999999999 2024-09-11T06:50:54.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.869270+0000", 2024-09-11T06:50:54.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000145976 2024-09-11T06:50:54.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:54.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.869538+0000", 2024-09-11T06:50:54.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026780599999999999 2024-09-11T06:50:54.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.396 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:54.396 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.870691+0000", 2024-09-11T06:50:54.396 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00115277 2024-09-11T06:50:54.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.396 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:54.396 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.871456+0000", 2024-09-11T06:50:54.396 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00076473800000000005 2024-09-11T06:50:54.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.396 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:54.396 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.871543+0000", 2024-09-11T06:50:54.397 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7040999999999996e-05 2024-09-11T06:50:54.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.397 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.397 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.397 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.871609+0000", 2024-09-11T06:50:54.397 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6315000000000005e-05 2024-09-11T06:50:54.397 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.397 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.397 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.397 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.397 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:20 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:54.397 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.877429+0000", 2024-09-11T06:50:54.397 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.47343214, 2024-09-11T06:50:54.397 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032382629999999999, 2024-09-11T06:50:54.398 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.398 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.398 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:54.398 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.398 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:54.398 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:54.398 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 20 2024-09-11T06:50:54.398 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.398 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.398 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.398 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.398 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877429+0000", 2024-09-11T06:50:54.398 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.399 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.399 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877429+0000", 2024-09-11T06:50:54.399 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.399 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.399 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877431+0000", 2024-09-11T06:50:54.399 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9429999999999999e-06 2024-09-11T06:50:54.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.399 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.399 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877444+0000", 2024-09-11T06:50:54.400 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2396e-05 2024-09-11T06:50:54.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.400 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.400 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.400 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877474+0000", 2024-09-11T06:50:54.400 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0239999999999998e-05 2024-09-11T06:50:54.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.400 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.400 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.400 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877496+0000", 2024-09-11T06:50:54.400 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1914999999999999e-05 2024-09-11T06:50:54.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.400 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.400 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.400 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877616+0000", 2024-09-11T06:50:54.401 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000119814 2024-09-11T06:50:54.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.401 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.401 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877836+0000", 2024-09-11T06:50:54.401 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022029600000000001 2024-09-11T06:50:54.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.401 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:54.401 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.878482+0000", 2024-09-11T06:50:54.401 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00064605200000000004 2024-09-11T06:50:54.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.401 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:54.402 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.879804+0000", 2024-09-11T06:50:54.402 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013216460000000001 2024-09-11T06:50:54.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.402 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.402 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:54.402 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.880519+0000", 2024-09-11T06:50:54.402 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00071547600000000005 2024-09-11T06:50:54.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.402 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.402 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:54.402 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.880606+0000", 2024-09-11T06:50:54.402 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6907e-05 2024-09-11T06:50:54.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.402 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.402 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.403 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.880668+0000", 2024-09-11T06:50:54.403 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1577999999999999e-05 2024-09-11T06:50:54.403 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.403 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.403 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.403 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.403 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:23 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:54.403 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.885067+0000", 2024-09-11T06:50:54.403 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.465794486, 2024-09-11T06:50:54.403 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028127320000000001, 2024-09-11T06:50:54.403 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.403 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.404 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:54.404 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.404 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:54.404 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:54.404 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23 2024-09-11T06:50:54.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.404 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.404 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.404 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.404 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885067+0000", 2024-09-11T06:50:54.404 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.404 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.405 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.405 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885067+0000", 2024-09-11T06:50:54.405 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.405 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.405 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885069+0000", 2024-09-11T06:50:54.405 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2000000000000001e-06 2024-09-11T06:50:54.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.405 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.405 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885081+0000", 2024-09-11T06:50:54.405 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1209999999999999e-05 2024-09-11T06:50:54.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.406 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.406 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885111+0000", 2024-09-11T06:50:54.406 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0318999999999999e-05 2024-09-11T06:50:54.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.406 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.406 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885132+0000", 2024-09-11T06:50:54.406 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0649000000000002e-05 2024-09-11T06:50:54.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.406 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.406 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885243+0000", 2024-09-11T06:50:54.407 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000111284 2024-09-11T06:50:54.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.407 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.407 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885387+0000", 2024-09-11T06:50:54.407 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144222 2024-09-11T06:50:54.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.407 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:54.407 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885699+0000", 2024-09-11T06:50:54.407 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00031239800000000003 2024-09-11T06:50:54.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.407 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:54.408 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.886817+0000", 2024-09-11T06:50:54.408 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001117191 2024-09-11T06:50:54.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.408 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.408 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:54.408 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.887684+0000", 2024-09-11T06:50:54.408 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00086699499999999996 2024-09-11T06:50:54.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.408 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.408 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:54.408 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.887772+0000", 2024-09-11T06:50:54.408 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8224000000000001e-05 2024-09-11T06:50:54.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.408 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.409 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.409 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.887880+0000", 2024-09-11T06:50:54.409 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010804 2024-09-11T06:50:54.409 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.409 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.409 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.409 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.409 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.409 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "MOSDPGPull(2.0 e70/69 cost 1048576)", 2024-09-11T06:50:54.409 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:35.528435+0000", 2024-09-11T06:50:54.409 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.822426749999998, 2024-09-11T06:50:54.409 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027390520000000001, 2024-09-11T06:50:54.409 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.409 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.410 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-11T06:50:54.410 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.410 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.410 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.410 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528435+0000", 2024-09-11T06:50:54.410 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.410 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.410 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.410 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528435+0000", 2024-09-11T06:50:54.410 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.410 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.410 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.411 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528437+0000", 2024-09-11T06:50:54.411 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5959999999999998e-06 2024-09-11T06:50:54.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.411 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.411 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528465+0000", 2024-09-11T06:50:54.411 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7436e-05 2024-09-11T06:50:54.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.411 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.411 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528499+0000", 2024-09-11T06:50:54.411 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3599000000000001e-05 2024-09-11T06:50:54.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.412 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.412 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528515+0000", 2024-09-11T06:50:54.412 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.677e-05 2024-09-11T06:50:54.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.412 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.412 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528652+0000", 2024-09-11T06:50:54.412 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000136546 2024-09-11T06:50:54.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.412 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.412 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.531174+0000", 2024-09-11T06:50:54.413 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0025221050000000002 2024-09-11T06:50:54.413 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.413 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.413 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.413 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "MOSDPGPull(2.0 e70/69 cost 1048576)", 2024-09-11T06:50:54.413 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:35.607413+0000", 2024-09-11T06:50:54.413 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.743448596, 2024-09-11T06:50:54.413 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0021226639999999998, 2024-09-11T06:50:54.413 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.413 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.413 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-11T06:50:54.413 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.414 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.414 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.607413+0000", 2024-09-11T06:50:54.414 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.414 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.414 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.607413+0000", 2024-09-11T06:50:54.414 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.414 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.414 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.607415+0000", 2024-09-11T06:50:54.414 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7069999999999999e-06 2024-09-11T06:50:54.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.415 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.415 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.415 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.607436+0000", 2024-09-11T06:50:54.415 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1151000000000001e-05 2024-09-11T06:50:54.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.415 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.415 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.415 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.607456+0000", 2024-09-11T06:50:54.415 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0559000000000001e-05 2024-09-11T06:50:54.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.415 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.415 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.416 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.607466+0000", 2024-09-11T06:50:54.416 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3419999999999997e-06 2024-09-11T06:50:54.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.416 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.416 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.607646+0000", 2024-09-11T06:50:54.416 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017984000000000001 2024-09-11T06:50:54.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.416 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.416 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.609536+0000", 2024-09-11T06:50:54.416 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001890065 2024-09-11T06:50:54.416 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.417 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.417 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.417 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4828.0:15 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-11T06:50:54.417 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:36.253730+0000", 2024-09-11T06:50:54.417 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.097131406999999, 2024-09-11T06:50:54.417 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0025257589999999998, 2024-09-11T06:50:54.417 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.417 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.417 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:54.417 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.417 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4828", 2024-09-11T06:50:54.417 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2410288650", 2024-09-11T06:50:54.418 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 15 2024-09-11T06:50:54.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.418 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.418 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.418 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253730+0000", 2024-09-11T06:50:54.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.418 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.418 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253730+0000", 2024-09-11T06:50:54.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.419 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.419 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253732+0000", 2024-09-11T06:50:54.419 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2400000000000002e-06 2024-09-11T06:50:54.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.419 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.419 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253744+0000", 2024-09-11T06:50:54.419 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1316e-05 2024-09-11T06:50:54.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.419 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.419 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253771+0000", 2024-09-11T06:50:54.419 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.724e-05 2024-09-11T06:50:54.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.420 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.420 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253785+0000", 2024-09-11T06:50:54.420 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3786e-05 2024-09-11T06:50:54.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.420 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.420 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253886+0000", 2024-09-11T06:50:54.420 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000100981 2024-09-11T06:50:54.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.420 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.420 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.254029+0000", 2024-09-11T06:50:54.421 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014311900000000001 2024-09-11T06:50:54.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.421 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.421 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:54.421 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.254317+0000", 2024-09-11T06:50:54.421 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00028809300000000001 2024-09-11T06:50:54.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.421 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.421 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:54.421 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.255403+0000", 2024-09-11T06:50:54.421 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001085639 2024-09-11T06:50:54.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.421 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.421 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:54.422 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.256107+0000", 2024-09-11T06:50:54.422 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070468799999999997 2024-09-11T06:50:54.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.422 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:54.422 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.256191+0000", 2024-09-11T06:50:54.422 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3714999999999994e-05 2024-09-11T06:50:54.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.422 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.422 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.256256+0000", 2024-09-11T06:50:54.422 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4942000000000003e-05 2024-09-11T06:50:54.422 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.423 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.423 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.423 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5313.0:26 2.6 2:7b712441:ns::rbd_object_map.11adc335555f:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:50:54.423 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.260230+0000", 2024-09-11T06:50:54.423 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.090631998999999, 2024-09-11T06:50:54.423 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002457201, 2024-09-11T06:50:54.423 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.423 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.423 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:54.423 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.423 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5313", 2024-09-11T06:50:54.424 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/624314229", 2024-09-11T06:50:54.424 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:50:54.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.424 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.424 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.424 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260230+0000", 2024-09-11T06:50:54.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.424 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.424 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260230+0000", 2024-09-11T06:50:54.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.425 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.425 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260232+0000", 2024-09-11T06:50:54.425 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.956e-06 2024-09-11T06:50:54.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.425 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.425 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260243+0000", 2024-09-11T06:50:54.425 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1687e-05 2024-09-11T06:50:54.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.425 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.425 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260269+0000", 2024-09-11T06:50:54.426 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6211999999999999e-05 2024-09-11T06:50:54.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.426 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.426 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.426 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260284+0000", 2024-09-11T06:50:54.426 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4053000000000001e-05 2024-09-11T06:50:54.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.426 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.426 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.426 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260391+0000", 2024-09-11T06:50:54.426 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000107278 2024-09-11T06:50:54.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.426 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.427 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.427 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260541+0000", 2024-09-11T06:50:54.427 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015033000000000001 2024-09-11T06:50:54.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.427 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 3", 2024-09-11T06:50:54.427 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260951+0000", 2024-09-11T06:50:54.427 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040981299999999997 2024-09-11T06:50:54.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.427 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:54.427 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262399+0000", 2024-09-11T06:50:54.428 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014475930000000001 2024-09-11T06:50:54.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.428 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:54.428 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262531+0000", 2024-09-11T06:50:54.428 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013239 2024-09-11T06:50:54.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.428 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:54.428 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262623+0000", 2024-09-11T06:50:54.428 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2361000000000006e-05 2024-09-11T06:50:54.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.429 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.429 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262687+0000", 2024-09-11T06:50:54.429 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3528000000000006e-05 2024-09-11T06:50:54.429 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.429 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.429 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.429 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5313.0:31 2.6 2:7b712441:ns::rbd_object_map.11adc335555f:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:50:54.429 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.266495+0000", 2024-09-11T06:50:54.429 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.084366225, 2024-09-11T06:50:54.429 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0026763870000000001, 2024-09-11T06:50:54.429 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.430 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.430 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:54.430 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.430 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5313", 2024-09-11T06:50:54.430 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/624314229", 2024-09-11T06:50:54.430 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 31 2024-09-11T06:50:54.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.430 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.430 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.430 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266495+0000", 2024-09-11T06:50:54.430 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.430 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.431 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266495+0000", 2024-09-11T06:50:54.431 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.431 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.431 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266498+0000", 2024-09-11T06:50:54.431 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2369999999999999e-06 2024-09-11T06:50:54.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.431 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.431 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266509+0000", 2024-09-11T06:50:54.431 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1459e-05 2024-09-11T06:50:54.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.432 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.432 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266540+0000", 2024-09-11T06:50:54.432 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1189999999999998e-05 2024-09-11T06:50:54.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.432 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.432 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266563+0000", 2024-09-11T06:50:54.432 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2529000000000001e-05 2024-09-11T06:50:54.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.432 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.432 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266708+0000", 2024-09-11T06:50:54.432 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014511 2024-09-11T06:50:54.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.433 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.433 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.433 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266962+0000", 2024-09-11T06:50:54.433 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000253897 2024-09-11T06:50:54.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.433 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.433 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 3", 2024-09-11T06:50:54.433 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.267430+0000", 2024-09-11T06:50:54.433 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00046799899999999998 2024-09-11T06:50:54.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.433 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.433 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:54.434 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.268765+0000", 2024-09-11T06:50:54.434 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013350359999999999 2024-09-11T06:50:54.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.434 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:54.434 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.269019+0000", 2024-09-11T06:50:54.434 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025397000000000001 2024-09-11T06:50:54.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.434 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:54.434 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.269105+0000", 2024-09-11T06:50:54.434 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5952000000000006e-05 2024-09-11T06:50:54.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.435 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.435 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.435 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.269172+0000", 2024-09-11T06:50:54.435 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7008000000000006e-05 2024-09-11T06:50:54.435 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.435 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.435 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.435 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.435 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:5 3.4 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:50:54.435 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.588582+0000", 2024-09-11T06:50:54.435 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.7622800719999994, 2024-09-11T06:50:54.435 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002889857, 2024-09-11T06:50:54.435 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.436 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.436 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:54.436 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.436 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:50:54.436 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:50:54.436 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 5 2024-09-11T06:50:54.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.436 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.436 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.436 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.436 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588582+0000", 2024-09-11T06:50:54.436 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.436 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.437 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.437 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588582+0000", 2024-09-11T06:50:54.437 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.437 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.437 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588583+0000", 2024-09-11T06:50:54.437 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.223e-06 2024-09-11T06:50:54.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.437 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.437 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588588+0000", 2024-09-11T06:50:54.437 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4299999999999997e-06 2024-09-11T06:50:54.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.438 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.438 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.438 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588606+0000", 2024-09-11T06:50:54.438 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7652999999999999e-05 2024-09-11T06:50:54.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.438 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.438 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.438 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588615+0000", 2024-09-11T06:50:54.438 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4399999999999994e-06 2024-09-11T06:50:54.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.438 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.438 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.438 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588725+0000", 2024-09-11T06:50:54.438 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010945799999999999 2024-09-11T06:50:54.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.439 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.439 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.439 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.589031+0000", 2024-09-11T06:50:54.439 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030590000000000001 2024-09-11T06:50:54.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.439 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.439 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-11T06:50:54.439 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.589521+0000", 2024-09-11T06:50:54.439 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00049061299999999999 2024-09-11T06:50:54.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.439 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.439 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:54.439 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.590792+0000", 2024-09-11T06:50:54.439 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012708039999999999 2024-09-11T06:50:54.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.440 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:54.440 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591339+0000", 2024-09-11T06:50:54.440 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00054710499999999999 2024-09-11T06:50:54.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.440 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:54.440 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591412+0000", 2024-09-11T06:50:54.440 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2811999999999993e-05 2024-09-11T06:50:54.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.440 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.441 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591471+0000", 2024-09-11T06:50:54.441 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9419000000000001e-05 2024-09-11T06:50:54.441 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.441 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.441 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.441 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.441 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:8 3.4 3:3f0de396:::rbd_header.1415bf940188:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:50:54.441 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.596120+0000", 2024-09-11T06:50:54.441 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.7547411880000006, 2024-09-11T06:50:54.441 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027911020000000002, 2024-09-11T06:50:54.441 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.441 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.442 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:54.442 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.442 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:50:54.442 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:50:54.442 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 8 2024-09-11T06:50:54.442 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.442 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.442 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.442 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.442 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596120+0000", 2024-09-11T06:50:54.442 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.442 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.442 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.442 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.443 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596120+0000", 2024-09-11T06:50:54.443 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.443 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.443 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.443 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596122+0000", 2024-09-11T06:50:54.443 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.513e-06 2024-09-11T06:50:54.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.443 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.443 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.443 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596130+0000", 2024-09-11T06:50:54.443 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4999999999999999e-06 2024-09-11T06:50:54.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.443 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.444 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.444 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596152+0000", 2024-09-11T06:50:54.444 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1846000000000001e-05 2024-09-11T06:50:54.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.444 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.444 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596163+0000", 2024-09-11T06:50:54.444 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1029999999999999e-05 2024-09-11T06:50:54.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.444 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.444 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596242+0000", 2024-09-11T06:50:54.444 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8275000000000005e-05 2024-09-11T06:50:54.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.445 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.445 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596439+0000", 2024-09-11T06:50:54.445 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000196999 2024-09-11T06:50:54.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.445 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-11T06:50:54.445 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596755+0000", 2024-09-11T06:50:54.445 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00031626100000000003 2024-09-11T06:50:54.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.445 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:54.446 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.598043+0000", 2024-09-11T06:50:54.446 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001288389 2024-09-11T06:50:54.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.446 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.446 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:54.446 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.598774+0000", 2024-09-11T06:50:54.446 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00073080499999999995 2024-09-11T06:50:54.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.446 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.446 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:54.446 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.598855+0000", 2024-09-11T06:50:54.446 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0615e-05 2024-09-11T06:50:54.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.446 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.447 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.447 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.598912+0000", 2024-09-11T06:50:54.447 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6869e-05 2024-09-11T06:50:54.447 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.447 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.447 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.447 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.447 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.447 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:9 3.0 3:00834abc:::rbd_object_map.1415bf940188:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:50:54.447 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.599161+0000", 2024-09-11T06:50:54.447 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.7517001709999995, 2024-09-11T06:50:54.447 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002826051, 2024-09-11T06:50:54.447 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.447 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.448 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:54.448 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.448 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:50:54.448 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:50:54.448 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 9 2024-09-11T06:50:54.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.448 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.448 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.448 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.448 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599161+0000", 2024-09-11T06:50:54.448 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.448 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.448 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.449 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599161+0000", 2024-09-11T06:50:54.449 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.449 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.449 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599163+0000", 2024-09-11T06:50:54.449 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8700000000000001e-06 2024-09-11T06:50:54.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.449 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.449 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599174+0000", 2024-09-11T06:50:54.449 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0416e-05 2024-09-11T06:50:54.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.450 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.450 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599202+0000", 2024-09-11T06:50:54.450 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7946e-05 2024-09-11T06:50:54.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.450 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.450 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599216+0000", 2024-09-11T06:50:54.450 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3934000000000001e-05 2024-09-11T06:50:54.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.450 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.450 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599345+0000", 2024-09-11T06:50:54.450 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012954000000000001 2024-09-11T06:50:54.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.451 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.451 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.451 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599540+0000", 2024-09-11T06:50:54.451 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019516299999999999 2024-09-11T06:50:54.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.451 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.451 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-11T06:50:54.451 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599796+0000", 2024-09-11T06:50:54.451 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025556500000000002 2024-09-11T06:50:54.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.451 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.451 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:54.452 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.601071+0000", 2024-09-11T06:50:54.452 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001275396 2024-09-11T06:50:54.452 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.452 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.452 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:54.452 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.601852+0000", 2024-09-11T06:50:54.452 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000780524 2024-09-11T06:50:54.452 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.452 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.452 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:54.452 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.601925+0000", 2024-09-11T06:50:54.452 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3574999999999999e-05 2024-09-11T06:50:54.452 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.452 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.453 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.453 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.601988+0000", 2024-09-11T06:50:54.453 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2121999999999999e-05 2024-09-11T06:50:54.453 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.453 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.453 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.453 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.453 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5038.0:12 3.4 3:3f0de396:::rbd_header.1415bf940188:head [watch watch cookie 140693532068064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:50:54.453 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.718024+0000", 2024-09-11T06:50:54.453 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.6328372330000001, 2024-09-11T06:50:54.453 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002724583, 2024-09-11T06:50:54.453 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.453 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.454 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:54.454 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.454 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5038", 2024-09-11T06:50:54.454 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/945190275", 2024-09-11T06:50:54.454 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:50:54.454 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.454 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.454 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.454 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.454 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718024+0000", 2024-09-11T06:50:54.454 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.454 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.454 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.454 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.455 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718024+0000", 2024-09-11T06:50:54.455 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.455 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.455 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718027+0000", 2024-09-11T06:50:54.455 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1430000000000001e-06 2024-09-11T06:50:54.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.455 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.455 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718037+0000", 2024-09-11T06:50:54.455 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0253e-05 2024-09-11T06:50:54.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.456 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.456 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.456 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718068+0000", 2024-09-11T06:50:54.456 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1050000000000003e-05 2024-09-11T06:50:54.456 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.456 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.456 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.456 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718086+0000", 2024-09-11T06:50:54.456 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8629000000000001e-05 2024-09-11T06:50:54.456 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.456 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.456 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.456 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718206+0000", 2024-09-11T06:50:54.456 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000119358 2024-09-11T06:50:54.457 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.457 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.457 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.457 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718425+0000", 2024-09-11T06:50:54.457 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000218975 2024-09-11T06:50:54.457 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.457 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.457 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-11T06:50:54.457 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718830+0000", 2024-09-11T06:50:54.457 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040538199999999998 2024-09-11T06:50:54.457 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.457 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.457 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:54.457 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.719849+0000", 2024-09-11T06:50:54.458 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010187589999999999 2024-09-11T06:50:54.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.458 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.458 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:54.458 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.720548+0000", 2024-09-11T06:50:54.458 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000699112 2024-09-11T06:50:54.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.458 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.458 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:54.458 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.720632+0000", 2024-09-11T06:50:54.458 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4314999999999995e-05 2024-09-11T06:50:54.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.458 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.459 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.459 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.720749+0000", 2024-09-11T06:50:54.459 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116607 2024-09-11T06:50:54.459 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.459 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.459 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.459 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.459 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:47 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:50:54.459 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.553228+0000", 2024-09-11T06:50:54.459 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.7976331429999997, 2024-09-11T06:50:54.459 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040683100000000003, 2024-09-11T06:50:54.459 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.459 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.460 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:54.460 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.460 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:50:54.460 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:50:54.460 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 47 2024-09-11T06:50:54.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.460 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.460 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.460 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.460 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553228+0000", 2024-09-11T06:50:54.460 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.460 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.460 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.461 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553228+0000", 2024-09-11T06:50:54.461 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.461 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.461 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553231+0000", 2024-09-11T06:50:54.461 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3130000000000001e-06 2024-09-11T06:50:54.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.461 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.461 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553242+0000", 2024-09-11T06:50:54.461 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1345999999999999e-05 2024-09-11T06:50:54.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553272+0000", 2024-09-11T06:50:54.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0179999999999999e-05 2024-09-11T06:50:54.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553287+0000", 2024-09-11T06:50:54.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4283e-05 2024-09-11T06:50:54.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553415+0000", 2024-09-11T06:50:54.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000128491 2024-09-11T06:50:54.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553651+0000", 2024-09-11T06:50:54.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000236328 2024-09-11T06:50:54.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:54.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.554356+0000", 2024-09-11T06:50:54.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070436299999999995 2024-09-11T06:50:54.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:54.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.556064+0000", 2024-09-11T06:50:54.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0017084439999999999 2024-09-11T06:50:54.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:54.464 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.556993+0000", 2024-09-11T06:50:54.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00092912400000000003 2024-09-11T06:50:54.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:54.464 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.557173+0000", 2024-09-11T06:50:54.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017940400000000001 2024-09-11T06:50:54.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.464 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.557297+0000", 2024-09-11T06:50:54.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000124034 2024-09-11T06:50:54.465 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.465 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.465 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.465 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:66 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:50:54.465 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.589465+0000", 2024-09-11T06:50:54.465 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.761396338, 2024-09-11T06:50:54.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039855489999999997, 2024-09-11T06:50:54.465 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.465 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.465 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:54.465 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.466 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:50:54.466 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:50:54.466 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 66 2024-09-11T06:50:54.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.466 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.466 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.466 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589465+0000", 2024-09-11T06:50:54.466 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589465+0000", 2024-09-11T06:50:54.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589468+0000", 2024-09-11T06:50:54.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3539999999999998e-06 2024-09-11T06:50:54.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589480+0000", 2024-09-11T06:50:54.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1946e-05 2024-09-11T06:50:54.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.469 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589512+0000", 2024-09-11T06:50:54.469 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2712999999999997e-05 2024-09-11T06:50:54.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.469 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589527+0000", 2024-09-11T06:50:54.469 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4686e-05 2024-09-11T06:50:54.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.469 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589673+0000", 2024-09-11T06:50:54.469 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014630700000000001 2024-09-11T06:50:54.470 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.470 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.470 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.470 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589936+0000", 2024-09-11T06:50:54.470 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026234499999999998 2024-09-11T06:50:54.470 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.470 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.470 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:54.470 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.591001+0000", 2024-09-11T06:50:54.470 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001065023 2024-09-11T06:50:54.470 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.470 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.470 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:54.471 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.592757+0000", 2024-09-11T06:50:54.471 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0017560849999999999 2024-09-11T06:50:54.471 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.471 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.471 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:54.471 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.593297+0000", 2024-09-11T06:50:54.471 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00054007600000000001 2024-09-11T06:50:54.471 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.471 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.471 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:54.471 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.593385+0000", 2024-09-11T06:50:54.471 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8327999999999999e-05 2024-09-11T06:50:54.471 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.471 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.471 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.472 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.593451+0000", 2024-09-11T06:50:54.472 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5685999999999994e-05 2024-09-11T06:50:54.472 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.472 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.472 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.472 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.472 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.472 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:54.474 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:50:54.474 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:54.471+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:54.575 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:54.575+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:54.600 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:54.600 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:54.600 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:50:54.600 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:54.614 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:50:54.676 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:54.675+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:54.737 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:54.737 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:54.737 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:50:54.737 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:50:54.738 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:54.751 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:50:54.797 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:50:54.794+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:50:54.870 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:50:54.872 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:54.873 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:50:54.873 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:50:54.873 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:50:54.873 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.873 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:20 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:54.873 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.717149+0000", 2024-09-11T06:50:54.873 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.151093337999999, 2024-09-11T06:50:54.873 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002913297, 2024-09-11T06:50:54.873 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.873 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.873 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:54.874 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.874 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:54.874 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:54.874 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 20 2024-09-11T06:50:54.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.874 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.874 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.874 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717149+0000", 2024-09-11T06:50:54.874 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.874 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.874 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717149+0000", 2024-09-11T06:50:54.874 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.875 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.875 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.875 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.875 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717152+0000", 2024-09-11T06:50:54.875 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.21e-06 2024-09-11T06:50:54.875 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.875 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.875 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.875 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717164+0000", 2024-09-11T06:50:54.875 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1912999999999999e-05 2024-09-11T06:50:54.875 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.875 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.875 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.876 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717191+0000", 2024-09-11T06:50:54.876 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7526e-05 2024-09-11T06:50:54.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.876 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.876 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717206+0000", 2024-09-11T06:50:54.876 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5113000000000001e-05 2024-09-11T06:50:54.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.876 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.876 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717355+0000", 2024-09-11T06:50:54.876 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014896500000000001 2024-09-11T06:50:54.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.877 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.877 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717607+0000", 2024-09-11T06:50:54.877 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000252017 2024-09-11T06:50:54.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.877 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:54.877 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.718128+0000", 2024-09-11T06:50:54.877 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00052114499999999998 2024-09-11T06:50:54.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.877 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:54.877 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.719432+0000", 2024-09-11T06:50:54.877 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013041629999999999 2024-09-11T06:50:54.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.878 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.878 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:54.878 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.719864+0000", 2024-09-11T06:50:54.878 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00043115299999999998 2024-09-11T06:50:54.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.878 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.878 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:54.878 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.719949+0000", 2024-09-11T06:50:54.878 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4862000000000002e-05 2024-09-11T06:50:54.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.878 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.879 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.879 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.720063+0000", 2024-09-11T06:50:54.879 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011423 2024-09-11T06:50:54.879 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.879 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.879 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.879 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.879 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:26 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 140250278007648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:54.879 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:50:54.879 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.138610202999999, 2024-09-11T06:50:54.879 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034645140000000001, 2024-09-11T06:50:54.879 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.880 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.880 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:54.880 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.880 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:54.880 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:54.880 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:50:54.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.880 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.880 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.880 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.880 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:50:54.880 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.881 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.881 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:50:54.881 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.881 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.881 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729634+0000", 2024-09-11T06:50:54.881 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0470000000000001e-06 2024-09-11T06:50:54.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.881 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.881 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729640+0000", 2024-09-11T06:50:54.881 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8860000000000001e-06 2024-09-11T06:50:54.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.882 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.882 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729654+0000", 2024-09-11T06:50:54.882 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:50:54.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.882 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.882 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729664+0000", 2024-09-11T06:50:54.882 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.594e-06 2024-09-11T06:50:54.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.882 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.882 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730422+0000", 2024-09-11T06:50:54.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000758648 2024-09-11T06:50:54.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.883 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.883 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730571+0000", 2024-09-11T06:50:54.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000148932 2024-09-11T06:50:54.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.883 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:54.883 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730936+0000", 2024-09-11T06:50:54.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036510900000000002 2024-09-11T06:50:54.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.883 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:54.884 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732159+0000", 2024-09-11T06:50:54.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001222302 2024-09-11T06:50:54.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.884 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:54.884 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732763+0000", 2024-09-11T06:50:54.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00060421599999999998 2024-09-11T06:50:54.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.884 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:54.884 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732916+0000", 2024-09-11T06:50:54.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015264 2024-09-11T06:50:54.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733097+0000", 2024-09-11T06:50:54.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000181504 2024-09-11T06:50:54.885 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.885 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.885 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.885 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:27 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:54.885 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:50:54.885 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.137636301000001, 2024-09-11T06:50:54.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003072852, 2024-09-11T06:50:54.885 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.885 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.886 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:54.886 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.886 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:54.886 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:54.886 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27 2024-09-11T06:50:54.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.886 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:50:54.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.887 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.887 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:50:54.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.887 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.887 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730608+0000", 2024-09-11T06:50:54.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8369999999999999e-06 2024-09-11T06:50:54.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.887 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.887 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730617+0000", 2024-09-11T06:50:54.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1859999999999992e-06 2024-09-11T06:50:54.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.888 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.888 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730636+0000", 2024-09-11T06:50:54.888 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9296e-05 2024-09-11T06:50:54.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.888 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.888 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730674+0000", 2024-09-11T06:50:54.888 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8609000000000003e-05 2024-09-11T06:50:54.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.888 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.889 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731496+0000", 2024-09-11T06:50:54.889 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00082112299999999999 2024-09-11T06:50:54.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.889 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:54.889 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731722+0000", 2024-09-11T06:50:54.889 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022651400000000001 2024-09-11T06:50:54.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.889 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.889 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733215+0000", 2024-09-11T06:50:54.889 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001493257 2024-09-11T06:50:54.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733452+0000", 2024-09-11T06:50:54.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023631600000000001 2024-09-11T06:50:54.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733679+0000", 2024-09-11T06:50:54.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022771400000000001 2024-09-11T06:50:54.890 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.890 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.890 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.891 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:73 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:54.891 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:50:54.891 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.137552017000001, 2024-09-11T06:50:54.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033454439999999999, 2024-09-11T06:50:54.891 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.891 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.891 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:54.891 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.891 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:54.891 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:54.891 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73 2024-09-11T06:50:54.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.891 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:50:54.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:50:54.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730694+0000", 2024-09-11T06:50:54.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5099999999999999e-06 2024-09-11T06:50:54.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.893 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.893 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730708+0000", 2024-09-11T06:50:54.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3237e-05 2024-09-11T06:50:54.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.893 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.893 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730738+0000", 2024-09-11T06:50:54.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0688999999999997e-05 2024-09-11T06:50:54.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.893 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.894 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730759+0000", 2024-09-11T06:50:54.894 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0818999999999998e-05 2024-09-11T06:50:54.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.894 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.894 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.894 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731841+0000", 2024-09-11T06:50:54.894 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010821819999999999 2024-09-11T06:50:54.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.894 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.894 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:54.894 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732092+0000", 2024-09-11T06:50:54.894 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025086299999999999 2024-09-11T06:50:54.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733730+0000", 2024-09-11T06:50:54.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0016374600000000001 2024-09-11T06:50:54.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733866+0000", 2024-09-11T06:50:54.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000136666 2024-09-11T06:50:54.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.734036+0000", 2024-09-11T06:50:54.896 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000170018 2024-09-11T06:50:54.896 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.896 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.896 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.897 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:18 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:54.897 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:50:54.897 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.994134828, 2024-09-11T06:50:54.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0031646579999999999, 2024-09-11T06:50:54.897 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.897 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.897 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:54.897 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.897 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:54.897 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:54.897 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18 2024-09-11T06:50:54.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.897 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.898 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.898 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:50:54.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.898 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.898 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:50:54.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.898 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.898 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874109+0000", 2024-09-11T06:50:54.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.426e-06 2024-09-11T06:50:54.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.899 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.899 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874118+0000", 2024-09-11T06:50:54.899 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6929999999999996e-06 2024-09-11T06:50:54.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.899 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.899 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874134+0000", 2024-09-11T06:50:54.899 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5503e-05 2024-09-11T06:50:54.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.899 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.899 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874144+0000", 2024-09-11T06:50:54.899 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0207000000000001e-05 2024-09-11T06:50:54.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874545+0000", 2024-09-11T06:50:54.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040120800000000001 2024-09-11T06:50:54.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874686+0000", 2024-09-11T06:50:54.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014128599999999999 2024-09-11T06:50:54.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:54.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875023+0000", 2024-09-11T06:50:54.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033706600000000001 2024-09-11T06:50:54.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:54.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.876217+0000", 2024-09-11T06:50:54.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011938999999999999 2024-09-11T06:50:54.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:54.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877042+0000", 2024-09-11T06:50:54.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00082499899999999998 2024-09-11T06:50:54.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.902 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:54.902 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877128+0000", 2024-09-11T06:50:54.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5903999999999996e-05 2024-09-11T06:50:54.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.902 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.902 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877273+0000", 2024-09-11T06:50:54.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144466 2024-09-11T06:50:54.902 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.902 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.902 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.903 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:24 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:54.903 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:50:54.903 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.979629121999999, 2024-09-11T06:50:54.903 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002980271, 2024-09-11T06:50:54.903 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.903 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.903 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:54.903 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.903 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:54.903 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:54.903 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 24 2024-09-11T06:50:54.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.903 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.904 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.904 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:50:54.904 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.904 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.904 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:50:54.904 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.904 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.904 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888616+0000", 2024-09-11T06:50:54.904 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0030000000000001e-06 2024-09-11T06:50:54.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.905 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.905 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888626+0000", 2024-09-11T06:50:54.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0502999999999999e-05 2024-09-11T06:50:54.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.905 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.905 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888655+0000", 2024-09-11T06:50:54.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8419999999999999e-05 2024-09-11T06:50:54.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.905 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.905 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888688+0000", 2024-09-11T06:50:54.906 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3212e-05 2024-09-11T06:50:54.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.906 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888829+0000", 2024-09-11T06:50:54.906 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014114299999999999 2024-09-11T06:50:54.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.906 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.889082+0000", 2024-09-11T06:50:54.906 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025279499999999999 2024-09-11T06:50:54.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.907 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:54.907 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.889577+0000", 2024-09-11T06:50:54.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00049536299999999997 2024-09-11T06:50:54.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.907 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:54.907 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.890811+0000", 2024-09-11T06:50:54.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001233858 2024-09-11T06:50:54.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.907 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:54.907 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891368+0000", 2024-09-11T06:50:54.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00055677400000000001 2024-09-11T06:50:54.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:54.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891456+0000", 2024-09-11T06:50:54.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8246999999999997e-05 2024-09-11T06:50:54.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891594+0000", 2024-09-11T06:50:54.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013795300000000001 2024-09-11T06:50:54.908 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.908 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.908 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.909 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:26 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 139740787512064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:54.909 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:50:54.909 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.975926478000002, 2024-09-11T06:50:54.909 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003613733, 2024-09-11T06:50:54.909 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.909 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.909 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:54.909 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.909 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:54.909 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:54.909 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:50:54.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.909 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.910 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.910 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:50:54.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.910 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.910 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:50:54.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.910 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.910 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892318+0000", 2024-09-11T06:50:54.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.623e-06 2024-09-11T06:50:54.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.911 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.911 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892325+0000", 2024-09-11T06:50:54.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:50:54.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.911 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.911 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892346+0000", 2024-09-11T06:50:54.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1246e-05 2024-09-11T06:50:54.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.911 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.912 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892362+0000", 2024-09-11T06:50:54.912 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5409000000000001e-05 2024-09-11T06:50:54.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.912 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.912 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.912 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892883+0000", 2024-09-11T06:50:54.912 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00052087599999999998 2024-09-11T06:50:54.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.912 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.912 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.912 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893123+0000", 2024-09-11T06:50:54.912 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000240769 2024-09-11T06:50:54.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:54.913 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893526+0000", 2024-09-11T06:50:54.913 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040300200000000002 2024-09-11T06:50:54.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:54.913 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894820+0000", 2024-09-11T06:50:54.913 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012934159999999999 2024-09-11T06:50:54.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:54.913 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895608+0000", 2024-09-11T06:50:54.913 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00078788100000000004 2024-09-11T06:50:54.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.914 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.914 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:54.914 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895724+0000", 2024-09-11T06:50:54.914 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011595999999999999 2024-09-11T06:50:54.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.914 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.914 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.914 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895930+0000", 2024-09-11T06:50:54.914 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000206391 2024-09-11T06:50:54.914 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.914 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.914 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.915 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:133 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:54.915 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:50:54.915 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.975128461000001, 2024-09-11T06:50:54.915 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035017360000000001, 2024-09-11T06:50:54.915 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.915 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.915 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:54.915 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.915 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:54.915 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:54.915 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 133 2024-09-11T06:50:54.915 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.916 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.916 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.916 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.916 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:50:54.916 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.916 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.916 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.916 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:50:54.916 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.916 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.916 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.917 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893116+0000", 2024-09-11T06:50:54.917 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1629999999999999e-06 2024-09-11T06:50:54.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.917 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.917 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.917 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893128+0000", 2024-09-11T06:50:54.917 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.11e-05 2024-09-11T06:50:54.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.917 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.917 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.917 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893156+0000", 2024-09-11T06:50:54.917 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8266e-05 2024-09-11T06:50:54.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.917 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.918 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.918 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893171+0000", 2024-09-11T06:50:54.918 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5426000000000001e-05 2024-09-11T06:50:54.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.918 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.918 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894561+0000", 2024-09-11T06:50:54.918 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013899959999999999 2024-09-11T06:50:54.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.918 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:54.918 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894781+0000", 2024-09-11T06:50:54.918 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021987100000000001 2024-09-11T06:50:54.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.919 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.919 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.919 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896349+0000", 2024-09-11T06:50:54.919 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015681779999999999 2024-09-11T06:50:54.919 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.919 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.919 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.919 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896458+0000", 2024-09-11T06:50:54.919 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000108187 2024-09-11T06:50:54.919 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.919 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.919 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.919 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896616+0000", 2024-09-11T06:50:54.919 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015854899999999999 2024-09-11T06:50:54.920 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.920 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.920 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.920 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.920 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.920 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:27 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:54.920 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:50:54.920 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.975117081, 2024-09-11T06:50:54.920 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003151989, 2024-09-11T06:50:54.920 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.920 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.920 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:54.920 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.921 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:54.921 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:54.921 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27 2024-09-11T06:50:54.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.921 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.921 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.921 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.921 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:50:54.921 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.921 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.921 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.921 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:50:54.921 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.922 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.922 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.922 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893128+0000", 2024-09-11T06:50:54.922 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.013e-06 2024-09-11T06:50:54.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.922 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.922 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.922 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893136+0000", 2024-09-11T06:50:54.922 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4190000000000002e-06 2024-09-11T06:50:54.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.922 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.922 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.923 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893158+0000", 2024-09-11T06:50:54.923 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1506999999999999e-05 2024-09-11T06:50:54.923 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.923 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.923 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.923 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893170+0000", 2024-09-11T06:50:54.923 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2741999999999999e-05 2024-09-11T06:50:54.923 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.923 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.923 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.923 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894255+0000", 2024-09-11T06:50:54.923 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001084453 2024-09-11T06:50:54.923 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.923 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.924 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:54.924 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894461+0000", 2024-09-11T06:50:54.924 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00020586700000000001 2024-09-11T06:50:54.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.924 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.924 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.924 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896033+0000", 2024-09-11T06:50:54.924 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001572092 2024-09-11T06:50:54.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.924 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.924 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.924 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896149+0000", 2024-09-11T06:50:54.925 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116636 2024-09-11T06:50:54.925 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.925 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.925 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.925 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896278+0000", 2024-09-11T06:50:54.925 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012826 2024-09-11T06:50:54.925 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.925 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.925 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.925 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.926 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.926 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:176 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch reconnect cookie 139735284585312 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:54.926 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:50:54.926 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.943750262000002, 2024-09-11T06:50:54.926 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033620960000000002, 2024-09-11T06:50:54.926 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.926 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.926 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:54.926 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.926 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:54.926 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:54.926 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 176 2024-09-11T06:50:54.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.927 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.927 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.927 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.927 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:50:54.927 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.927 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.927 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.927 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:50:54.927 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.927 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.927 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.927 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924494+0000", 2024-09-11T06:50:54.928 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.967e-06 2024-09-11T06:50:54.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.928 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.928 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924501+0000", 2024-09-11T06:50:54.928 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2659999999999998e-06 2024-09-11T06:50:54.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.928 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.928 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924520+0000", 2024-09-11T06:50:54.928 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9038999999999998e-05 2024-09-11T06:50:54.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.929 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.929 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924535+0000", 2024-09-11T06:50:54.929 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5359999999999999e-05 2024-09-11T06:50:54.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.929 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.929 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.925544+0000", 2024-09-11T06:50:54.929 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010085109999999999 2024-09-11T06:50:54.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.929 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.929 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.925671+0000", 2024-09-11T06:50:54.929 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012756599999999999 2024-09-11T06:50:54.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.930 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.930 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.927855+0000", 2024-09-11T06:50:54.930 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0021833870000000002 2024-09-11T06:50:54.930 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.930 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.930 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.930 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4828.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch watch cookie 140587969811264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-11T06:50:54.930 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:50:54.930 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.622047303999999, 2024-09-11T06:50:54.930 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032778859999999998, 2024-09-11T06:50:54.931 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.931 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.931 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:54.931 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.931 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4828", 2024-09-11T06:50:54.931 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2410288650", 2024-09-11T06:50:54.931 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:50:54.931 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.931 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.931 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.931 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.931 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:50:54.931 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.931 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.932 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.932 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:50:54.932 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.932 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.932 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246198+0000", 2024-09-11T06:50:54.932 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.667e-06 2024-09-11T06:50:54.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.932 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.932 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246209+0000", 2024-09-11T06:50:54.933 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0409e-05 2024-09-11T06:50:54.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.933 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.933 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246241+0000", 2024-09-11T06:50:54.933 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2592999999999998e-05 2024-09-11T06:50:54.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.933 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.933 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246260+0000", 2024-09-11T06:50:54.933 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9343000000000001e-05 2024-09-11T06:50:54.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.934 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.934 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246466+0000", 2024-09-11T06:50:54.934 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000205701 2024-09-11T06:50:54.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.934 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.934 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246713+0000", 2024-09-11T06:50:54.934 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024646999999999999 2024-09-11T06:50:54.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.934 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:54.934 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.247133+0000", 2024-09-11T06:50:54.934 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000420536 2024-09-11T06:50:54.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.935 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.935 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:54.935 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.248541+0000", 2024-09-11T06:50:54.935 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001407801 2024-09-11T06:50:54.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.935 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.935 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:54.935 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249241+0000", 2024-09-11T06:50:54.935 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070043500000000001 2024-09-11T06:50:54.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.935 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.935 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:54.935 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249326+0000", 2024-09-11T06:50:54.935 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5037999999999995e-05 2024-09-11T06:50:54.936 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.936 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.936 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249473+0000", 2024-09-11T06:50:54.936 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014689299999999999 2024-09-11T06:50:54.936 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.936 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.936 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.936 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.936 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5313.0:25 2.1 2:967e1786:ns::rbd_object_map.11adc335555f.0000000000000004:head [writefull 0~38] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:50:54.936 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.256788+0000", 2024-09-11T06:50:54.936 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.611454305000001, 2024-09-11T06:50:54.936 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0030446449999999999, 2024-09-11T06:50:54.937 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.937 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.937 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:54.937 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.937 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5313", 2024-09-11T06:50:54.937 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/624314229", 2024-09-11T06:50:54.937 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 25 2024-09-11T06:50:54.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.937 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.937 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.937 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.937 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256788+0000", 2024-09-11T06:50:54.937 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.938 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.938 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256788+0000", 2024-09-11T06:50:54.938 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.938 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.938 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256791+0000", 2024-09-11T06:50:54.938 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5770000000000001e-06 2024-09-11T06:50:54.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.938 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.938 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256802+0000", 2024-09-11T06:50:54.939 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0742e-05 2024-09-11T06:50:54.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.939 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.939 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.939 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256823+0000", 2024-09-11T06:50:54.939 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0999999999999999e-05 2024-09-11T06:50:54.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.939 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.939 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.939 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256841+0000", 2024-09-11T06:50:54.939 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8099000000000001e-05 2024-09-11T06:50:54.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.939 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.939 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.939 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256988+0000", 2024-09-11T06:50:54.940 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014756400000000001 2024-09-11T06:50:54.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.940 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.940 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.257234+0000", 2024-09-11T06:50:54.940 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024524399999999999 2024-09-11T06:50:54.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.940 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:54.940 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.257446+0000", 2024-09-11T06:50:54.940 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000212241 2024-09-11T06:50:54.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.940 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:54.941 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259292+0000", 2024-09-11T06:50:54.941 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001845937 2024-09-11T06:50:54.941 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.941 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.941 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:54.941 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259681+0000", 2024-09-11T06:50:54.941 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00038940699999999998 2024-09-11T06:50:54.941 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.941 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.941 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:54.941 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259770+0000", 2024-09-11T06:50:54.941 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8385000000000002e-05 2024-09-11T06:50:54.941 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.942 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.942 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.942 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259833+0000", 2024-09-11T06:50:54.942 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3448999999999995e-05 2024-09-11T06:50:54.942 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.942 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.942 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.942 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.942 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.942 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5328.0:14 2.1 2:967e1786:ns::rbd_object_map.11adc335555f.0000000000000004:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:50:54.942 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.396257+0000", 2024-09-11T06:50:54.942 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.471985856, 2024-09-11T06:50:54.942 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032125370000000001, 2024-09-11T06:50:54.942 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.943 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.943 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:54.943 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.943 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5328", 2024-09-11T06:50:54.943 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2386493698", 2024-09-11T06:50:54.943 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 14 2024-09-11T06:50:54.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.943 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.943 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.944 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.944 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396257+0000", 2024-09-11T06:50:54.944 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.944 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.944 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396257+0000", 2024-09-11T06:50:54.944 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.944 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.944 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396259+0000", 2024-09-11T06:50:54.944 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4830000000000002e-06 2024-09-11T06:50:54.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.945 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.945 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.945 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396271+0000", 2024-09-11T06:50:54.945 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.115e-05 2024-09-11T06:50:54.945 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.945 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.945 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.945 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396292+0000", 2024-09-11T06:50:54.945 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1761999999999999e-05 2024-09-11T06:50:54.945 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.945 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.945 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.945 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396314+0000", 2024-09-11T06:50:54.946 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1982999999999999e-05 2024-09-11T06:50:54.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.946 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.946 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396409+0000", 2024-09-11T06:50:54.946 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4250000000000001e-05 2024-09-11T06:50:54.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.946 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.946 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396464+0000", 2024-09-11T06:50:54.946 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5884000000000002e-05 2024-09-11T06:50:54.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.946 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.947 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.399469+0000", 2024-09-11T06:50:54.947 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0030050250000000001 2024-09-11T06:50:54.947 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.947 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.947 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.947 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.947 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:6 3.5 3:b2cc3336:::rbd_id.testrbdnbd21607:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:50:54.947 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.591899+0000", 2024-09-11T06:50:54.947 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.2763433549999998, 2024-09-11T06:50:54.947 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032382999999999999, 2024-09-11T06:50:54.947 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.947 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.947 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:54.947 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.948 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:50:54.948 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:50:54.948 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 6 2024-09-11T06:50:54.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.948 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.948 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.948 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591899+0000", 2024-09-11T06:50:54.948 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.948 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.948 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591899+0000", 2024-09-11T06:50:54.949 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.949 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.949 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591902+0000", 2024-09-11T06:50:54.949 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0770000000000002e-06 2024-09-11T06:50:54.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.949 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.949 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591913+0000", 2024-09-11T06:50:54.949 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1182999999999999e-05 2024-09-11T06:50:54.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.949 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.950 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591940+0000", 2024-09-11T06:50:54.950 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7233000000000002e-05 2024-09-11T06:50:54.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.950 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.950 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591954+0000", 2024-09-11T06:50:54.950 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4503e-05 2024-09-11T06:50:54.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.950 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.950 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.592071+0000", 2024-09-11T06:50:54.950 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116731 2024-09-11T06:50:54.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.951 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.951 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.592274+0000", 2024-09-11T06:50:54.951 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00020320900000000001 2024-09-11T06:50:54.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.951 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 3", 2024-09-11T06:50:54.951 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.592576+0000", 2024-09-11T06:50:54.951 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030179000000000002 2024-09-11T06:50:54.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.951 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:54.952 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.594072+0000", 2024-09-11T06:50:54.952 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001495496 2024-09-11T06:50:54.952 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.952 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.952 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:54.952 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.594945+0000", 2024-09-11T06:50:54.952 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00087379799999999996 2024-09-11T06:50:54.952 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.952 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.952 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:54.952 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.595023+0000", 2024-09-11T06:50:54.952 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7751999999999997e-05 2024-09-11T06:50:54.952 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.953 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.953 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.595138+0000", 2024-09-11T06:50:54.953 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000114528 2024-09-11T06:50:54.953 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.953 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.953 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.953 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:44 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:50:54.953 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.540846+0000", 2024-09-11T06:50:54.954 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.3273972230000002, 2024-09-11T06:50:54.954 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032138639999999999, 2024-09-11T06:50:54.954 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.954 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.954 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:54.954 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.954 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:50:54.954 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:50:54.954 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 44 2024-09-11T06:50:54.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.954 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.954 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.955 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.955 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540846+0000", 2024-09-11T06:50:54.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.955 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.955 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540846+0000", 2024-09-11T06:50:54.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.955 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.956 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540848+0000", 2024-09-11T06:50:54.956 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5469999999999999e-06 2024-09-11T06:50:54.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.956 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.956 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540863+0000", 2024-09-11T06:50:54.956 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4613000000000001e-05 2024-09-11T06:50:54.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.956 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.956 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540895+0000", 2024-09-11T06:50:54.957 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2153000000000002e-05 2024-09-11T06:50:54.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.957 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.957 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540912+0000", 2024-09-11T06:50:54.957 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7405999999999999e-05 2024-09-11T06:50:54.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.957 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.957 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.541111+0000", 2024-09-11T06:50:54.957 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019849599999999999 2024-09-11T06:50:54.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.958 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.541269+0000", 2024-09-11T06:50:54.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015792399999999999 2024-09-11T06:50:54.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.958 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:54.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.541853+0000", 2024-09-11T06:50:54.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00058443799999999995 2024-09-11T06:50:54.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.959 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:54.959 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.543133+0000", 2024-09-11T06:50:54.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012797850000000001 2024-09-11T06:50:54.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.959 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.959 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:54.959 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.543905+0000", 2024-09-11T06:50:54.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000771655 2024-09-11T06:50:54.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.959 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.959 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:54.960 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.543993+0000", 2024-09-11T06:50:54.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8168e-05 2024-09-11T06:50:54.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.960 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.960 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.544059+0000", 2024-09-11T06:50:54.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6679000000000003e-05 2024-09-11T06:50:54.960 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.960 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.960 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.961 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5553.0:24 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 139650593197904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:50:54.961 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.566567+0000", 2024-09-11T06:50:54.961 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.3016759530000002, 2024-09-11T06:50:54.961 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034350790000000002, 2024-09-11T06:50:54.961 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.961 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.961 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:54.961 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.961 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5553", 2024-09-11T06:50:54.961 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3590511604", 2024-09-11T06:50:54.961 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 24 2024-09-11T06:50:54.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.962 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566567+0000", 2024-09-11T06:50:54.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566567+0000", 2024-09-11T06:50:54.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.963 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566569+0000", 2024-09-11T06:50:54.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.973e-06 2024-09-11T06:50:54.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.963 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566578+0000", 2024-09-11T06:50:54.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8170000000000005e-06 2024-09-11T06:50:54.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.964 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.964 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566602+0000", 2024-09-11T06:50:54.964 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4596e-05 2024-09-11T06:50:54.964 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.964 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.964 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.964 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566619+0000", 2024-09-11T06:50:54.964 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7099999999999999e-05 2024-09-11T06:50:54.964 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.964 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.964 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.965 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.567321+0000", 2024-09-11T06:50:54.965 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070168200000000002 2024-09-11T06:50:54.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.965 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.965 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.965 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.567454+0000", 2024-09-11T06:50:54.965 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013286400000000001 2024-09-11T06:50:54.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.965 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.965 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:54.965 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.567674+0000", 2024-09-11T06:50:54.965 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000219679 2024-09-11T06:50:54.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.965 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.966 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:54.966 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.569014+0000", 2024-09-11T06:50:54.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013404129999999999 2024-09-11T06:50:54.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.966 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:54.966 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.569634+0000", 2024-09-11T06:50:54.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00062029200000000004 2024-09-11T06:50:54.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.966 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:54.966 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.569805+0000", 2024-09-11T06:50:54.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000170366 2024-09-11T06:50:54.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.967 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.967 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.570002+0000", 2024-09-11T06:50:54.967 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019729700000000001 2024-09-11T06:50:54.967 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.967 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.967 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.967 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:82 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch reconnect cookie 139888024456048 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:50:54.967 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.690185+0000", 2024-09-11T06:50:54.967 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.1780579509999995, 2024-09-11T06:50:54.967 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0045884799999999998, 2024-09-11T06:50:54.968 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.968 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.968 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:54.968 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.968 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:50:54.968 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:50:54.968 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 82 2024-09-11T06:50:54.968 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.968 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.968 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.968 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.968 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690185+0000", 2024-09-11T06:50:54.968 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.968 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.969 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.969 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690185+0000", 2024-09-11T06:50:54.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.969 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.969 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690186+0000", 2024-09-11T06:50:54.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8400000000000002e-07 2024-09-11T06:50:54.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.969 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.969 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690190+0000", 2024-09-11T06:50:54.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5499999999999996e-06 2024-09-11T06:50:54.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.970 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.970 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690203+0000", 2024-09-11T06:50:54.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2228999999999999e-05 2024-09-11T06:50:54.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.970 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.970 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690228+0000", 2024-09-11T06:50:54.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5587000000000001e-05 2024-09-11T06:50:54.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.691298+0000", 2024-09-11T06:50:54.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001070218 2024-09-11T06:50:54.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.691511+0000", 2024-09-11T06:50:54.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021301299999999999 2024-09-11T06:50:54.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.694773+0000", 2024-09-11T06:50:54.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003261899 2024-09-11T06:50:54.972 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.972 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.972 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.972 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:83 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify cookie 139887621709056] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:50:54.972 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.690242+0000", 2024-09-11T06:50:54.972 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.1780007789999996, 2024-09-11T06:50:54.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003040067, 2024-09-11T06:50:54.972 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.972 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.972 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:54.972 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.973 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:50:54.973 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:50:54.973 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 83 2024-09-11T06:50:54.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.973 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690242+0000", 2024-09-11T06:50:54.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690242+0000", 2024-09-11T06:50:54.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690243+0000", 2024-09-11T06:50:54.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0640000000000001e-06 2024-09-11T06:50:54.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690247+0000", 2024-09-11T06:50:54.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3760000000000001e-06 2024-09-11T06:50:54.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690258+0000", 2024-09-11T06:50:54.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0874e-05 2024-09-11T06:50:54.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690266+0000", 2024-09-11T06:50:54.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5429999999999996e-06 2024-09-11T06:50:54.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.692564+0000", 2024-09-11T06:50:54.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0022977010000000001 2024-09-11T06:50:54.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.976 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.692882+0000", 2024-09-11T06:50:54.976 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00031862699999999998 2024-09-11T06:50:54.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.976 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.976 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.976 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693282+0000", 2024-09-11T06:50:54.976 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000399882 2024-09-11T06:50:54.976 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.976 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.976 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.976 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.976 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:84 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify cookie 139887823033168] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:50:54.976 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:50:54.977 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.1779661749999999, 2024-09-11T06:50:54.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035622620000000001, 2024-09-11T06:50:54.977 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.977 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.977 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:54.977 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.977 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:50:54.977 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:50:54.977 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 84 2024-09-11T06:50:54.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.977 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.978 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:50:54.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.978 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.978 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:50:54.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.978 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.978 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:50:54.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5300000000000003e-07 2024-09-11T06:50:54.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.979 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690281+0000", 2024-09-11T06:50:54.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9330000000000003e-06 2024-09-11T06:50:54.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.979 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690291+0000", 2024-09-11T06:50:54.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7359999999999996e-06 2024-09-11T06:50:54.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.979 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690298+0000", 2024-09-11T06:50:54.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3699999999999997e-06 2024-09-11T06:50:54.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.980 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.980 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.980 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693380+0000", 2024-09-11T06:50:54.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0030811060000000001 2024-09-11T06:50:54.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.980 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.980 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.980 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693599+0000", 2024-09-11T06:50:54.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021947199999999999 2024-09-11T06:50:54.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.980 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.980 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.981 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693839+0000", 2024-09-11T06:50:54.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023989200000000001 2024-09-11T06:50:54.981 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.981 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.981 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.981 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5098.0:25 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 140617967560800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:50:54.981 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.697727+0000", 2024-09-11T06:50:54.981 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.1705155989999998, 2024-09-11T06:50:54.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035341330000000001, 2024-09-11T06:50:54.981 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:54.981 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:54.981 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:54.982 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:54.982 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5098", 2024-09-11T06:50:54.982 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/844541331", 2024-09-11T06:50:54.982 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 25 2024-09-11T06:50:54.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.982 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:54.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.982 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:54.982 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697727+0000", 2024-09-11T06:50:54.982 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.982 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:54.982 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697727+0000", 2024-09-11T06:50:54.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:54.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:54.983 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697729+0000", 2024-09-11T06:50:54.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.33e-06 2024-09-11T06:50:54.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:54.983 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697734+0000", 2024-09-11T06:50:54.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6590000000000004e-06 2024-09-11T06:50:54.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:54.984 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697749+0000", 2024-09-11T06:50:54.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4867e-05 2024-09-11T06:50:54.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.984 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:54.984 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697759+0000", 2024-09-11T06:50:54.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0073e-05 2024-09-11T06:50:54.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.984 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:54.984 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.698715+0000", 2024-09-11T06:50:54.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00095545799999999996 2024-09-11T06:50:54.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.984 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:54.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.698850+0000", 2024-09-11T06:50:54.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000135814 2024-09-11T06:50:54.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:54.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.699085+0000", 2024-09-11T06:50:54.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023507199999999999 2024-09-11T06:50:54.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:54.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.700258+0000", 2024-09-11T06:50:54.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011726600000000001 2024-09-11T06:50:54.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:54.986 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.701061+0000", 2024-09-11T06:50:54.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00080256100000000003 2024-09-11T06:50:54.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:54.986 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.701144+0000", 2024-09-11T06:50:54.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3188000000000004e-05 2024-09-11T06:50:54.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:54.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:54.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:54.986 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.701261+0000", 2024-09-11T06:50:54.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000117451 2024-09-11T06:50:54.987 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.987 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.987 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.987 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:54.987 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:54.987 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:54.989 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:50:54.990 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:54.987+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:55.015 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:50:55.015 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:50:55.031 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:50:55.031 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:50:55.048 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:50:55.048 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:50:55.067 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:50:55.067 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:50:55.087 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:50:55.087 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:50:55.089 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:55.082+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:55.108 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:50:55.109 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:50:55.120 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:55.120 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:55.134 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:50:55.190 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:55.186+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:55.258 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:55.258 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:55.273 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:50:55.291 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:55.286+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:55.392 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:55.390+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:55.396 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:55.396 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:55.411 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:50:55.493 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:55.490+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:55.530 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:55.530 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:55.544 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:50:55.593 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:55.590+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:55.667 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:55.668 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:55.682 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:50:55.728 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:55.727+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:55.796 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:55.795+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:55.809 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:55.809 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:55.824 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:50:55.872 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T06:50:55.912 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:55.911+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:55.990 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:55.991 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:55.997 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:55.995+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:56.004 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:50:56.098 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:56.095+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:56.177 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:56.177 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:56.191 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:50:56.198 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:56.194+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:56.299 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:56.295+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:56.357 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:56.357 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:56.372 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:50:56.417 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:56.414+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:56.467 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 1, 3, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T06:50:56.467 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T06:50:56.467 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-11T06:50:56.467 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-11T06:50:56.501 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:56.499+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:56.501 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:56.501 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:56.516 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:50:56.602 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:56.599+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:56.685 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:56.685 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:56.699 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:50:56.703 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:56.699+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:56.765 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:50:56.765 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:56.765 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 84, 2024-09-11T06:50:56.765 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "bd5dcb33-e9f7-4a5f-a26c-a451452e8aa2", 2024-09-11T06:50:56.765 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2024-09-11T06:48:52.191706+0000", 2024-09-11T06:50:56.765 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2024-09-11T06:50:53.798049+0000", 2024-09-11T06:50:56.765 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2024-09-11T06:49:04.007203+0000", 2024-09-11T06:50:56.765 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2024-09-11T06:50:46.287457+0000", 2024-09-11T06:50:56.765 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-11T06:50:56.766 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2024-09-11T06:50:56.766 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2024-09-11T06:50:56.766 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2024-09-11T06:50:56.766 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2024-09-11T06:50:56.766 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2024-09-11T06:50:56.766 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2024-09-11T06:50:56.766 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-11T06:50:56.766 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2024-09-11T06:50:56.766 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.94999998807907104, 2024-09-11T06:50:56.766 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-11T06:50:56.766 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-11T06:50:56.766 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2024-09-11T06:50:56.767 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 3, 2024-09-11T06:50:56.767 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 6, 2024-09-11T06:50:56.767 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2024-09-11T06:50:56.767 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "jewel", 2024-09-11T06:50:56.767 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "squid", 2024-09-11T06:50:56.767 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2024-09-11T06:50:56.767 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2024-09-11T06:50:56.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.767 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2024-09-11T06:50:56.767 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2024-09-11T06:50:56.767 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2024-09-11T06:49:05.084515+0000", 2024-09-11T06:50:56.767 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-11T06:50:56.767 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2024-09-11T06:50:56.767 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2024-09-11T06:50:56.768 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2024-09-11T06:50:56.768 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2024-09-11T06:50:56.768 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2024-09-11T06:50:56.768 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2024-09-11T06:50:56.768 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2024-09-11T06:50:56.768 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T06:50:56.768 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T06:50:56.768 INFO:teuthology.orchestra.run.smithi071.stdout: "is_stretch_pool": false, 2024-09-11T06:50:56.768 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2024-09-11T06:50:56.768 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2024-09-11T06:50:56.768 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2024-09-11T06:50:56.768 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2024-09-11T06:50:56.768 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2024-09-11T06:50:56.769 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2024-09-11T06:50:56.769 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2024-09-11T06:50:56.769 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2024-09-11T06:50:56.769 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2024-09-11T06:50:56.769 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2024-09-11T06:50:56.769 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2024-09-11T06:50:56.769 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2024-09-11T06:50:56.769 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2024-09-11T06:50:56.769 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2024-09-11T06:50:56.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.769 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "14", 2024-09-11T06:50:56.769 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2024-09-11T06:50:56.769 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T06:50:56.770 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T06:50:56.770 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2024-09-11T06:50:56.770 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2024-09-11T06:50:56.770 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2024-09-11T06:50:56.770 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2024-09-11T06:50:56.770 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2024-09-11T06:50:56.770 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2024-09-11T06:50:56.770 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2024-09-11T06:50:56.770 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2024-09-11T06:50:56.770 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2024-09-11T06:50:56.770 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2024-09-11T06:50:56.770 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2024-09-11T06:50:56.770 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2024-09-11T06:50:56.771 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2024-09-11T06:50:56.771 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2024-09-11T06:50:56.771 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2024-09-11T06:50:56.771 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T06:50:56.771 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T06:50:56.771 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T06:50:56.771 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2024-09-11T06:50:56.771 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2024-09-11T06:50:56.771 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2024-09-11T06:50:56.771 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2024-09-11T06:50:56.771 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2024-09-11T06:50:56.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.771 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2024-09-11T06:50:56.772 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2024-09-11T06:50:56.772 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2024-09-11T06:50:56.772 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2024-09-11T06:50:56.772 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2024-09-11T06:50:56.772 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T06:50:56.772 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2024-09-11T06:50:56.772 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2024-09-11T06:50:56.772 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2024-09-11T06:50:56.772 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2024-09-11T06:50:56.772 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2024-09-11T06:50:56.772 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2024-09-11T06:50:56.772 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2024-09-11T06:50:56.772 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2024-09-11T06:50:56.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.773 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2024-09-11T06:50:56.773 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2024-09-11T06:50:56.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.773 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2024-09-11T06:50:56.773 INFO:teuthology.orchestra.run.smithi071.stdout: "score_type": "Fair distribution", 2024-09-11T06:50:56.773 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 6.059999942779541, 2024-09-11T06:50:56.773 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 6.059999942779541, 2024-09-11T06:50:56.773 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.33000001311302185, 2024-09-11T06:50:56.773 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2024-09-11T06:50:56.773 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2024-09-11T06:50:56.773 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 0.85340124368667603, 2024-09-11T06:50:56.773 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 0.85000002384185791, 2024-09-11T06:50:56.773 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 0.85000002384185791 2024-09-11T06:50:56.774 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.774 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 2, 2024-09-11T06:50:56.774 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "rbd", 2024-09-11T06:50:56.774 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2024-09-11T06:49:09.530167+0000", 2024-09-11T06:50:56.774 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 8193, 2024-09-11T06:50:56.774 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T06:50:56.774 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2024-09-11T06:50:56.774 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2024-09-11T06:50:56.774 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2024-09-11T06:50:56.774 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2024-09-11T06:50:56.774 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2024-09-11T06:50:56.774 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2024-09-11T06:50:56.775 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T06:50:56.775 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T06:50:56.775 INFO:teuthology.orchestra.run.smithi071.stdout: "is_stretch_pool": false, 2024-09-11T06:50:56.775 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2024-09-11T06:50:56.775 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2024-09-11T06:50:56.775 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 8, 2024-09-11T06:50:56.775 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 8, 2024-09-11T06:50:56.775 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 8, 2024-09-11T06:50:56.775 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 8, 2024-09-11T06:50:56.775 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 8, 2024-09-11T06:50:56.775 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2024-09-11T06:50:56.775 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2024-09-11T06:50:56.776 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2024-09-11T06:50:56.776 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2024-09-11T06:50:56.776 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2024-09-11T06:50:56.776 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2024-09-11T06:50:56.776 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2024-09-11T06:50:56.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.776 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "82", 2024-09-11T06:50:56.776 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2024-09-11T06:50:56.776 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T06:50:56.776 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T06:50:56.776 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2024-09-11T06:50:56.776 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2024-09-11T06:50:56.776 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 5, 2024-09-11T06:50:56.776 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 82, 2024-09-11T06:50:56.777 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2024-09-11T06:50:56.777 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2024-09-11T06:50:56.777 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2024-09-11T06:50:56.777 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2024-09-11T06:50:56.777 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2024-09-11T06:50:56.777 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2024-09-11T06:50:56.777 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2024-09-11T06:50:56.777 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2024-09-11T06:50:56.777 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2024-09-11T06:50:56.777 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2024-09-11T06:50:56.777 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2024-09-11T06:50:56.777 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T06:50:56.778 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T06:50:56.778 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T06:50:56.781 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2024-09-11T06:50:56.781 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2024-09-11T06:50:56.781 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2024-09-11T06:50:56.781 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2024-09-11T06:50:56.781 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2024-09-11T06:50:56.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.781 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2024-09-11T06:50:56.781 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2024-09-11T06:50:56.781 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2024-09-11T06:50:56.781 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2024-09-11T06:50:56.781 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2024-09-11T06:50:56.781 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T06:50:56.781 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2024-09-11T06:50:56.782 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2024-09-11T06:50:56.782 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2024-09-11T06:50:56.782 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2024-09-11T06:50:56.782 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2024-09-11T06:50:56.782 INFO:teuthology.orchestra.run.smithi071.stdout: "options": {}, 2024-09-11T06:50:56.782 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2024-09-11T06:50:56.782 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd": {} 2024-09-11T06:50:56.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.782 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2024-09-11T06:50:56.782 INFO:teuthology.orchestra.run.smithi071.stdout: "score_type": "Fair distribution", 2024-09-11T06:50:56.782 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 2.25, 2024-09-11T06:50:56.782 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 2.25, 2024-09-11T06:50:56.782 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2024-09-11T06:50:56.783 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2.25, 2024-09-11T06:50:56.783 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2.25, 2024-09-11T06:50:56.783 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 0.85340124368667603, 2024-09-11T06:50:56.783 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 0.85000002384185791, 2024-09-11T06:50:56.783 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 0.85000002384185791 2024-09-11T06:50:56.783 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.783 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 3, 2024-09-11T06:50:56.783 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "new_default_pool21607", 2024-09-11T06:50:56.784 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2024-09-11T06:50:15.393063+0000", 2024-09-11T06:50:56.784 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 8193, 2024-09-11T06:50:56.784 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T06:50:56.784 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2024-09-11T06:50:56.784 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2024-09-11T06:50:56.784 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2024-09-11T06:50:56.784 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2024-09-11T06:50:56.784 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2024-09-11T06:50:56.784 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2024-09-11T06:50:56.784 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T06:50:56.784 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T06:50:56.784 INFO:teuthology.orchestra.run.smithi071.stdout: "is_stretch_pool": false, 2024-09-11T06:50:56.784 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2024-09-11T06:50:56.785 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2024-09-11T06:50:56.785 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 8, 2024-09-11T06:50:56.785 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 8, 2024-09-11T06:50:56.785 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 8, 2024-09-11T06:50:56.785 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 8, 2024-09-11T06:50:56.785 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 8, 2024-09-11T06:50:56.785 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2024-09-11T06:50:56.786 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2024-09-11T06:50:56.786 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2024-09-11T06:50:56.786 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2024-09-11T06:50:56.786 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2024-09-11T06:50:56.786 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2024-09-11T06:50:56.786 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2024-09-11T06:50:56.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.787 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "58", 2024-09-11T06:50:56.787 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2024-09-11T06:50:56.787 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T06:50:56.787 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T06:50:56.788 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2024-09-11T06:50:56.788 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2024-09-11T06:50:56.788 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 2, 2024-09-11T06:50:56.788 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 58, 2024-09-11T06:50:56.788 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2024-09-11T06:50:56.788 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2024-09-11T06:50:56.789 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2024-09-11T06:50:56.789 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2024-09-11T06:50:56.789 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2024-09-11T06:50:56.789 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2024-09-11T06:50:56.789 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2024-09-11T06:50:56.789 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2024-09-11T06:50:56.789 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2024-09-11T06:50:56.789 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2024-09-11T06:50:56.789 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2024-09-11T06:50:56.789 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T06:50:56.790 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T06:50:56.790 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T06:50:56.790 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2024-09-11T06:50:56.790 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2024-09-11T06:50:56.790 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2024-09-11T06:50:56.790 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2024-09-11T06:50:56.790 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2024-09-11T06:50:56.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.790 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2024-09-11T06:50:56.790 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2024-09-11T06:50:56.790 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2024-09-11T06:50:56.790 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2024-09-11T06:50:56.790 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2024-09-11T06:50:56.790 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T06:50:56.790 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2024-09-11T06:50:56.791 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2024-09-11T06:50:56.791 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2024-09-11T06:50:56.791 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2024-09-11T06:50:56.791 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2024-09-11T06:50:56.791 INFO:teuthology.orchestra.run.smithi071.stdout: "options": {}, 2024-09-11T06:50:56.791 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2024-09-11T06:50:56.791 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd": {} 2024-09-11T06:50:56.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.791 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2024-09-11T06:50:56.791 INFO:teuthology.orchestra.run.smithi071.stdout: "score_type": "Fair distribution", 2024-09-11T06:50:56.791 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 2.2699999809265137, 2024-09-11T06:50:56.791 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 2.2699999809265137, 2024-09-11T06:50:56.791 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.82999998331069946, 2024-09-11T06:50:56.791 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-11T06:50:56.792 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-11T06:50:56.792 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 0.85340124368667603, 2024-09-11T06:50:56.792 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 0.85000002384185791, 2024-09-11T06:50:56.792 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 0.85000002384185791 2024-09-11T06:50:56.792 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.792 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.792 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-11T06:50:56.792 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2024-09-11T06:50:56.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.792 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2024-09-11T06:50:56.792 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "3c9e35bf-d3db-49db-b10c-d2cf38c7eb7e", 2024-09-11T06:50:56.792 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-11T06:50:56.792 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-11T06:50:56.793 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-11T06:50:56.793 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 0.1204071044921875, 2024-09-11T06:50:56.793 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-11T06:50:56.793 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-11T06:50:56.793 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 11, 2024-09-11T06:50:56.793 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 71, 2024-09-11T06:50:56.793 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-11T06:50:56.793 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-11T06:50:56.793 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-11T06:50:56.793 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:50:56.793 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.793 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:50:56.793 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2024-09-11T06:50:56.793 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1158147777 2024-09-11T06:50:56.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.794 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:50:56.794 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2024-09-11T06:50:56.794 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1158147777 2024-09-11T06:50:56.794 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.794 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.794 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-11T06:50:56.794 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:50:56.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.794 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:50:56.794 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2024-09-11T06:50:56.794 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1158147777 2024-09-11T06:50:56.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.795 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:50:56.795 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2024-09-11T06:50:56.795 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1158147777 2024-09-11T06:50:56.795 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.795 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.795 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-11T06:50:56.795 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:50:56.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.795 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:50:56.795 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2024-09-11T06:50:56.795 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1158147777 2024-09-11T06:50:56.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.796 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:50:56.796 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2024-09-11T06:50:56.796 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1158147777 2024-09-11T06:50:56.796 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.796 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.796 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-11T06:50:56.796 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:50:56.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.796 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:50:56.796 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2024-09-11T06:50:56.796 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1158147777 2024-09-11T06:50:56.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.797 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:50:56.797 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2024-09-11T06:50:56.797 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1158147777 2024-09-11T06:50:56.797 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.797 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.797 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/1158147777", 2024-09-11T06:50:56.797 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/1158147777", 2024-09-11T06:50:56.797 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/1158147777", 2024-09-11T06:50:56.797 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/1158147777", 2024-09-11T06:50:56.797 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-11T06:50:56.798 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-11T06:50:56.798 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-11T06:50:56.798 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.798 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2024-09-11T06:50:56.798 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "693649dc-db7c-43cf-bdbe-87f6128ef311", 2024-09-11T06:50:56.798 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-11T06:50:56.798 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-11T06:50:56.798 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-11T06:50:56.798 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-11T06:50:56.798 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-11T06:50:56.798 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-11T06:50:56.798 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 11, 2024-09-11T06:50:56.799 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 80, 2024-09-11T06:50:56.799 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-11T06:50:56.799 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-11T06:50:56.799 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-11T06:50:56.799 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:50:56.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.799 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:50:56.799 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2024-09-11T06:50:56.799 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 620364628 2024-09-11T06:50:56.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.799 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:50:56.799 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2024-09-11T06:50:56.800 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 620364628 2024-09-11T06:50:56.800 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.800 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.800 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-11T06:50:56.800 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:50:56.800 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.800 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:50:56.800 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2024-09-11T06:50:56.800 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 620364628 2024-09-11T06:50:56.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.800 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.800 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:50:56.801 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2024-09-11T06:50:56.801 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 620364628 2024-09-11T06:50:56.801 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.801 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.801 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-11T06:50:56.801 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:50:56.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.801 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:50:56.801 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2024-09-11T06:50:56.801 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 620364628 2024-09-11T06:50:56.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.802 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:50:56.802 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2024-09-11T06:50:56.802 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 620364628 2024-09-11T06:50:56.802 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.802 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.802 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-11T06:50:56.802 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:50:56.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.802 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:50:56.802 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2024-09-11T06:50:56.802 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 620364628 2024-09-11T06:50:56.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.803 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:50:56.803 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2024-09-11T06:50:56.803 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 620364628 2024-09-11T06:50:56.803 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.803 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.803 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/620364628", 2024-09-11T06:50:56.803 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6811/620364628", 2024-09-11T06:50:56.803 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6815/620364628", 2024-09-11T06:50:56.803 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6813/620364628", 2024-09-11T06:50:56.803 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-11T06:50:56.803 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-11T06:50:56.803 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-11T06:50:56.804 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.804 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2024-09-11T06:50:56.804 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "89421ae7-fead-409c-83f7-ede424db6004", 2024-09-11T06:50:56.804 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-11T06:50:56.804 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-11T06:50:56.804 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-11T06:50:56.804 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-11T06:50:56.804 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-11T06:50:56.804 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-11T06:50:56.804 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 11, 2024-09-11T06:50:56.804 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 69, 2024-09-11T06:50:56.805 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-11T06:50:56.805 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-11T06:50:56.805 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-11T06:50:56.805 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:50:56.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.805 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:50:56.805 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2024-09-11T06:50:56.805 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926882521 2024-09-11T06:50:56.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.805 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:50:56.805 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2024-09-11T06:50:56.805 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926882521 2024-09-11T06:50:56.805 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.806 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.806 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-11T06:50:56.806 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:50:56.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.806 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:50:56.806 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2024-09-11T06:50:56.806 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926882521 2024-09-11T06:50:56.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.806 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:50:56.806 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2024-09-11T06:50:56.807 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926882521 2024-09-11T06:50:56.807 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.807 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.807 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-11T06:50:56.807 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:50:56.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.807 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:50:56.807 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2024-09-11T06:50:56.807 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926882521 2024-09-11T06:50:56.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.807 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:50:56.807 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2024-09-11T06:50:56.808 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926882521 2024-09-11T06:50:56.808 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.808 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.808 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-11T06:50:56.808 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:50:56.808 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.808 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:50:56.808 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2024-09-11T06:50:56.808 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926882521 2024-09-11T06:50:56.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.808 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.808 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:50:56.808 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2024-09-11T06:50:56.809 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926882521 2024-09-11T06:50:56.809 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.809 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.809 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6817/2926882521", 2024-09-11T06:50:56.809 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6819/2926882521", 2024-09-11T06:50:56.809 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/2926882521", 2024-09-11T06:50:56.809 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6821/2926882521", 2024-09-11T06:50:56.809 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-11T06:50:56.809 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-11T06:50:56.809 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-11T06:50:56.809 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.810 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2024-09-11T06:50:56.810 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "f4512e48-63f1-42b6-81be-594bd82e3210", 2024-09-11T06:50:56.810 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-11T06:50:56.810 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-11T06:50:56.810 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-11T06:50:56.810 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-11T06:50:56.810 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-11T06:50:56.810 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-11T06:50:56.810 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 11, 2024-09-11T06:50:56.810 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 72, 2024-09-11T06:50:56.810 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-11T06:50:56.810 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-11T06:50:56.810 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-11T06:50:56.811 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:50:56.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.811 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:50:56.811 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6808", 2024-09-11T06:50:56.811 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 558942315 2024-09-11T06:50:56.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.811 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:50:56.811 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6809", 2024-09-11T06:50:56.811 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 558942315 2024-09-11T06:50:56.811 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.811 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.812 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-11T06:50:56.812 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:50:56.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.812 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:50:56.812 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6810", 2024-09-11T06:50:56.812 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 558942315 2024-09-11T06:50:56.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.812 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:50:56.812 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6811", 2024-09-11T06:50:56.812 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 558942315 2024-09-11T06:50:56.812 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.812 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.813 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-11T06:50:56.813 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:50:56.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.813 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:50:56.813 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6814", 2024-09-11T06:50:56.813 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 558942315 2024-09-11T06:50:56.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.813 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:50:56.813 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6815", 2024-09-11T06:50:56.813 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 558942315 2024-09-11T06:50:56.813 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.813 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.814 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-11T06:50:56.814 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:50:56.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.814 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:50:56.814 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6812", 2024-09-11T06:50:56.814 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 558942315 2024-09-11T06:50:56.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.814 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:50:56.814 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6813", 2024-09-11T06:50:56.814 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 558942315 2024-09-11T06:50:56.814 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.815 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.815 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.89:6809/558942315", 2024-09-11T06:50:56.815 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.89:6811/558942315", 2024-09-11T06:50:56.815 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.89:6815/558942315", 2024-09-11T06:50:56.815 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.89:6813/558942315", 2024-09-11T06:50:56.815 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-11T06:50:56.815 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-11T06:50:56.815 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-11T06:50:56.815 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.815 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2024-09-11T06:50:56.815 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "52d47b6f-af30-455e-8326-e53c598911ae", 2024-09-11T06:50:56.816 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-11T06:50:56.816 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-11T06:50:56.816 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-11T06:50:56.816 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-11T06:50:56.816 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-11T06:50:56.816 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-11T06:50:56.816 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 11, 2024-09-11T06:50:56.816 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 60, 2024-09-11T06:50:56.816 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-11T06:50:56.816 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-11T06:50:56.816 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-11T06:50:56.816 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:50:56.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.817 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:50:56.817 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6816", 2024-09-11T06:50:56.817 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3878488003 2024-09-11T06:50:56.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.817 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.817 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:50:56.817 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6817", 2024-09-11T06:50:56.817 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3878488003 2024-09-11T06:50:56.817 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.817 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.817 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-11T06:50:56.817 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:50:56.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.818 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:50:56.818 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6818", 2024-09-11T06:50:56.818 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3878488003 2024-09-11T06:50:56.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.818 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:50:56.818 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6819", 2024-09-11T06:50:56.818 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3878488003 2024-09-11T06:50:56.818 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.818 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.818 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-11T06:50:56.818 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:50:56.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.819 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:50:56.819 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6822", 2024-09-11T06:50:56.819 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3878488003 2024-09-11T06:50:56.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.819 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:50:56.819 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6823", 2024-09-11T06:50:56.819 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3878488003 2024-09-11T06:50:56.819 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.819 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.819 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-11T06:50:56.820 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:50:56.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.820 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:50:56.820 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6820", 2024-09-11T06:50:56.820 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3878488003 2024-09-11T06:50:56.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.820 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:50:56.820 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6821", 2024-09-11T06:50:56.820 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3878488003 2024-09-11T06:50:56.820 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.820 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.820 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.89:6817/3878488003", 2024-09-11T06:50:56.821 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.89:6819/3878488003", 2024-09-11T06:50:56.821 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.89:6823/3878488003", 2024-09-11T06:50:56.821 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.89:6821/3878488003", 2024-09-11T06:50:56.821 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-11T06:50:56.821 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-11T06:50:56.821 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-11T06:50:56.821 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.821 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2024-09-11T06:50:56.821 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "6b4f9416-0ba0-4c58-883e-8cbdb362adcc", 2024-09-11T06:50:56.821 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-11T06:50:56.821 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-11T06:50:56.821 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-11T06:50:56.822 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-11T06:50:56.822 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-11T06:50:56.822 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-11T06:50:56.822 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 11, 2024-09-11T06:50:56.822 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 80, 2024-09-11T06:50:56.822 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-11T06:50:56.822 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-11T06:50:56.822 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-11T06:50:56.822 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:50:56.822 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.822 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:50:56.822 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6800", 2024-09-11T06:50:56.822 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2840204107 2024-09-11T06:50:56.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.823 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:50:56.823 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6801", 2024-09-11T06:50:56.823 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2840204107 2024-09-11T06:50:56.823 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.823 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.823 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-11T06:50:56.823 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:50:56.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.823 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:50:56.823 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6802", 2024-09-11T06:50:56.823 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2840204107 2024-09-11T06:50:56.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.824 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:50:56.824 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6803", 2024-09-11T06:50:56.824 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2840204107 2024-09-11T06:50:56.824 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.824 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.824 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-11T06:50:56.824 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:50:56.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.824 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:50:56.824 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6806", 2024-09-11T06:50:56.824 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2840204107 2024-09-11T06:50:56.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.825 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:50:56.825 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6807", 2024-09-11T06:50:56.825 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2840204107 2024-09-11T06:50:56.825 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.825 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.825 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-11T06:50:56.825 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:50:56.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.825 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:50:56.825 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6804", 2024-09-11T06:50:56.826 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2840204107 2024-09-11T06:50:56.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.826 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:50:56.826 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6805", 2024-09-11T06:50:56.826 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2840204107 2024-09-11T06:50:56.826 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.826 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.826 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.89:6801/2840204107", 2024-09-11T06:50:56.826 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.89:6803/2840204107", 2024-09-11T06:50:56.826 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.89:6807/2840204107", 2024-09-11T06:50:56.826 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.89:6805/2840204107", 2024-09-11T06:50:56.827 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-11T06:50:56.827 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-11T06:50:56.827 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-11T06:50:56.827 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:56.827 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.827 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-11T06:50:56.827 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2024-09-11T06:50:56.827 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.827 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2024-09-11T06:50:56.827 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-11T06:50:56.827 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-11T06:50:56.827 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-11T06:50:56.828 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540701547738038271, 2024-09-11T06:50:56.828 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-11T06:50:56.828 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-11T06:49:00.282785+0000", 2024-09-11T06:50:56.828 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-11T06:50:56.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.828 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2024-09-11T06:50:56.828 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-11T06:50:56.828 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-11T06:50:56.828 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-11T06:50:56.828 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540701547738038271, 2024-09-11T06:50:56.828 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-11T06:50:56.830 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-11T06:49:00.381259+0000", 2024-09-11T06:50:56.830 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-11T06:50:56.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.830 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2024-09-11T06:50:56.830 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-11T06:50:56.830 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-11T06:50:56.830 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-11T06:50:56.830 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540701547738038271, 2024-09-11T06:50:56.831 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-11T06:50:56.831 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-11T06:49:00.986977+0000", 2024-09-11T06:50:56.831 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-11T06:50:56.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.831 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2024-09-11T06:50:56.831 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-11T06:50:56.831 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-11T06:50:56.831 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-11T06:50:56.831 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540701547738038271, 2024-09-11T06:50:56.831 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-11T06:50:56.831 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-11T06:49:00.732329+0000", 2024-09-11T06:50:56.831 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-11T06:50:56.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.832 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2024-09-11T06:50:56.832 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-11T06:50:56.832 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-11T06:50:56.832 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-11T06:50:56.832 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540701547738038271, 2024-09-11T06:50:56.832 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-11T06:50:56.832 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-11T06:49:00.844988+0000", 2024-09-11T06:50:56.832 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-11T06:50:56.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:56.832 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2024-09-11T06:50:56.832 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-11T06:50:56.833 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-11T06:50:56.833 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-11T06:50:56.833 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540701547738038271, 2024-09-11T06:50:56.833 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-11T06:50:56.833 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-11T06:49:00.639989+0000", 2024-09-11T06:50:56.833 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-11T06:50:56.833 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.833 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-11T06:50:56.833 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2024-09-11T06:50:56.833 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [], 2024-09-11T06:50:56.833 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2024-09-11T06:50:56.833 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2024-09-11T06:50:56.833 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2024-09-11T06:50:56.834 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2024-09-11T06:50:56.834 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2024-09-11T06:50:56.834 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2024-09-11T06:50:56.834 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2024-09-11T06:50:56.834 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2024-09-11T06:50:56.834 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2024-09-11T06:50:56.834 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2024-09-11T06:50:56.834 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2024-09-11T06:50:56.834 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2024-09-11T06:50:56.834 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:56.834 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2024-09-11T06:50:56.834 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2024-09-11T06:50:56.834 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2024-09-11T06:50:56.835 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2024-09-11T06:50:56.835 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2024-09-11T06:50:56.835 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2024-09-11T06:50:56.835 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2024-09-11T06:50:56.835 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2024-09-11T06:50:56.835 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2024-09-11T06:50:56.835 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2024-09-11T06:50:56.835 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2024-09-11T06:50:56.835 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:56.835 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:56.837 INFO:tasks.thrashosds.thrasher:j is {'epoch': 84, 'fsid': 'bd5dcb33-e9f7-4a5f-a26c-a451452e8aa2', 'created': '2024-09-11T06:48:52.191706+0000', 'modified': '2024-09-11T06:50:53.798049+0000', 'last_up_change': '2024-09-11T06:49:04.007203+0000', 'last_in_change': '2024-09-11T06:50:46.287457+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T06:49:05.084515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.853401243686676, 'average_primary_affinity': 0.8500000238418579, 'average_primary_affinity_weighted': 0.8500000238418579}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-11T06:49:09.530167+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 82, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.853401243686676, 'average_primary_affinity': 0.8500000238418579, 'average_primary_affinity_weighted': 0.8500000238418579}}, {'pool': 3, 'pool_name': 'new_default_pool21607', 'create_time': '2024-09-11T06:50:15.393063+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 58, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.2699999809265137, 'score_stable': 2.2699999809265137, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.853401243686676, 'average_primary_affinity': 0.8500000238418579, 'average_primary_affinity_weighted': 0.8500000238418579}}], 'osds': [{'osd': 0, 'uuid': '3c9e35bf-d3db-49db-b10c-d2cf38c7eb7e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.1204071044921875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 71, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6800', 'nonce': 1158147777}, {'type': 'v1', 'addr': '172.21.15.71:6801', 'nonce': 1158147777}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6802', 'nonce': 1158147777}, {'type': 'v1', 'addr': '172.21.15.71:6803', 'nonce': 1158147777}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6806', 'nonce': 1158147777}, {'type': 'v1', 'addr': '172.21.15.71:6807', 'nonce': 1158147777}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6804', 'nonce': 1158147777}, {'type': 'v1', 'addr': '172.21.15.71:6805', 'nonce': 1158147777}]}, 'public_addr': '172.21.15.71:6801/1158147777', 'cluster_addr': '172.21.15.71:6803/1158147777', 'heartbeat_back_addr': '172.21.15.71:6807/1158147777', 'heartbeat_front_addr': '172.21.15.71:6805/1158147777', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '693649dc-db7c-43cf-bdbe-87f6128ef311', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 80, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6808', 'nonce': 620364628}, {'type': 'v1', 'addr': '172.21.15.71:6809', 'nonce': 620364628}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6810', 'nonce': 620364628}, {'type': 'v1', 'addr': '172.21.15.71:6811', 'nonce': 620364628}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6814', 'nonce': 620364628}, {'type': 'v1', 'addr': '172.21.15.71:6815', 'nonce': 620364628}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6812', 'nonce': 620364628}, {'type': 'v1', 'addr': '172.21.15.71:6813', 'nonce': 620364628}]}, 'public_addr': '172.21.15.71:6809/620364628', 'cluster_addr': '172.21.15.71:6811/620364628', 'heartbeat_back_addr': '172.21.15.71:6815/620364628', 'heartbeat_front_addr': '172.21.15.71:6813/620364628', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '89421ae7-fead-409c-83f7-ede424db6004', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 69, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6816', 'nonce': 2926882521}, {'type': 'v1', 'addr': '172.21.15.71:6817', 'nonce': 2926882521}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6818', 'nonce': 2926882521}, {'type': 'v1', 'addr': '172.21.15.71:6819', 'nonce': 2926882521}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6822', 'nonce': 2926882521}, {'type': 'v1', 'addr': '172.21.15.71:6823', 'nonce': 2926882521}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6820', 'nonce': 2926882521}, {'type': 'v1', 'addr': '172.21.15.71:6821', 'nonce': 2926882521}]}, 'public_addr': '172.21.15.71:6817/2926882521', 'cluster_addr': '172.21.15.71:6819/2926882521', 'heartbeat_back_addr': '172.21.15.71:6823/2926882521', 'heartbeat_front_addr': '172.21.15.71:6821/2926882521', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'f4512e48-63f1-42b6-81be-594bd82e3210', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 72, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6808', 'nonce': 558942315}, {'type': 'v1', 'addr': '172.21.15.89:6809', 'nonce': 558942315}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6810', 'nonce': 558942315}, {'type': 'v1', 'addr': '172.21.15.89:6811', 'nonce': 558942315}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6814', 'nonce': 558942315}, {'type': 'v1', 'addr': '172.21.15.89:6815', 'nonce': 558942315}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6812', 'nonce': 558942315}, {'type': 'v1', 'addr': '172.21.15.89:6813', 'nonce': 558942315}]}, 'public_addr': '172.21.15.89:6809/558942315', 'cluster_addr': '172.21.15.89:6811/558942315', 'heartbeat_back_addr': '172.21.15.89:6815/558942315', 'heartbeat_front_addr': '172.21.15.89:6813/558942315', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '52d47b6f-af30-455e-8326-e53c598911ae', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 60, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6816', 'nonce': 3878488003}, {'type': 'v1', 'addr': '172.21.15.89:6817', 'nonce': 3878488003}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6818', 'nonce': 3878488003}, {'type': 'v1', 'addr': '172.21.15.89:6819', 'nonce': 3878488003}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6822', 'nonce': 3878488003}, {'type': 'v1', 'addr': '172.21.15.89:6823', 'nonce': 3878488003}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6820', 'nonce': 3878488003}, {'type': 'v1', 'addr': '172.21.15.89:6821', 'nonce': 3878488003}]}, 'public_addr': '172.21.15.89:6817/3878488003', 'cluster_addr': '172.21.15.89:6819/3878488003', 'heartbeat_back_addr': '172.21.15.89:6823/3878488003', 'heartbeat_front_addr': '172.21.15.89:6821/3878488003', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '6b4f9416-0ba0-4c58-883e-8cbdb362adcc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 80, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6800', 'nonce': 2840204107}, {'type': 'v1', 'addr': '172.21.15.89:6801', 'nonce': 2840204107}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6802', 'nonce': 2840204107}, {'type': 'v1', 'addr': '172.21.15.89:6803', 'nonce': 2840204107}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6806', 'nonce': 2840204107}, {'type': 'v1', 'addr': '172.21.15.89:6807', 'nonce': 2840204107}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6804', 'nonce': 2840204107}, {'type': 'v1', 'addr': '172.21.15.89:6805', 'nonce': 2840204107}]}, 'public_addr': '172.21.15.89:6801/2840204107', 'cluster_addr': '172.21.15.89:6803/2840204107', 'heartbeat_back_addr': '172.21.15.89:6807/2840204107', 'heartbeat_front_addr': '172.21.15.89:6805/2840204107', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T06:49:00.282785+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T06:49:00.381259+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T06:49:00.986977+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T06:49:00.732329+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T06:49:00.844988+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T06:49:00.639989+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-11T06:50:56.838 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:50:56.838 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:56.838 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:56.838 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is set 2024-09-11T06:50:56.839 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:50:56.840 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:56.830+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:56.840 INFO:tasks.workunit.client.0.smithi176.stderr: Creating snap: 10% complete... Creating snap: 20% complete... Creating snap: 100% complete...done. 2024-09-11T06:50:56.846 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo dd if=/tmp/tmp.IGWYJYR7sz/data of=/dev/nbd1 bs=1M count=1 oflag=direct 2024-09-11T06:50:56.846 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:56.846 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:56.848 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:56.849 INFO:tasks.workunit.client.0.smithi176.stderr:++ which dd 2024-09-11T06:50:56.850 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/dd 2024-09-11T06:50:56.850 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:56.851 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/dd if=/tmp/tmp.IGWYJYR7sz/data of=/dev/nbd1 bs=1M count=1 oflag=direct 2024-09-11T06:50:56.877 INFO:tasks.workunit.client.0.smithi176.stderr:1+0 records in 2024-09-11T06:50:56.877 INFO:tasks.workunit.client.0.smithi176.stderr:1+0 records out 2024-09-11T06:50:56.877 INFO:tasks.workunit.client.0.smithi176.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160068 s, 65.5 MB/s 2024-09-11T06:50:56.877 INFO:tasks.workunit.client.0.smithi176.stderr:+ unmap_device /dev/nbd1 23729 2024-09-11T06:50:56.877 INFO:tasks.workunit.client.0.smithi176.stderr:+ local args=/dev/nbd1 2024-09-11T06:50:56.877 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pid=23729 2024-09-11T06:50:56.877 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd1 2024-09-11T06:50:56.878 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:50:56.878 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:50:56.878 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:50:56.878 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:50:56.878 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:50:56.879 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:50:56.879 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd1 2024-09-11T06:50:56.938 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:56.935+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:57.007 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:57.007 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:57.021 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:50:57.039 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:57.035+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:57.125 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:50:57.132 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:57.132 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:57.141 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:57.139+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:57.146 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:50:57.172 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:50:57.172 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:50:57.189 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-09-11T06:50:57.037377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":136905559,"num_objects":57,"num_object_clones":2,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":3246,"num_read_kb":110223,"num_write":750,"num_write_kb":267831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":186911175,"num_keys_recovered":22,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":706,"ondisk_log_size":706,"up":34,"acting":34,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":456156,"kb_used_data":273188,"kb_used_omap":9,"kb_used_meta":182838,"kb_avail":565774884,"statfs":{"total":579820584960,"available":579353481216,"internally_reserved":0,"allocated":279744512,"data_stored":288615107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9565,"internal_metadata":187226787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.488529"},"pg_stats":[{"pgid":"1.0","version":"14'117","reported_seq":306,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T06:50:53.810974+0000","last_change":"2024-09-11T06:50:37.941212+0000","last_active":"2024-09-11T06:50:53.810974+0000","last_peered":"2024-09-11T06:50:53.810974+0000","last_clean":"2024-09-11T06:50:53.810974+0000","last_became_active":"2024-09-11T06:50:37.940448+0000","last_became_peered":"2024-09-11T06:50:37.940448+0000","last_unstale":"2024-09-11T06:50:53.810974+0000","last_undegraded":"2024-09-11T06:50:53.810974+0000","last_fullsized":"2024-09-11T06:50:53.810974+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_clean_scrub_stamp":"2024-09-11T06:49:05.088839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:48:59.833858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"72'39","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T06:50:54.006142+0000","last_change":"2024-09-11T06:50:47.401354+0000","last_active":"2024-09-11T06:50:54.006142+0000","last_peered":"2024-09-11T06:50:54.006142+0000","last_clean":"2024-09-11T06:50:54.006142+0000","last_became_active":"2024-09-11T06:50:47.306211+0000","last_became_peered":"2024-09-11T06:50:47.306211+0000","last_unstale":"2024-09-11T06:50:54.006142+0000","last_undegraded":"2024-09-11T06:50:54.006142+0000","last_fullsized":"2024-09-11T06:50:54.006142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:40:38.848844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011212469999999999,"stat_sum":{"num_bytes":12582950,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":70,"num_read_kb":8196,"num_write":38,"num_write_kb":28673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":25165862,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"58'2","reported_seq":104,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T06:50:53.941203+0000","last_change":"2024-09-11T06:50:47.310511+0000","last_active":"2024-09-11T06:50:53.941203+0000","last_peered":"2024-09-11T06:50:53.941203+0000","last_clean":"2024-09-11T06:50:53.941203+0000","last_became_active":"2024-09-11T06:50:47.304328+0000","last_became_peered":"2024-09-11T06:50:47.304328+0000","last_unstale":"2024-09-11T06:50:53.941203+0000","last_undegraded":"2024-09-11T06:50:53.941203+0000","last_fullsized":"2024-09-11T06:50:53.941203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:21:15.806857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011764950000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"68'57","reported_seq":200,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T06:50:53.811047+0000","last_change":"2024-09-11T06:50:35.668519+0000","last_active":"2024-09-11T06:50:53.811047+0000","last_peered":"2024-09-11T06:50:53.811047+0000","last_clean":"2024-09-11T06:50:53.811047+0000","last_became_active":"2024-09-11T06:50:35.238279+0000","last_became_peered":"2024-09-11T06:50:35.238279+0000","last_unstale":"2024-09-11T06:50:53.811047+0000","last_undegraded":"2024-09-11T06:50:53.811047+0000","last_fullsized":"2024-09-11T06:50:53.811047+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:50:29.844342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077615599999999996,"stat_sum":{"num_bytes":20971520,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":150,"num_read_kb":23060,"num_write":56,"num_write_kb":45056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":41943040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":70,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T06:50:54.006049+0000","last_change":"2024-09-11T06:50:19.703255+0000","last_active":"2024-09-11T06:50:54.006049+0000","last_peered":"2024-09-11T06:50:54.006049+0000","last_clean":"2024-09-11T06:50:54.006049+0000","last_became_active":"2024-09-11T06:50:17.089296+0000","last_became_peered":"2024-09-11T06:50:17.089296+0000","last_unstale":"2024-09-11T06:50:54.006049+0000","last_undegraded":"2024-09-11T06:50:54.006049+0000","last_fullsized":"2024-09-11T06:50:54.006049+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:42:18.684413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001159373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"82'126","reported_seq":1223,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T06:50:53.808662+0000","last_change":"2024-09-11T06:49:55.948809+0000","last_active":"2024-09-11T06:50:53.808662+0000","last_peered":"2024-09-11T06:50:53.808662+0000","last_clean":"2024-09-11T06:50:53.808662+0000","last_became_active":"2024-09-11T06:49:55.948499+0000","last_became_peered":"2024-09-11T06:49:55.948499+0000","last_unstale":"2024-09-11T06:50:53.808662+0000","last_undegraded":"2024-09-11T06:50:53.808662+0000","last_fullsized":"2024-09-11T06:50:53.808662+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:25.318791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010163780000000001,"stat_sum":{"num_bytes":25165862,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1614,"num_read_kb":23620,"num_write":67,"num_write_kb":40968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"79'1","reported_seq":72,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T06:50:53.941168+0000","last_change":"2024-09-11T06:50:19.097182+0000","last_active":"2024-09-11T06:50:53.941168+0000","last_peered":"2024-09-11T06:50:53.941168+0000","last_clean":"2024-09-11T06:50:53.941168+0000","last_became_active":"2024-09-11T06:50:17.097783+0000","last_became_peered":"2024-09-11T06:50:17.097783+0000","last_unstale":"2024-09-11T06:50:53.941168+0000","last_undegraded":"2024-09-11T06:50:53.941168+0000","last_fullsized":"2024-09-11T06:50:53.941168+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:35:49.244377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088028100000000001,"stat_sum":{"num_bytes":35,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"54'4","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T06:50:54.006195+0000","last_change":"2024-09-11T06:50:47.453549+0000","last_active":"2024-09-11T06:50:54.006195+0000","last_peered":"2024-09-11T06:50:54.006195+0000","last_clean":"2024-09-11T06:50:54.006195+0000","last_became_active":"2024-09-11T06:50:47.306692+0000","last_became_peered":"2024-09-11T06:50:47.306692+0000","last_unstale":"2024-09-11T06:50:54.006195+0000","last_undegraded":"2024-09-11T06:50:54.006195+0000","last_fullsized":"2024-09-11T06:50:54.006195+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:58:34.794461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001910244,"stat_sum":{"num_bytes":8388627,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":8194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":8388684,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"56'1","reported_seq":71,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T06:50:54.006077+0000","last_change":"2024-09-11T06:50:19.703261+0000","last_active":"2024-09-11T06:50:54.006077+0000","last_peered":"2024-09-11T06:50:54.006077+0000","last_clean":"2024-09-11T06:50:54.006077+0000","last_became_active":"2024-09-11T06:50:17.089261+0000","last_became_peered":"2024-09-11T06:50:17.089261+0000","last_unstale":"2024-09-11T06:50:54.006077+0000","last_undegraded":"2024-09-11T06:50:54.006077+0000","last_fullsized":"2024-09-11T06:50:54.006077+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:00:38.491173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001141853,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T06:50:54.006350+0000","last_change":"2024-09-11T06:50:47.312793+0000","last_active":"2024-09-11T06:50:54.006350+0000","last_peered":"2024-09-11T06:50:54.006350+0000","last_clean":"2024-09-11T06:50:54.006350+0000","last_became_active":"2024-09-11T06:50:47.305853+0000","last_became_peered":"2024-09-11T06:50:47.305853+0000","last_unstale":"2024-09-11T06:50:54.006350+0000","last_undegraded":"2024-09-11T06:50:54.006350+0000","last_fullsized":"2024-09-11T06:50:54.006350+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:59:39.429938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"82'112","reported_seq":285,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T06:50:53.941343+0000","last_change":"2024-09-11T06:49:56.122331+0000","last_active":"2024-09-11T06:50:53.941343+0000","last_peered":"2024-09-11T06:50:53.941343+0000","last_clean":"2024-09-11T06:50:53.941343+0000","last_became_active":"2024-09-11T06:49:56.121998+0000","last_became_peered":"2024-09-11T06:49:56.121998+0000","last_unstale":"2024-09-11T06:50:53.941343+0000","last_undegraded":"2024-09-11T06:50:53.941343+0000","last_fullsized":"2024-09-11T06:50:53.941343+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:53:28.328404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582950,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":524,"num_read_kb":8599,"num_write":178,"num_write_kb":20619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"79'4","reported_seq":85,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T06:50:53.941330+0000","last_change":"2024-09-11T06:50:19.097504+0000","last_active":"2024-09-11T06:50:53.941330+0000","last_peered":"2024-09-11T06:50:53.941330+0000","last_clean":"2024-09-11T06:50:53.941330+0000","last_became_active":"2024-09-11T06:50:17.096243+0000","last_became_peered":"2024-09-11T06:50:17.096243+0000","last_unstale":"2024-09-11T06:50:53.941330+0000","last_undegraded":"2024-09-11T06:50:53.941330+0000","last_fullsized":"2024-09-11T06:50:53.941330+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:44:45.211005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085501399999999997,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":25,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"82'86","reported_seq":247,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T06:50:53.811115+0000","last_change":"2024-09-11T06:50:35.337013+0000","last_active":"2024-09-11T06:50:53.811115+0000","last_peered":"2024-09-11T06:50:53.811115+0000","last_clean":"2024-09-11T06:50:53.811115+0000","last_became_active":"2024-09-11T06:50:35.236333+0000","last_became_peered":"2024-09-11T06:50:35.236333+0000","last_unstale":"2024-09-11T06:50:53.811115+0000","last_undegraded":"2024-09-11T06:50:53.811115+0000","last_fullsized":"2024-09-11T06:50:53.811115+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:37:23.719780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095172900000000001,"stat_sum":{"num_bytes":23068704,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":340,"num_read_kb":17580,"num_write":84,"num_write_kb":39939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":41943104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":71,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T06:50:53.807487+0000","last_change":"2024-09-11T06:50:19.096344+0000","last_active":"2024-09-11T06:50:53.807487+0000","last_peered":"2024-09-11T06:50:53.807487+0000","last_clean":"2024-09-11T06:50:53.807487+0000","last_became_active":"2024-09-11T06:50:17.096560+0000","last_became_peered":"2024-09-11T06:50:17.096560+0000","last_unstale":"2024-09-11T06:50:53.807487+0000","last_undegraded":"2024-09-11T06:50:53.807487+0000","last_fullsized":"2024-09-11T06:50:53.807487+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:03:19.807744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053029399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"74'43","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T06:50:53.941327+0000","last_change":"2024-09-11T06:50:35.320796+0000","last_active":"2024-09-11T06:50:53.941327+0000","last_peered":"2024-09-11T06:50:53.941327+0000","last_clean":"2024-09-11T06:50:53.941327+0000","last_became_active":"2024-09-11T06:50:35.241589+0000","last_became_peered":"2024-09-11T06:50:35.241589+0000","last_unstale":"2024-09-11T06:50:53.941327+0000","last_undegraded":"2024-09-11T06:50:53.941327+0000","last_fullsized":"2024-09-11T06:50:53.941327+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:00:46.064173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388646,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":118,"num_read_kb":8286,"num_write":59,"num_write_kb":16420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":16777292,"num_keys_recovered":6,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":71,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T06:50:53.808048+0000","last_change":"2024-09-11T06:50:19.688101+0000","last_active":"2024-09-11T06:50:53.808048+0000","last_peered":"2024-09-11T06:50:53.808048+0000","last_clean":"2024-09-11T06:50:53.808048+0000","last_became_active":"2024-09-11T06:50:17.096959+0000","last_became_peered":"2024-09-11T06:50:17.096959+0000","last_unstale":"2024-09-11T06:50:53.808048+0000","last_undegraded":"2024-09-11T06:50:53.808048+0000","last_fullsized":"2024-09-11T06:50:53.808048+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:54:30.491263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059833200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"82'113","reported_seq":158,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T06:50:53.811139+0000","last_change":"2024-09-11T06:50:35.519269+0000","last_active":"2024-09-11T06:50:53.811139+0000","last_peered":"2024-09-11T06:50:53.811139+0000","last_clean":"2024-09-11T06:50:53.811139+0000","last_became_active":"2024-09-11T06:50:35.238011+0000","last_became_peered":"2024-09-11T06:50:35.238011+0000","last_unstale":"2024-09-11T06:50:53.811139+0000","last_undegraded":"2024-09-11T06:50:53.811139+0000","last_fullsized":"2024-09-11T06:50:53.811139+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:26.423095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001403348,"stat_sum":{"num_bytes":25165862,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":267,"num_read_kb":20741,"num_write":99,"num_write_kb":65540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":50331648,"num_keys_recovered":16,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":70,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":31,"num_read_kb":25,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":73,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":16,"acting":16,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":136315121,"num_objects":49,"num_object_clones":2,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":3099,"num_read_kb":110098,"num_write":585,"num_write_kb":265409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":184549630,"num_keys_recovered":22,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":272695296,"data_stored":285213154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":580,"ondisk_log_size":580,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640280,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48484,"kb_used_data":21456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94323356,"statfs":{"total":96636764160,"available":96587116544,"internally_reserved":0,"allocated":21970944,"data_stored":21345909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640281,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56716,"kb_used_data":25528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94315124,"statfs":{"total":96636764160,"available":96578686976,"internally_reserved":0,"allocated":26140672,"data_stored":25535218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640281,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":108508,"kb_used_data":77320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94263332,"statfs":{"total":96636764160,"available":96525651968,"internally_reserved":0,"allocated":79175680,"data_stored":84845912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640282,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69016,"kb_used_data":37828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94302824,"statfs":{"total":96636764160,"available":96566091776,"internally_reserved":0,"allocated":38735872,"data_stored":38118219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640282,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105888,"kb_used_data":74700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94265952,"statfs":{"total":96636764160,"available":96528334848,"internally_reserved":0,"allocated":76492800,"data_stored":75867015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640283,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67544,"kb_used_data":36356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94304296,"statfs":{"total":96636764160,"available":96567599104,"internally_reserved":0,"allocated":37228544,"data_stored":42902834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":35663872,"data_stored":41943110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":75513856,"data_stored":75497605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":37756928,"data_stored":37748812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":77611008,"data_stored":83886188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":25169920,"data_stored":25165862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20979712,"data_stored":20971577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":54,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":51,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":35,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:50:57.191 INFO:tasks.thrashosds.thrasher:Setting 3.5 to [5, 1, 2, 3] 2024-09-11T06:50:57.191 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.5', '5', '1', '2', '3'] 2024-09-11T06:50:57.191 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.5 5 1 2 3 2024-09-11T06:50:57.282 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:57.282 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:57.283 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:57.278+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:57.290 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:50:57.290 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:50:57.296 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:50:57.310 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:50:57.310 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:50:57.327 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:50:57.327 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:50:57.343 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:57.339+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:57.358 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:50:57.358 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:50:57.379 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:50:57.379 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:50:57.399 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:50:57.399 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:50:57.439 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:57.439 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:57.439 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:50:57.439 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:57.443 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:57.438+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:57.452 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:50:57.545 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:57.543+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:57.616 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:57.616 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:50:57.616 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:50:57.616 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:50:57.616 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:57.629 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:50:57.646 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:57.643+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:57.747 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:57.742+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:57.751 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:50:57.751 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:50:57.751 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:50:57.751 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:50:57.751 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.751 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:20 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:57.751 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.717149+0000", 2024-09-11T06:50:57.751 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.029432898, 2024-09-11T06:50:57.751 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002913297, 2024-09-11T06:50:57.751 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:57.751 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:57.751 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:57.751 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:57.752 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:57.752 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:57.752 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 20 2024-09-11T06:50:57.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.752 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:57.752 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.752 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:57.752 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717149+0000", 2024-09-11T06:50:57.752 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.753 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:57.753 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717149+0000", 2024-09-11T06:50:57.753 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.753 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:57.753 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717152+0000", 2024-09-11T06:50:57.753 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.21e-06 2024-09-11T06:50:57.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.753 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:57.754 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717164+0000", 2024-09-11T06:50:57.754 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1912999999999999e-05 2024-09-11T06:50:57.754 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.754 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.754 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:57.754 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717191+0000", 2024-09-11T06:50:57.754 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7526e-05 2024-09-11T06:50:57.754 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.754 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.754 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:57.754 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717206+0000", 2024-09-11T06:50:57.754 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5113000000000001e-05 2024-09-11T06:50:57.754 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.754 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.754 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:57.755 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717355+0000", 2024-09-11T06:50:57.755 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014896500000000001 2024-09-11T06:50:57.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.755 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:57.755 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717607+0000", 2024-09-11T06:50:57.755 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000252017 2024-09-11T06:50:57.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.755 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:57.755 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.718128+0000", 2024-09-11T06:50:57.755 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00052114499999999998 2024-09-11T06:50:57.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.756 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:57.756 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.719432+0000", 2024-09-11T06:50:57.756 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013041629999999999 2024-09-11T06:50:57.756 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.756 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.756 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:57.756 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.719864+0000", 2024-09-11T06:50:57.756 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00043115299999999998 2024-09-11T06:50:57.756 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.756 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.756 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:57.756 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.719949+0000", 2024-09-11T06:50:57.756 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4862000000000002e-05 2024-09-11T06:50:57.756 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.756 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.757 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:57.757 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.720063+0000", 2024-09-11T06:50:57.757 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011423 2024-09-11T06:50:57.757 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.757 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:57.757 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.757 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.757 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:26 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 140250278007648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:57.757 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:50:57.757 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.016949763, 2024-09-11T06:50:57.757 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034645140000000001, 2024-09-11T06:50:57.757 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:57.757 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:57.757 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:57.758 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:57.758 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:57.758 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:57.758 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:50:57.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.758 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:57.758 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.758 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:57.758 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:50:57.758 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.758 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.758 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:57.758 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:50:57.758 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.759 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:57.759 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729634+0000", 2024-09-11T06:50:57.759 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0470000000000001e-06 2024-09-11T06:50:57.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.759 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:57.759 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729640+0000", 2024-09-11T06:50:57.759 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8860000000000001e-06 2024-09-11T06:50:57.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.759 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:57.760 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729654+0000", 2024-09-11T06:50:57.760 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:50:57.760 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.760 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.760 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:57.761 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729664+0000", 2024-09-11T06:50:57.761 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.594e-06 2024-09-11T06:50:57.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.761 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.761 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:57.761 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730422+0000", 2024-09-11T06:50:57.761 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000758648 2024-09-11T06:50:57.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.761 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.761 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:57.761 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730571+0000", 2024-09-11T06:50:57.761 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000148932 2024-09-11T06:50:57.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.762 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:57.762 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730936+0000", 2024-09-11T06:50:57.762 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036510900000000002 2024-09-11T06:50:57.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.762 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:57.762 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732159+0000", 2024-09-11T06:50:57.762 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001222302 2024-09-11T06:50:57.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.762 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:57.762 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732763+0000", 2024-09-11T06:50:57.762 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00060421599999999998 2024-09-11T06:50:57.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.763 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.763 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:57.763 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732916+0000", 2024-09-11T06:50:57.763 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015264 2024-09-11T06:50:57.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.763 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.763 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:57.763 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733097+0000", 2024-09-11T06:50:57.763 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000181504 2024-09-11T06:50:57.763 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.763 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:57.764 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.764 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:27 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:57.764 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:50:57.764 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.015975861000001, 2024-09-11T06:50:57.764 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003072852, 2024-09-11T06:50:57.764 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:57.764 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:57.764 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:57.764 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:57.764 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:50:57.764 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:50:57.764 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27 2024-09-11T06:50:57.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.765 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:57.765 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.765 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:57.765 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:50:57.765 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.765 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.765 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:57.765 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:50:57.765 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.765 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.765 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:57.765 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730608+0000", 2024-09-11T06:50:57.765 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8369999999999999e-06 2024-09-11T06:50:57.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.766 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:57.766 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730617+0000", 2024-09-11T06:50:57.766 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1859999999999992e-06 2024-09-11T06:50:57.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.766 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:57.766 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730636+0000", 2024-09-11T06:50:57.766 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9296e-05 2024-09-11T06:50:57.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.766 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:57.766 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730674+0000", 2024-09-11T06:50:57.766 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8609000000000003e-05 2024-09-11T06:50:57.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.767 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:57.767 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731496+0000", 2024-09-11T06:50:57.767 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00082112299999999999 2024-09-11T06:50:57.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.767 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:57.767 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731722+0000", 2024-09-11T06:50:57.767 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022651400000000001 2024-09-11T06:50:57.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.767 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:57.768 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733215+0000", 2024-09-11T06:50:57.768 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001493257 2024-09-11T06:50:57.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.768 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.768 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:57.768 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733452+0000", 2024-09-11T06:50:57.768 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023631600000000001 2024-09-11T06:50:57.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.768 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.768 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:57.768 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733679+0000", 2024-09-11T06:50:57.768 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022771400000000001 2024-09-11T06:50:57.768 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.768 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:57.769 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.769 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.769 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:73 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:57.769 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:50:57.769 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.015891577000001, 2024-09-11T06:50:57.769 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033454439999999999, 2024-09-11T06:50:57.769 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:57.769 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:57.769 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:57.769 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:57.769 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:57.769 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:57.769 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73 2024-09-11T06:50:57.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.770 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:57.770 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.770 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:57.770 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:50:57.770 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.770 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.770 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.770 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:57.770 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:50:57.770 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.770 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.771 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:57.771 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730694+0000", 2024-09-11T06:50:57.771 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5099999999999999e-06 2024-09-11T06:50:57.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.771 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:57.771 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730708+0000", 2024-09-11T06:50:57.771 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3237e-05 2024-09-11T06:50:57.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.772 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:57.772 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730738+0000", 2024-09-11T06:50:57.772 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0688999999999997e-05 2024-09-11T06:50:57.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.772 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:57.772 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730759+0000", 2024-09-11T06:50:57.772 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0818999999999998e-05 2024-09-11T06:50:57.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.772 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:57.772 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731841+0000", 2024-09-11T06:50:57.772 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010821819999999999 2024-09-11T06:50:57.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.773 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:57.773 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732092+0000", 2024-09-11T06:50:57.773 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025086299999999999 2024-09-11T06:50:57.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.773 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:57.773 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733730+0000", 2024-09-11T06:50:57.773 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0016374600000000001 2024-09-11T06:50:57.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.773 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:57.773 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733866+0000", 2024-09-11T06:50:57.773 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000136666 2024-09-11T06:50:57.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.774 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:57.774 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.734036+0000", 2024-09-11T06:50:57.774 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000170018 2024-09-11T06:50:57.774 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.774 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:57.774 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.774 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:18 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:57.774 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:50:57.774 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 26.872474388000001, 2024-09-11T06:50:57.774 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0031646579999999999, 2024-09-11T06:50:57.774 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:57.775 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:57.775 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:57.775 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:57.775 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:57.775 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:57.775 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18 2024-09-11T06:50:57.775 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.775 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:57.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.775 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:57.775 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:50:57.775 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.775 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:57.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:50:57.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:57.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874109+0000", 2024-09-11T06:50:57.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.426e-06 2024-09-11T06:50:57.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:57.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874118+0000", 2024-09-11T06:50:57.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6929999999999996e-06 2024-09-11T06:50:57.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.777 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:57.777 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874134+0000", 2024-09-11T06:50:57.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5503e-05 2024-09-11T06:50:57.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.777 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:57.777 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874144+0000", 2024-09-11T06:50:57.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0207000000000001e-05 2024-09-11T06:50:57.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.777 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:57.777 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874545+0000", 2024-09-11T06:50:57.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040120800000000001 2024-09-11T06:50:57.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.778 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:57.778 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874686+0000", 2024-09-11T06:50:57.778 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014128599999999999 2024-09-11T06:50:57.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.778 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:57.778 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875023+0000", 2024-09-11T06:50:57.778 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033706600000000001 2024-09-11T06:50:57.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.778 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:57.778 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.876217+0000", 2024-09-11T06:50:57.778 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011938999999999999 2024-09-11T06:50:57.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.779 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.779 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:57.779 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877042+0000", 2024-09-11T06:50:57.779 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00082499899999999998 2024-09-11T06:50:57.779 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.779 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.779 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:57.779 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877128+0000", 2024-09-11T06:50:57.779 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5903999999999996e-05 2024-09-11T06:50:57.779 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.779 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.779 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:57.779 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877273+0000", 2024-09-11T06:50:57.779 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144466 2024-09-11T06:50:57.780 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.780 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:57.780 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.780 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.780 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.780 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:24 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:57.780 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:50:57.780 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 26.857968681999999, 2024-09-11T06:50:57.780 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002980271, 2024-09-11T06:50:57.780 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:57.780 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:57.780 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:57.780 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:57.780 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:57.781 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:57.781 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 24 2024-09-11T06:50:57.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.781 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:57.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:57.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:50:57.782 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.782 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:57.782 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:50:57.782 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.782 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:57.782 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888616+0000", 2024-09-11T06:50:57.782 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0030000000000001e-06 2024-09-11T06:50:57.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.783 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:57.783 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888626+0000", 2024-09-11T06:50:57.783 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0502999999999999e-05 2024-09-11T06:50:57.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.783 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:57.783 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888655+0000", 2024-09-11T06:50:57.783 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8419999999999999e-05 2024-09-11T06:50:57.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.783 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:57.783 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888688+0000", 2024-09-11T06:50:57.783 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3212e-05 2024-09-11T06:50:57.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.784 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.784 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:57.784 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888829+0000", 2024-09-11T06:50:57.784 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014114299999999999 2024-09-11T06:50:57.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.784 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.784 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:57.784 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.889082+0000", 2024-09-11T06:50:57.784 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025279499999999999 2024-09-11T06:50:57.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.784 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.784 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:57.784 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.889577+0000", 2024-09-11T06:50:57.785 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00049536299999999997 2024-09-11T06:50:57.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.785 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.785 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:57.785 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.890811+0000", 2024-09-11T06:50:57.785 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001233858 2024-09-11T06:50:57.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.785 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.785 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:57.785 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891368+0000", 2024-09-11T06:50:57.785 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00055677400000000001 2024-09-11T06:50:57.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.785 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.786 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:57.786 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891456+0000", 2024-09-11T06:50:57.786 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8246999999999997e-05 2024-09-11T06:50:57.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.786 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:57.786 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891594+0000", 2024-09-11T06:50:57.786 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013795300000000001 2024-09-11T06:50:57.786 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.786 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:57.786 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.787 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:26 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 139740787512064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:57.787 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:50:57.787 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 26.854266037999999, 2024-09-11T06:50:57.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003613733, 2024-09-11T06:50:57.787 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:57.787 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:57.787 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:57.787 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:57.787 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:57.787 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:57.787 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:50:57.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.788 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:57.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.788 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:57.788 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:50:57.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.788 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:57.788 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:50:57.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.788 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:57.789 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892318+0000", 2024-09-11T06:50:57.789 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.623e-06 2024-09-11T06:50:57.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.789 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.789 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:57.789 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892325+0000", 2024-09-11T06:50:57.789 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:50:57.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.789 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.789 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:57.789 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892346+0000", 2024-09-11T06:50:57.789 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1246e-05 2024-09-11T06:50:57.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.790 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.790 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:57.790 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892362+0000", 2024-09-11T06:50:57.790 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5409000000000001e-05 2024-09-11T06:50:57.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.790 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.790 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:57.790 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892883+0000", 2024-09-11T06:50:57.790 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00052087599999999998 2024-09-11T06:50:57.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.790 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.790 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:57.790 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893123+0000", 2024-09-11T06:50:57.790 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000240769 2024-09-11T06:50:57.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.791 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.791 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:57.791 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893526+0000", 2024-09-11T06:50:57.791 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040300200000000002 2024-09-11T06:50:57.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.791 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.791 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:57.791 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894820+0000", 2024-09-11T06:50:57.791 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012934159999999999 2024-09-11T06:50:57.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.791 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.791 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:57.792 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895608+0000", 2024-09-11T06:50:57.792 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00078788100000000004 2024-09-11T06:50:57.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.792 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:57.792 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895724+0000", 2024-09-11T06:50:57.792 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011595999999999999 2024-09-11T06:50:57.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.793 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.793 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:57.793 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895930+0000", 2024-09-11T06:50:57.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000206391 2024-09-11T06:50:57.793 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.793 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:57.793 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.793 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.793 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:133 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:57.793 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:50:57.793 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 26.853468021000001, 2024-09-11T06:50:57.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035017360000000001, 2024-09-11T06:50:57.793 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:57.794 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:57.794 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:57.794 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:57.794 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:57.794 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:57.794 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 133 2024-09-11T06:50:57.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.794 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:57.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.794 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:57.794 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:50:57.794 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.794 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:57.794 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:50:57.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.795 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:57.795 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893116+0000", 2024-09-11T06:50:57.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1629999999999999e-06 2024-09-11T06:50:57.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.795 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:57.795 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893128+0000", 2024-09-11T06:50:57.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.11e-05 2024-09-11T06:50:57.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.795 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:57.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893156+0000", 2024-09-11T06:50:57.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8266e-05 2024-09-11T06:50:57.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:57.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893171+0000", 2024-09-11T06:50:57.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5426000000000001e-05 2024-09-11T06:50:57.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:57.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894561+0000", 2024-09-11T06:50:57.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013899959999999999 2024-09-11T06:50:57.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:57.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894781+0000", 2024-09-11T06:50:57.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021987100000000001 2024-09-11T06:50:57.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:57.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896349+0000", 2024-09-11T06:50:57.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015681779999999999 2024-09-11T06:50:57.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:57.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896458+0000", 2024-09-11T06:50:57.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000108187 2024-09-11T06:50:57.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:57.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896616+0000", 2024-09-11T06:50:57.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015854899999999999 2024-09-11T06:50:57.798 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.798 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:57.798 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.798 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:27 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:57.798 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:50:57.798 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 26.853456641000001, 2024-09-11T06:50:57.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003151989, 2024-09-11T06:50:57.799 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:57.799 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:57.799 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:57.799 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:57.799 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:50:57.799 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:50:57.799 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27 2024-09-11T06:50:57.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.799 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:57.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.799 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:57.799 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:50:57.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.800 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.800 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:57.800 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:50:57.800 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.800 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.800 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:57.800 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893128+0000", 2024-09-11T06:50:57.800 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.013e-06 2024-09-11T06:50:57.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.800 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.800 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:57.800 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893136+0000", 2024-09-11T06:50:57.800 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4190000000000002e-06 2024-09-11T06:50:57.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.801 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:57.801 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893158+0000", 2024-09-11T06:50:57.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1506999999999999e-05 2024-09-11T06:50:57.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.801 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:57.801 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893170+0000", 2024-09-11T06:50:57.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2741999999999999e-05 2024-09-11T06:50:57.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:57.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894255+0000", 2024-09-11T06:50:57.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001084453 2024-09-11T06:50:57.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:57.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894461+0000", 2024-09-11T06:50:57.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00020586700000000001 2024-09-11T06:50:57.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.803 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:57.803 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896033+0000", 2024-09-11T06:50:57.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001572092 2024-09-11T06:50:57.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.803 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:57.803 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896149+0000", 2024-09-11T06:50:57.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116636 2024-09-11T06:50:57.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.804 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:57.804 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896278+0000", 2024-09-11T06:50:57.804 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012826 2024-09-11T06:50:57.804 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.804 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:57.804 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.804 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:176 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch reconnect cookie 139735284585312 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:50:57.804 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:50:57.804 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 26.822089821999999, 2024-09-11T06:50:57.804 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033620960000000002, 2024-09-11T06:50:57.805 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:57.805 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:57.805 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:57.805 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:57.805 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:50:57.805 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:50:57.805 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 176 2024-09-11T06:50:57.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.805 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:57.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.805 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:57.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:50:57.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:57.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:50:57.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:57.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924494+0000", 2024-09-11T06:50:57.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.967e-06 2024-09-11T06:50:57.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.807 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:57.807 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924501+0000", 2024-09-11T06:50:57.807 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2659999999999998e-06 2024-09-11T06:50:57.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.807 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:57.807 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924520+0000", 2024-09-11T06:50:57.807 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9038999999999998e-05 2024-09-11T06:50:57.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.808 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.808 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:57.808 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924535+0000", 2024-09-11T06:50:57.808 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5359999999999999e-05 2024-09-11T06:50:57.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.808 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.808 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:57.808 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.925544+0000", 2024-09-11T06:50:57.808 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010085109999999999 2024-09-11T06:50:57.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.809 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:57.809 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.925671+0000", 2024-09-11T06:50:57.809 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012756599999999999 2024-09-11T06:50:57.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.809 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:57.809 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.927855+0000", 2024-09-11T06:50:57.809 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0021833870000000002 2024-09-11T06:50:57.809 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.809 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:57.809 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.810 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4828.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch watch cookie 140587969811264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-11T06:50:57.810 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:50:57.810 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.500386863999999, 2024-09-11T06:50:57.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032778859999999998, 2024-09-11T06:50:57.810 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:57.810 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:57.810 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:57.810 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:57.810 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4828", 2024-09-11T06:50:57.810 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2410288650", 2024-09-11T06:50:57.810 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:50:57.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.811 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:57.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.811 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:57.811 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:50:57.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.811 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:57.811 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:50:57.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.811 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:57.811 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246198+0000", 2024-09-11T06:50:57.812 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.667e-06 2024-09-11T06:50:57.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.812 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:57.812 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246209+0000", 2024-09-11T06:50:57.812 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0409e-05 2024-09-11T06:50:57.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.812 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:57.812 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246241+0000", 2024-09-11T06:50:57.812 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2592999999999998e-05 2024-09-11T06:50:57.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.813 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:57.813 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246260+0000", 2024-09-11T06:50:57.813 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9343000000000001e-05 2024-09-11T06:50:57.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.813 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:57.813 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246466+0000", 2024-09-11T06:50:57.813 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000205701 2024-09-11T06:50:57.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.814 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:57.814 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246713+0000", 2024-09-11T06:50:57.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024646999999999999 2024-09-11T06:50:57.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.814 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:57.814 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.247133+0000", 2024-09-11T06:50:57.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000420536 2024-09-11T06:50:57.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.814 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:57.814 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.248541+0000", 2024-09-11T06:50:57.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001407801 2024-09-11T06:50:57.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:57.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249241+0000", 2024-09-11T06:50:57.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070043500000000001 2024-09-11T06:50:57.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:57.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249326+0000", 2024-09-11T06:50:57.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5037999999999995e-05 2024-09-11T06:50:57.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:57.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249473+0000", 2024-09-11T06:50:57.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014689299999999999 2024-09-11T06:50:57.815 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.815 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:57.816 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.816 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5313.0:25 2.1 2:967e1786:ns::rbd_object_map.11adc335555f.0000000000000004:head [writefull 0~38] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:50:57.816 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.256788+0000", 2024-09-11T06:50:57.816 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.489793864999999, 2024-09-11T06:50:57.816 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0030446449999999999, 2024-09-11T06:50:57.816 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:57.816 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:57.816 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:57.816 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:57.816 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5313", 2024-09-11T06:50:57.816 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/624314229", 2024-09-11T06:50:57.816 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 25 2024-09-11T06:50:57.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.817 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:57.817 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.817 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:57.817 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256788+0000", 2024-09-11T06:50:57.817 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.817 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.817 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:57.817 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256788+0000", 2024-09-11T06:50:57.817 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.817 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.817 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:57.817 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256791+0000", 2024-09-11T06:50:57.817 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5770000000000001e-06 2024-09-11T06:50:57.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.818 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:57.818 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256802+0000", 2024-09-11T06:50:57.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0742e-05 2024-09-11T06:50:57.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.818 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:57.818 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256823+0000", 2024-09-11T06:50:57.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0999999999999999e-05 2024-09-11T06:50:57.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:57.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256841+0000", 2024-09-11T06:50:57.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8099000000000001e-05 2024-09-11T06:50:57.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:57.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256988+0000", 2024-09-11T06:50:57.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014756400000000001 2024-09-11T06:50:57.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:57.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.257234+0000", 2024-09-11T06:50:57.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024524399999999999 2024-09-11T06:50:57.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:57.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.257446+0000", 2024-09-11T06:50:57.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000212241 2024-09-11T06:50:57.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:57.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259292+0000", 2024-09-11T06:50:57.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001845937 2024-09-11T06:50:57.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:57.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259681+0000", 2024-09-11T06:50:57.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00038940699999999998 2024-09-11T06:50:57.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.821 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:57.821 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259770+0000", 2024-09-11T06:50:57.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8385000000000002e-05 2024-09-11T06:50:57.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.821 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:57.821 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259833+0000", 2024-09-11T06:50:57.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3448999999999995e-05 2024-09-11T06:50:57.821 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.821 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:57.821 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.822 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.822 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5328.0:14 2.1 2:967e1786:ns::rbd_object_map.11adc335555f.0000000000000004:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:50:57.822 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.396257+0000", 2024-09-11T06:50:57.822 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.350325416, 2024-09-11T06:50:57.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032125370000000001, 2024-09-11T06:50:57.822 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:57.822 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:57.822 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:57.822 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:57.822 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5328", 2024-09-11T06:50:57.822 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2386493698", 2024-09-11T06:50:57.822 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 14 2024-09-11T06:50:57.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.823 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:57.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.823 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:57.823 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396257+0000", 2024-09-11T06:50:57.823 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.823 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:57.823 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396257+0000", 2024-09-11T06:50:57.823 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.824 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:57.824 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396259+0000", 2024-09-11T06:50:57.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4830000000000002e-06 2024-09-11T06:50:57.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.824 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:57.824 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396271+0000", 2024-09-11T06:50:57.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.115e-05 2024-09-11T06:50:57.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:57.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396292+0000", 2024-09-11T06:50:57.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1761999999999999e-05 2024-09-11T06:50:57.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:57.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396314+0000", 2024-09-11T06:50:57.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1982999999999999e-05 2024-09-11T06:50:57.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:57.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396409+0000", 2024-09-11T06:50:57.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4250000000000001e-05 2024-09-11T06:50:57.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.826 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:57.826 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396464+0000", 2024-09-11T06:50:57.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5884000000000002e-05 2024-09-11T06:50:57.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.826 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:57.826 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.399469+0000", 2024-09-11T06:50:57.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0030050250000000001 2024-09-11T06:50:57.826 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.826 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:57.826 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.827 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:6 3.5 3:b2cc3336:::rbd_id.testrbdnbd21607:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:50:57.827 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.591899+0000", 2024-09-11T06:50:57.827 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.154682915, 2024-09-11T06:50:57.827 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032382999999999999, 2024-09-11T06:50:57.827 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:57.827 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:57.827 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:57.827 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:57.827 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:50:57.827 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:50:57.827 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 6 2024-09-11T06:50:57.827 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.827 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:57.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:57.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591899+0000", 2024-09-11T06:50:57.828 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:57.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591899+0000", 2024-09-11T06:50:57.828 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:57.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591902+0000", 2024-09-11T06:50:57.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0770000000000002e-06 2024-09-11T06:50:57.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:57.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591913+0000", 2024-09-11T06:50:57.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1182999999999999e-05 2024-09-11T06:50:57.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:57.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591940+0000", 2024-09-11T06:50:57.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7233000000000002e-05 2024-09-11T06:50:57.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.830 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:57.830 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591954+0000", 2024-09-11T06:50:57.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4503e-05 2024-09-11T06:50:57.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.830 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:57.830 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.592071+0000", 2024-09-11T06:50:57.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116731 2024-09-11T06:50:57.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.830 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:57.830 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.592274+0000", 2024-09-11T06:50:57.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00020320900000000001 2024-09-11T06:50:57.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 3", 2024-09-11T06:50:57.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.592576+0000", 2024-09-11T06:50:57.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030179000000000002 2024-09-11T06:50:57.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:57.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.594072+0000", 2024-09-11T06:50:57.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001495496 2024-09-11T06:50:57.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:57.832 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.594945+0000", 2024-09-11T06:50:57.832 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00087379799999999996 2024-09-11T06:50:57.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:57.832 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.595023+0000", 2024-09-11T06:50:57.832 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7751999999999997e-05 2024-09-11T06:50:57.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:57.832 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.595138+0000", 2024-09-11T06:50:57.832 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000114528 2024-09-11T06:50:57.833 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.833 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:57.833 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.833 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:44 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:50:57.833 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.540846+0000", 2024-09-11T06:50:57.833 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.2057367830000008, 2024-09-11T06:50:57.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032138639999999999, 2024-09-11T06:50:57.833 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:57.833 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:57.833 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:57.833 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:57.834 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:50:57.834 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:50:57.834 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 44 2024-09-11T06:50:57.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.834 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:57.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.834 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:57.834 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540846+0000", 2024-09-11T06:50:57.834 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:57.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540846+0000", 2024-09-11T06:50:57.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:57.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540848+0000", 2024-09-11T06:50:57.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5469999999999999e-06 2024-09-11T06:50:57.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:57.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540863+0000", 2024-09-11T06:50:57.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4613000000000001e-05 2024-09-11T06:50:57.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:57.836 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540895+0000", 2024-09-11T06:50:57.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2153000000000002e-05 2024-09-11T06:50:57.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:57.836 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540912+0000", 2024-09-11T06:50:57.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7405999999999999e-05 2024-09-11T06:50:57.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:57.836 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.541111+0000", 2024-09-11T06:50:57.837 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019849599999999999 2024-09-11T06:50:57.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.837 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.837 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:57.837 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.541269+0000", 2024-09-11T06:50:57.837 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015792399999999999 2024-09-11T06:50:57.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.837 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.837 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:57.837 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.541853+0000", 2024-09-11T06:50:57.837 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00058443799999999995 2024-09-11T06:50:57.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.837 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.837 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:57.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.543133+0000", 2024-09-11T06:50:57.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012797850000000001 2024-09-11T06:50:57.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:57.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.543905+0000", 2024-09-11T06:50:57.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000771655 2024-09-11T06:50:57.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:57.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.543993+0000", 2024-09-11T06:50:57.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8168e-05 2024-09-11T06:50:57.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.839 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:57.839 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.544059+0000", 2024-09-11T06:50:57.839 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6679000000000003e-05 2024-09-11T06:50:57.839 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.839 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:57.839 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.839 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5553.0:24 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 139650593197904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:50:57.839 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.566567+0000", 2024-09-11T06:50:57.839 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.1800155130000007, 2024-09-11T06:50:57.839 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034350790000000002, 2024-09-11T06:50:57.839 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:57.839 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:57.839 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:57.840 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:57.840 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5553", 2024-09-11T06:50:57.840 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3590511604", 2024-09-11T06:50:57.840 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 24 2024-09-11T06:50:57.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.840 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:57.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.840 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:57.840 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566567+0000", 2024-09-11T06:50:57.840 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.840 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:57.840 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566567+0000", 2024-09-11T06:50:57.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:57.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566569+0000", 2024-09-11T06:50:57.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.973e-06 2024-09-11T06:50:57.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:57.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566578+0000", 2024-09-11T06:50:57.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8170000000000005e-06 2024-09-11T06:50:57.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:57.842 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566602+0000", 2024-09-11T06:50:57.842 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4596e-05 2024-09-11T06:50:57.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.842 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:57.842 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566619+0000", 2024-09-11T06:50:57.842 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7099999999999999e-05 2024-09-11T06:50:57.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.842 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:57.842 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.567321+0000", 2024-09-11T06:50:57.842 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070168200000000002 2024-09-11T06:50:57.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.842 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:57.843 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.567454+0000", 2024-09-11T06:50:57.843 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013286400000000001 2024-09-11T06:50:57.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.843 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.843 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:57.843 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.567674+0000", 2024-09-11T06:50:57.843 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000219679 2024-09-11T06:50:57.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.843 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.843 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:57.843 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.569014+0000", 2024-09-11T06:50:57.843 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013404129999999999 2024-09-11T06:50:57.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.844 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:57.844 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.569634+0000", 2024-09-11T06:50:57.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00062029200000000004 2024-09-11T06:50:57.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.844 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:57.844 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.569805+0000", 2024-09-11T06:50:57.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000170366 2024-09-11T06:50:57.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.844 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:57.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.570002+0000", 2024-09-11T06:50:57.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019729700000000001 2024-09-11T06:50:57.845 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.845 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:57.845 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.845 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:82 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch reconnect cookie 139888024456048 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:50:57.845 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.690185+0000", 2024-09-11T06:50:57.845 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.0563975110000001, 2024-09-11T06:50:57.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0045884799999999998, 2024-09-11T06:50:57.846 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:57.846 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:57.846 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:57.846 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:57.846 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:50:57.846 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:50:57.846 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 82 2024-09-11T06:50:57.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.846 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:57.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:57.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690185+0000", 2024-09-11T06:50:57.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.847 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:57.847 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690185+0000", 2024-09-11T06:50:57.847 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.847 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:57.847 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690186+0000", 2024-09-11T06:50:57.847 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8400000000000002e-07 2024-09-11T06:50:57.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.847 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:57.847 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690190+0000", 2024-09-11T06:50:57.847 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5499999999999996e-06 2024-09-11T06:50:57.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.848 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:57.848 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690203+0000", 2024-09-11T06:50:57.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2228999999999999e-05 2024-09-11T06:50:57.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.848 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:57.848 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690228+0000", 2024-09-11T06:50:57.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5587000000000001e-05 2024-09-11T06:50:57.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.848 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:57.848 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.691298+0000", 2024-09-11T06:50:57.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001070218 2024-09-11T06:50:57.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.849 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:57.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.691511+0000", 2024-09-11T06:50:57.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021301299999999999 2024-09-11T06:50:57.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.849 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:57.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.694773+0000", 2024-09-11T06:50:57.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003261899 2024-09-11T06:50:57.849 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.849 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:57.849 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.849 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:83 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify cookie 139887621709056] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:50:57.850 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.690242+0000", 2024-09-11T06:50:57.850 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.0563403390000001, 2024-09-11T06:50:57.850 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003040067, 2024-09-11T06:50:57.850 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:57.850 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:57.850 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:57.850 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:57.850 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:50:57.850 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:50:57.850 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 83 2024-09-11T06:50:57.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.850 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:57.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.850 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:57.850 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690242+0000", 2024-09-11T06:50:57.850 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:57.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690242+0000", 2024-09-11T06:50:57.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:57.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690243+0000", 2024-09-11T06:50:57.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0640000000000001e-06 2024-09-11T06:50:57.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:57.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690247+0000", 2024-09-11T06:50:57.852 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3760000000000001e-06 2024-09-11T06:50:57.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.852 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.852 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:57.852 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690258+0000", 2024-09-11T06:50:57.852 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0874e-05 2024-09-11T06:50:57.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.852 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.852 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:57.852 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690266+0000", 2024-09-11T06:50:57.852 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5429999999999996e-06 2024-09-11T06:50:57.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.852 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.853 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:57.853 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.692564+0000", 2024-09-11T06:50:57.853 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0022977010000000001 2024-09-11T06:50:57.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.853 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.853 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:57.853 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.692882+0000", 2024-09-11T06:50:57.853 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00031862699999999998 2024-09-11T06:50:57.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.854 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:57.854 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693282+0000", 2024-09-11T06:50:57.854 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000399882 2024-09-11T06:50:57.854 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.854 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:57.854 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.854 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:84 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify cookie 139887823033168] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:50:57.854 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:50:57.854 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.0563057350000005, 2024-09-11T06:50:57.854 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035622620000000001, 2024-09-11T06:50:57.854 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:57.854 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:57.855 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:50:57.855 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:57.855 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:50:57.855 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:50:57.855 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 84 2024-09-11T06:50:57.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.855 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:57.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.855 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:57.856 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:50:57.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.856 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:57.856 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:50:57.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.856 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:57.856 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:50:57.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5300000000000003e-07 2024-09-11T06:50:57.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.857 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:57.857 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690281+0000", 2024-09-11T06:50:57.857 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9330000000000003e-06 2024-09-11T06:50:57.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.857 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:57.857 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690291+0000", 2024-09-11T06:50:57.857 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7359999999999996e-06 2024-09-11T06:50:57.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.857 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:57.857 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690298+0000", 2024-09-11T06:50:57.857 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3699999999999997e-06 2024-09-11T06:50:57.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.858 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:57.858 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693380+0000", 2024-09-11T06:50:57.858 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0030811060000000001 2024-09-11T06:50:57.858 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.858 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:57.858 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693599+0000", 2024-09-11T06:50:57.858 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021947199999999999 2024-09-11T06:50:57.858 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.858 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:57.858 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693839+0000", 2024-09-11T06:50:57.858 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023989200000000001 2024-09-11T06:50:57.859 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.859 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:57.859 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.859 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.859 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5098.0:25 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 140617967560800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:50:57.859 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.697727+0000", 2024-09-11T06:50:57.859 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.0488551590000004, 2024-09-11T06:50:57.859 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035341330000000001, 2024-09-11T06:50:57.859 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:50:57.859 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:50:57.859 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:57.859 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:50:57.860 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5098", 2024-09-11T06:50:57.860 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/844541331", 2024-09-11T06:50:57.860 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 25 2024-09-11T06:50:57.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.860 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:50:57.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.860 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:50:57.860 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697727+0000", 2024-09-11T06:50:57.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.860 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:50:57.860 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697727+0000", 2024-09-11T06:50:57.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:50:57.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.861 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:50:57.861 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697729+0000", 2024-09-11T06:50:57.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.33e-06 2024-09-11T06:50:57.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.861 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:50:57.861 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697734+0000", 2024-09-11T06:50:57.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6590000000000004e-06 2024-09-11T06:50:57.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.862 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:50:57.862 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697749+0000", 2024-09-11T06:50:57.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4867e-05 2024-09-11T06:50:57.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.862 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:50:57.862 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697759+0000", 2024-09-11T06:50:57.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0073e-05 2024-09-11T06:50:57.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.862 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:50:57.862 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.698715+0000", 2024-09-11T06:50:57.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00095545799999999996 2024-09-11T06:50:57.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:50:57.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.698850+0000", 2024-09-11T06:50:57.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000135814 2024-09-11T06:50:57.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:57.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.699085+0000", 2024-09-11T06:50:57.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023507199999999999 2024-09-11T06:50:57.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:50:57.864 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.700258+0000", 2024-09-11T06:50:57.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011726600000000001 2024-09-11T06:50:57.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.864 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:57.864 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.701061+0000", 2024-09-11T06:50:57.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00080256100000000003 2024-09-11T06:50:57.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.864 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:50:57.864 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.701144+0000", 2024-09-11T06:50:57.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3188000000000004e-05 2024-09-11T06:50:57.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:50:57.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:50:57.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:50:57.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.701261+0000", 2024-09-11T06:50:57.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000117451 2024-09-11T06:50:57.865 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.866 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:57.866 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.866 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:50:57.866 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:50:57.866 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:50:57.868 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:50:57.870 INFO:teuthology.orchestra.run.smithi071.stderr:set 3.5 pg_upmap_items mapping to [5->1,2->3] 2024-09-11T06:50:57.870 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:57.862+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:57.969 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:57.966+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:58.001 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:58.001 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:58.001 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:50:58.001 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:58.017 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:50:58.070 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:58.066+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:58.171 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:58.166+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:58.189 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:58.189 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:58.190 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:50:58.190 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:50:58.190 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:58.204 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:50:58.272 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:58.266+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:58.332 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:58.332 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:58.333 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:58.333 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:50:58.333 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.333 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:58.333 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:58.333 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 112.057298076, 2024-09-11T06:50:58.333 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283336, 2024-09-11T06:50:58.333 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.333 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.333 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:58.333 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:58.333 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:58.334 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:58.334 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 87 2024-09-11T06:50:58.334 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.334 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.334 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.334 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.334 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:58.334 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.334 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.334 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.334 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.335 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:50:58.335 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.335 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.335 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.335 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.335 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271768+0000", 2024-09-11T06:50:58.335 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7599999999999996e-07 2024-09-11T06:50:58.335 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.335 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.335 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.335 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271860+0000", 2024-09-11T06:50:58.335 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2639999999999994e-05 2024-09-11T06:50:58.335 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.336 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.336 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.336 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271873+0000", 2024-09-11T06:50:58.336 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3139000000000001e-05 2024-09-11T06:50:58.336 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.336 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.336 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.336 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271882+0000", 2024-09-11T06:50:58.336 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2369999999999992e-06 2024-09-11T06:50:58.336 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.336 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.336 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.336 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277150+0000", 2024-09-11T06:50:58.337 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052684419999999999 2024-09-11T06:50:58.337 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.337 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.337 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:58.337 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277273+0000", 2024-09-11T06:50:58.337 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012278899999999999 2024-09-11T06:50:58.337 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.337 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.337 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:58.337 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277466+0000", 2024-09-11T06:50:58.337 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193578 2024-09-11T06:50:58.337 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.337 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.338 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:58.338 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.280808+0000", 2024-09-11T06:50:58.338 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003341971 2024-09-11T06:50:58.338 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.338 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.338 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:58.338 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284419+0000", 2024-09-11T06:50:58.338 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003610982 2024-09-11T06:50:58.338 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.338 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.338 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:58.338 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284531+0000", 2024-09-11T06:50:58.338 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011121299999999999 2024-09-11T06:50:58.338 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.339 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.339 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.339 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284600+0000", 2024-09-11T06:50:58.339 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9592999999999996e-05 2024-09-11T06:50:58.339 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.339 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.339 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.339 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.339 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.339 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:58.339 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:58.339 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 112.057167824, 2024-09-11T06:50:58.339 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283314, 2024-09-11T06:50:58.340 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.340 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.340 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:58.340 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:58.340 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:58.340 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:58.340 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 88 2024-09-11T06:50:58.340 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.340 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.340 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.340 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.340 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:58.340 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.341 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.341 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.341 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.341 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:50:58.341 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.341 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.341 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.341 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.341 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271898+0000", 2024-09-11T06:50:58.341 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2999999999999999e-07 2024-09-11T06:50:58.341 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.341 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.341 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.341 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271904+0000", 2024-09-11T06:50:58.342 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.767e-06 2024-09-11T06:50:58.342 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.342 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.342 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.342 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271913+0000", 2024-09-11T06:50:58.342 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7200000000000001e-06 2024-09-11T06:50:58.342 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.342 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.342 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.342 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271920+0000", 2024-09-11T06:50:58.342 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7100000000000001e-06 2024-09-11T06:50:58.342 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.342 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.342 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.343 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278684+0000", 2024-09-11T06:50:58.343 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0067642650000000002 2024-09-11T06:50:58.343 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.343 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.343 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:58.343 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278838+0000", 2024-09-11T06:50:58.343 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015371499999999999 2024-09-11T06:50:58.343 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.343 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.343 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:58.343 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.279040+0000", 2024-09-11T06:50:58.343 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020207499999999999 2024-09-11T06:50:58.343 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.343 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.344 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:58.344 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.281448+0000", 2024-09-11T06:50:58.344 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002407686 2024-09-11T06:50:58.344 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.344 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.344 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:58.344 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284615+0000", 2024-09-11T06:50:58.344 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031675499999999999 2024-09-11T06:50:58.344 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.344 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.344 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:58.344 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284682+0000", 2024-09-11T06:50:58.344 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7032000000000005e-05 2024-09-11T06:50:58.344 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.345 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.345 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.345 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284730+0000", 2024-09-11T06:50:58.345 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7790000000000002e-05 2024-09-11T06:50:58.345 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.345 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.345 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.345 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.345 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.345 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:58.345 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:58.345 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 111.88831862399999, 2024-09-11T06:50:58.345 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014944941, 2024-09-11T06:50:58.346 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.346 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.346 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:58.346 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:58.346 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:58.346 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:58.346 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 111 2024-09-11T06:50:58.346 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.346 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.346 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.346 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.346 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:58.346 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.346 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.347 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.347 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.347 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:50:58.347 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.347 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.347 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.347 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.347 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440748+0000", 2024-09-11T06:50:58.347 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:50:58.347 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.347 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.347 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.347 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440760+0000", 2024-09-11T06:50:58.348 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1897e-05 2024-09-11T06:50:58.348 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.348 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.348 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.348 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440782+0000", 2024-09-11T06:50:58.348 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2552999999999999e-05 2024-09-11T06:50:58.348 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.348 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.348 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.348 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440793+0000", 2024-09-11T06:50:58.348 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0643e-05 2024-09-11T06:50:58.348 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.348 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.349 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.349 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449755+0000", 2024-09-11T06:50:58.349 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089622620000000004 2024-09-11T06:50:58.349 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.349 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.349 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:58.349 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449918+0000", 2024-09-11T06:50:58.349 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163086 2024-09-11T06:50:58.349 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.349 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.349 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:58.349 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450150+0000", 2024-09-11T06:50:58.349 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023204500000000001 2024-09-11T06:50:58.349 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.350 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.350 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:58.350 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454533+0000", 2024-09-11T06:50:58.350 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0043831349999999998 2024-09-11T06:50:58.350 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.350 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.350 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:58.350 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455511+0000", 2024-09-11T06:50:58.350 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0009776050000000001 2024-09-11T06:50:58.350 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.350 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.350 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:58.350 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455597+0000", 2024-09-11T06:50:58.351 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6446000000000005e-05 2024-09-11T06:50:58.351 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.351 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.351 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.351 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455691+0000", 2024-09-11T06:50:58.351 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3832000000000006e-05 2024-09-11T06:50:58.351 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.351 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.351 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.351 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.351 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.351 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:58.351 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:58.351 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 111.88824886099999, 2024-09-11T06:50:58.352 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018885411000000001, 2024-09-11T06:50:58.352 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.352 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.352 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:58.352 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:58.352 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:58.352 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:58.352 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 112 2024-09-11T06:50:58.352 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.352 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.352 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.352 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.352 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:58.353 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.353 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.353 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.353 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.353 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:50:58.353 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.353 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.353 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.353 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.353 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440817+0000", 2024-09-11T06:50:58.353 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:50:58.353 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.353 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.354 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.354 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441003+0000", 2024-09-11T06:50:58.354 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018661200000000001 2024-09-11T06:50:58.354 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.354 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.354 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.354 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441027+0000", 2024-09-11T06:50:58.354 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3920000000000001e-05 2024-09-11T06:50:58.354 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.354 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.354 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.354 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441062+0000", 2024-09-11T06:50:58.354 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4663000000000003e-05 2024-09-11T06:50:58.354 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.355 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.355 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.355 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451981+0000", 2024-09-11T06:50:58.355 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010918926000000001 2024-09-11T06:50:58.355 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.355 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.355 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:58.355 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452111+0000", 2024-09-11T06:50:58.355 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129529 2024-09-11T06:50:58.355 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.355 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.355 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:58.355 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452311+0000", 2024-09-11T06:50:58.355 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020013799999999999 2024-09-11T06:50:58.356 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.356 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.356 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:58.356 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455809+0000", 2024-09-11T06:50:58.356 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003498443 2024-09-11T06:50:58.356 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.356 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.356 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:58.356 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459566+0000", 2024-09-11T06:50:58.356 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003757271 2024-09-11T06:50:58.356 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.356 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.356 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:58.357 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459640+0000", 2024-09-11T06:50:58.357 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3949000000000006e-05 2024-09-11T06:50:58.357 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.357 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.357 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.357 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459701+0000", 2024-09-11T06:50:58.357 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1117000000000005e-05 2024-09-11T06:50:58.357 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.357 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.357 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.357 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.357 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.357 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:58.357 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:58.358 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 111.88797001, 2024-09-11T06:50:58.358 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018760915, 2024-09-11T06:50:58.358 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.358 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.358 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:58.358 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:58.358 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:58.358 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:58.358 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 113 2024-09-11T06:50:58.358 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.358 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.358 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.358 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.359 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:58.359 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.359 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.359 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.359 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.359 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:50:58.359 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.359 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.359 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.359 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.359 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441096+0000", 2024-09-11T06:50:58.359 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4130000000000001e-06 2024-09-11T06:50:58.359 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.359 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.360 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.360 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441110+0000", 2024-09-11T06:50:58.360 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3900000000000001e-05 2024-09-11T06:50:58.360 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.360 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.360 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.360 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441131+0000", 2024-09-11T06:50:58.360 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0636999999999999e-05 2024-09-11T06:50:58.360 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.360 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.360 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.360 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441147+0000", 2024-09-11T06:50:58.360 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6512999999999999e-05 2024-09-11T06:50:58.360 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.361 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.361 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.361 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453274+0000", 2024-09-11T06:50:58.361 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012126764999999999 2024-09-11T06:50:58.361 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.361 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.361 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:58.361 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453498+0000", 2024-09-11T06:50:58.361 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000223568 2024-09-11T06:50:58.362 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.362 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.362 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:58.362 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453872+0000", 2024-09-11T06:50:58.362 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037467099999999998 2024-09-11T06:50:58.362 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.362 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.362 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:58.362 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458709+0000", 2024-09-11T06:50:58.362 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0048365680000000003 2024-09-11T06:50:58.362 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.362 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.362 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:58.362 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459716+0000", 2024-09-11T06:50:58.363 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010075240000000001 2024-09-11T06:50:58.363 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.363 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.363 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:58.363 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459779+0000", 2024-09-11T06:50:58.363 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2336999999999996e-05 2024-09-11T06:50:58.363 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.363 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.363 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.363 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459856+0000", 2024-09-11T06:50:58.363 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7019000000000002e-05 2024-09-11T06:50:58.363 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.363 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.363 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.364 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.364 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.364 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:58.364 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:58.364 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 111.887893271, 2024-09-11T06:50:58.364 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018963114, 2024-09-11T06:50:58.364 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.364 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.364 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:58.364 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:58.364 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:58.364 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:58.364 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 114 2024-09-11T06:50:58.365 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.365 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.365 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.365 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.365 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:58.365 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.365 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.365 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.365 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.365 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:58.365 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.365 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.365 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.365 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.366 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:50:58.366 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9299999999999996e-07 2024-09-11T06:50:58.366 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.366 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.366 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.366 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441180+0000", 2024-09-11T06:50:58.366 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7200000000000006e-06 2024-09-11T06:50:58.366 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.366 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.366 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.366 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441265+0000", 2024-09-11T06:50:58.366 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4753000000000007e-05 2024-09-11T06:50:58.366 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.366 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.367 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.367 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441288+0000", 2024-09-11T06:50:58.367 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2860000000000001e-05 2024-09-11T06:50:58.367 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.367 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.367 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.367 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454585+0000", 2024-09-11T06:50:58.367 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013297286 2024-09-11T06:50:58.367 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.367 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.367 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:58.367 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454714+0000", 2024-09-11T06:50:58.367 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000128465 2024-09-11T06:50:58.367 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.368 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.368 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:58.368 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454909+0000", 2024-09-11T06:50:58.368 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019522499999999999 2024-09-11T06:50:58.368 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.368 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.368 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:58.368 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459868+0000", 2024-09-11T06:50:58.368 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0049591239999999997 2024-09-11T06:50:58.368 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.368 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.368 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:58.368 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459988+0000", 2024-09-11T06:50:58.369 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000120532 2024-09-11T06:50:58.369 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.369 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.369 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:58.369 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460071+0000", 2024-09-11T06:50:58.369 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2256000000000007e-05 2024-09-11T06:50:58.369 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.369 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.369 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.369 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460135+0000", 2024-09-11T06:50:58.369 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3999999999999997e-05 2024-09-11T06:50:58.369 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.369 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.369 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.370 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.370 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.370 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:58.370 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:58.370 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 111.887735642, 2024-09-11T06:50:58.370 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.019036849000000002, 2024-09-11T06:50:58.370 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.370 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.370 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:58.370 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:58.370 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:58.370 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:58.370 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 115 2024-09-11T06:50:58.370 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.371 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.371 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.371 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.371 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:58.371 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.371 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.371 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.371 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.371 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:50:58.371 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.371 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.371 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.371 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.371 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441330+0000", 2024-09-11T06:50:58.372 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1930000000000001e-06 2024-09-11T06:50:58.372 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.372 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.372 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.372 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441544+0000", 2024-09-11T06:50:58.372 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021405799999999999 2024-09-11T06:50:58.372 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.372 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.372 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.372 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441572+0000", 2024-09-11T06:50:58.372 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7317e-05 2024-09-11T06:50:58.372 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.372 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.373 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.373 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441593+0000", 2024-09-11T06:50:58.373 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1702999999999999e-05 2024-09-11T06:50:58.373 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.373 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.373 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.373 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455891+0000", 2024-09-11T06:50:58.373 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01429708 2024-09-11T06:50:58.373 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.373 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.373 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:58.373 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456046+0000", 2024-09-11T06:50:58.373 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000155682 2024-09-11T06:50:58.373 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.374 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.374 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:58.374 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456242+0000", 2024-09-11T06:50:58.374 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000195692 2024-09-11T06:50:58.374 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.374 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.374 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:58.374 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459876+0000", 2024-09-11T06:50:58.374 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036336480000000002 2024-09-11T06:50:58.374 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.375 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.375 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:58.375 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460245+0000", 2024-09-11T06:50:58.375 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036901700000000002 2024-09-11T06:50:58.375 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.375 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.375 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:58.375 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460316+0000", 2024-09-11T06:50:58.375 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1846000000000003e-05 2024-09-11T06:50:58.375 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.375 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.375 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.375 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460366+0000", 2024-09-11T06:50:58.375 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9613000000000001e-05 2024-09-11T06:50:58.376 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.376 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.376 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.376 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.376 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.376 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:58.376 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:58.376 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 111.88741356200001, 2024-09-11T06:50:58.376 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01904432, 2024-09-11T06:50:58.376 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.376 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.376 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:58.376 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:58.377 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:58.377 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:58.377 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 116 2024-09-11T06:50:58.377 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.377 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.377 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.377 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.377 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:58.377 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.377 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.378 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.378 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.378 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:50:58.378 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.378 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.378 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.378 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.378 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441653+0000", 2024-09-11T06:50:58.378 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:50:58.378 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.378 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.378 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.378 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441665+0000", 2024-09-11T06:50:58.378 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1653e-05 2024-09-11T06:50:58.379 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.379 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.379 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.379 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441684+0000", 2024-09-11T06:50:58.379 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9490000000000001e-05 2024-09-11T06:50:58.379 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.379 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.379 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.379 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441699+0000", 2024-09-11T06:50:58.379 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5044e-05 2024-09-11T06:50:58.379 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.379 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.379 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.380 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457297+0000", 2024-09-11T06:50:58.380 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015598241000000001 2024-09-11T06:50:58.380 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.380 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.380 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:58.380 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457577+0000", 2024-09-11T06:50:58.380 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027930799999999998 2024-09-11T06:50:58.380 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.380 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.380 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:58.380 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457957+0000", 2024-09-11T06:50:58.380 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00038088 2024-09-11T06:50:58.380 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.381 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.381 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:58.381 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460432+0000", 2024-09-11T06:50:58.381 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024743349999999998 2024-09-11T06:50:58.381 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.381 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.381 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:58.381 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460584+0000", 2024-09-11T06:50:58.381 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015232900000000001 2024-09-11T06:50:58.381 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.381 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.381 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:58.381 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460646+0000", 2024-09-11T06:50:58.381 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1883000000000004e-05 2024-09-11T06:50:58.382 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.382 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.382 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.382 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460696+0000", 2024-09-11T06:50:58.382 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9589999999999998e-05 2024-09-11T06:50:58.382 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.382 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.382 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.382 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.382 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.382 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:58.382 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:58.382 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 111.887341496, 2024-09-11T06:50:58.383 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.021834473, 2024-09-11T06:50:58.383 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.383 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.383 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:58.383 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:58.383 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:58.383 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:58.383 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 117 2024-09-11T06:50:58.383 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.383 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.383 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.384 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.384 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:58.384 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.384 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.384 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.384 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.384 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:50:58.384 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.384 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.384 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.384 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.384 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441725+0000", 2024-09-11T06:50:58.384 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5069999999999999e-06 2024-09-11T06:50:58.384 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.385 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.385 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.385 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441738+0000", 2024-09-11T06:50:58.385 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2813000000000001e-05 2024-09-11T06:50:58.385 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.385 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.385 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.385 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441759+0000", 2024-09-11T06:50:58.385 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1246999999999998e-05 2024-09-11T06:50:58.385 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.385 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.385 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.385 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441779+0000", 2024-09-11T06:50:58.386 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0392999999999999e-05 2024-09-11T06:50:58.386 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.386 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.386 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.386 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458804+0000", 2024-09-11T06:50:58.386 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.017024182999999998 2024-09-11T06:50:58.386 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.386 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.386 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:58.386 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458977+0000", 2024-09-11T06:50:58.386 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000173926 2024-09-11T06:50:58.386 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.386 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.386 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:58.386 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459170+0000", 2024-09-11T06:50:58.387 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000192184 2024-09-11T06:50:58.387 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.387 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.387 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:58.387 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.462300+0000", 2024-09-11T06:50:58.387 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031304969999999999 2024-09-11T06:50:58.387 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.387 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.387 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:58.387 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463408+0000", 2024-09-11T06:50:58.387 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011077839999999999 2024-09-11T06:50:58.387 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.387 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.388 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:58.388 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463497+0000", 2024-09-11T06:50:58.388 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8828999999999999e-05 2024-09-11T06:50:58.388 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.388 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.388 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.388 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463558+0000", 2024-09-11T06:50:58.388 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1110000000000003e-05 2024-09-11T06:50:58.388 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.388 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.388 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.389 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.389 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.389 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:58.389 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:58.389 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 111.51069999400001, 2024-09-11T06:50:58.389 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013136297, 2024-09-11T06:50:58.389 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.389 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.389 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:58.389 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:58.389 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:58.389 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:58.390 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 138 2024-09-11T06:50:58.390 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.390 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.390 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.390 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.390 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:58.390 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.390 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.390 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.390 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.390 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:58.390 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.390 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.391 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.391 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.391 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:50:58.391 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.89e-07 2024-09-11T06:50:58.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.391 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.391 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.391 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818369+0000", 2024-09-11T06:50:58.391 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4170000000000001e-06 2024-09-11T06:50:58.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.391 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.391 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.391 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818376+0000", 2024-09-11T06:50:58.391 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4270000000000002e-06 2024-09-11T06:50:58.392 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.392 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.392 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.392 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818383+0000", 2024-09-11T06:50:58.392 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3060000000000003e-06 2024-09-11T06:50:58.392 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.392 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.392 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.392 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824783+0000", 2024-09-11T06:50:58.392 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064003020000000001 2024-09-11T06:50:58.393 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.393 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.393 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:58.393 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825013+0000", 2024-09-11T06:50:58.393 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023039499999999999 2024-09-11T06:50:58.393 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.393 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.393 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:58.393 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825484+0000", 2024-09-11T06:50:58.393 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000470562 2024-09-11T06:50:58.393 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.393 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.393 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:58.394 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828633+0000", 2024-09-11T06:50:58.394 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031496689999999999 2024-09-11T06:50:58.394 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.394 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.394 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:58.394 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831413+0000", 2024-09-11T06:50:58.394 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027793219999999999 2024-09-11T06:50:58.394 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.394 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.394 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:58.394 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831450+0000", 2024-09-11T06:50:58.394 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7370999999999998e-05 2024-09-11T06:50:58.394 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.394 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.395 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.395 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831501+0000", 2024-09-11T06:50:58.395 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1036999999999999e-05 2024-09-11T06:50:58.395 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.395 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.395 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.395 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.395 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:58.395 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:58.395 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 111.510675514, 2024-09-11T06:50:58.395 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013223011, 2024-09-11T06:50:58.395 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.395 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.396 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:58.396 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:58.396 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:58.396 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:58.396 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 139 2024-09-11T06:50:58.396 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.396 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.396 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.396 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:58.396 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.396 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.397 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.397 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:50:58.397 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.397 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.397 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.397 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818390+0000", 2024-09-11T06:50:58.397 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5600000000000001e-07 2024-09-11T06:50:58.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.397 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.397 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.397 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818394+0000", 2024-09-11T06:50:58.397 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5929999999999999e-06 2024-09-11T06:50:58.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.398 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.398 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.398 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818399+0000", 2024-09-11T06:50:58.398 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7740000000000004e-06 2024-09-11T06:50:58.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.398 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.398 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.398 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818412+0000", 2024-09-11T06:50:58.398 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2625e-05 2024-09-11T06:50:58.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.398 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.398 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.398 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826205+0000", 2024-09-11T06:50:58.399 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0077930730000000002 2024-09-11T06:50:58.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.399 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.399 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:58.399 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826281+0000", 2024-09-11T06:50:58.399 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5876000000000003e-05 2024-09-11T06:50:58.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.399 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.399 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:58.399 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826384+0000", 2024-09-11T06:50:58.399 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103838 2024-09-11T06:50:58.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.399 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.399 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:58.400 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830642+0000", 2024-09-11T06:50:58.400 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004257083 2024-09-11T06:50:58.400 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.400 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.400 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:58.400 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831509+0000", 2024-09-11T06:50:58.400 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086730900000000003 2024-09-11T06:50:58.400 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.400 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.400 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:58.400 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831547+0000", 2024-09-11T06:50:58.400 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7793000000000002e-05 2024-09-11T06:50:58.400 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.400 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.401 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.401 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831612+0000", 2024-09-11T06:50:58.401 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5691000000000005e-05 2024-09-11T06:50:58.401 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.401 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.401 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.401 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:58.401 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:58.401 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 111.51064682000001, 2024-09-11T06:50:58.401 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016098024999999998, 2024-09-11T06:50:58.402 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.402 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.402 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:58.402 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:58.402 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:58.402 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:58.402 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 140 2024-09-11T06:50:58.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.402 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.402 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.402 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:58.403 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.403 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.403 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.403 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:58.403 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.403 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.404 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.404 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:50:58.404 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5199999999999998e-07 2024-09-11T06:50:58.404 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.404 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.404 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.404 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818479+0000", 2024-09-11T06:50:58.404 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0368000000000002e-05 2024-09-11T06:50:58.404 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.404 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.404 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.404 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818487+0000", 2024-09-11T06:50:58.404 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2339999999999994e-06 2024-09-11T06:50:58.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.405 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.405 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.405 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818491+0000", 2024-09-11T06:50:58.405 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4329999999999997e-06 2024-09-11T06:50:58.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.405 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.405 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.405 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828080+0000", 2024-09-11T06:50:58.405 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0095890449999999992 2024-09-11T06:50:58.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.405 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.405 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:58.405 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828130+0000", 2024-09-11T06:50:58.406 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9239000000000001e-05 2024-09-11T06:50:58.406 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.406 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.406 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:58.406 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828208+0000", 2024-09-11T06:50:58.406 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.852e-05 2024-09-11T06:50:58.406 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.406 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.406 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:58.406 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831625+0000", 2024-09-11T06:50:58.406 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034169410000000002 2024-09-11T06:50:58.406 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.406 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.406 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:58.407 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834411+0000", 2024-09-11T06:50:58.407 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002786111 2024-09-11T06:50:58.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.407 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.407 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:58.407 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834457+0000", 2024-09-11T06:50:58.407 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6041000000000002e-05 2024-09-11T06:50:58.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.407 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.407 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.407 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834516+0000", 2024-09-11T06:50:58.407 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8740999999999999e-05 2024-09-11T06:50:58.408 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.408 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.408 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.408 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.408 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.408 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:50:58.408 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:58.408 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 111.51056575299999, 2024-09-11T06:50:58.408 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016588772000000002, 2024-09-11T06:50:58.408 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.408 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.408 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:58.408 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:58.408 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:50:58.409 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:50:58.409 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 141 2024-09-11T06:50:58.409 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.409 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.409 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.409 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.409 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:58.409 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.409 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.409 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.409 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.409 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:50:58.409 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.410 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.410 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.410 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818500+0000", 2024-09-11T06:50:58.410 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0800000000000005e-07 2024-09-11T06:50:58.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.410 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.410 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.410 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818505+0000", 2024-09-11T06:50:58.410 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2429999999999996e-06 2024-09-11T06:50:58.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.410 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.410 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.411 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818510+0000", 2024-09-11T06:50:58.411 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9339999999999997e-06 2024-09-11T06:50:58.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.411 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.411 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.411 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818514+0000", 2024-09-11T06:50:58.411 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7270000000000001e-06 2024-09-11T06:50:58.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.411 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.411 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.411 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828711+0000", 2024-09-11T06:50:58.411 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010197031 2024-09-11T06:50:58.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.411 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.412 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:58.412 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828853+0000", 2024-09-11T06:50:58.412 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014201200000000001 2024-09-11T06:50:58.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.412 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.412 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:58.412 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.829115+0000", 2024-09-11T06:50:58.412 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026257500000000003 2024-09-11T06:50:58.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.412 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.412 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:58.412 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831629+0000", 2024-09-11T06:50:58.412 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025138169999999998 2024-09-11T06:50:58.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.413 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:58.413 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834991+0000", 2024-09-11T06:50:58.413 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033624459999999998 2024-09-11T06:50:58.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.413 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:58.413 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835032+0000", 2024-09-11T06:50:58.413 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1001000000000003e-05 2024-09-11T06:50:58.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.413 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.414 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835088+0000", 2024-09-11T06:50:58.414 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5478e-05 2024-09-11T06:50:58.414 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.414 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.414 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.414 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.414 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5235.0:1 2.5 2:b2cc3336:::rbd_id.testrbdnbd21607:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-11T06:50:58.414 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:50:58.414 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 23.769763731000001, 2024-09-11T06:50:58.414 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.68353462499999995, 2024-09-11T06:50:58.414 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.414 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.414 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:58.415 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:58.415 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5235", 2024-09-11T06:50:58.415 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2008977164", 2024-09-11T06:50:58.415 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:50:58.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.415 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.415 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.415 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.415 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:50:58.415 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.415 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.415 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.416 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:50:58.416 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.416 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.416 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.416 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559303+0000", 2024-09-11T06:50:58.416 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1830000000000001e-06 2024-09-11T06:50:58.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.416 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.416 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.416 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559314+0000", 2024-09-11T06:50:58.416 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0396e-05 2024-09-11T06:50:58.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.417 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.417 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.417 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559335+0000", 2024-09-11T06:50:58.417 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1183e-05 2024-09-11T06:50:58.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.417 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.417 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.417 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559354+0000", 2024-09-11T06:50:58.417 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9225999999999999e-05 2024-09-11T06:50:58.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.417 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.418 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.418 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559522+0000", 2024-09-11T06:50:58.418 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000168021 2024-09-11T06:50:58.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.418 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.418 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:50:58.418 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559543+0000", 2024-09-11T06:50:58.418 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0616999999999999e-05 2024-09-11T06:50:58.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.418 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.418 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.418 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.238584+0000", 2024-09-11T06:50:58.419 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.67904107499999999 2024-09-11T06:50:58.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.419 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.419 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for missing object", 2024-09-11T06:50:58.419 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.238933+0000", 2024-09-11T06:50:58.419 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034879600000000001 2024-09-11T06:50:58.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.419 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.419 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.419 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.241968+0000", 2024-09-11T06:50:58.419 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030350830000000001 2024-09-11T06:50:58.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.419 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.419 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:58.419 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.242281+0000", 2024-09-11T06:50:58.420 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00031299499999999999 2024-09-11T06:50:58.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.420 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.420 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.420 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.242836+0000", 2024-09-11T06:50:58.420 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00055504999999999997 2024-09-11T06:50:58.420 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.420 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.420 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.421 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.421 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5235.0:18 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 0~1024 [fadvise_sequential]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T06:50:58.421 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:50:58.421 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 23.067271292000001, 2024-09-11T06:50:58.421 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.048993760999999997, 2024-09-11T06:50:58.421 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.421 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.421 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:58.421 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:58.421 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5235", 2024-09-11T06:50:58.421 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2008977164", 2024-09-11T06:50:58.421 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 18 2024-09-11T06:50:58.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.422 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.422 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.422 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.422 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:50:58.422 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.422 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.422 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.422 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:50:58.423 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.423 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.423 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.423 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261797+0000", 2024-09-11T06:50:58.423 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2059999999999999e-06 2024-09-11T06:50:58.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.423 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.423 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.423 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261812+0000", 2024-09-11T06:50:58.423 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5159e-05 2024-09-11T06:50:58.424 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.424 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.424 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.424 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261852+0000", 2024-09-11T06:50:58.424 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.9906e-05 2024-09-11T06:50:58.424 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.424 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.424 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.424 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261865+0000", 2024-09-11T06:50:58.425 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.312e-05 2024-09-11T06:50:58.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.425 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.425 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.425 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.263166+0000", 2024-09-11T06:50:58.425 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001301134 2024-09-11T06:50:58.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.425 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.425 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for missing object", 2024-09-11T06:50:58.425 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.263529+0000", 2024-09-11T06:50:58.425 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036279900000000002 2024-09-11T06:50:58.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.425 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.426 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.426 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.300567+0000", 2024-09-11T06:50:58.426 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.037038326000000003 2024-09-11T06:50:58.426 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.426 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.426 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for rw locks", 2024-09-11T06:50:58.426 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.300800+0000", 2024-09-11T06:50:58.426 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023323300000000001 2024-09-11T06:50:58.426 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.426 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.426 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.427 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.306709+0000", 2024-09-11T06:50:58.427 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0059087690000000003 2024-09-11T06:50:58.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.427 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.427 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:58.427 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.306850+0000", 2024-09-11T06:50:58.427 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014106200000000001 2024-09-11T06:50:58.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.427 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.427 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:58.427 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.307121+0000", 2024-09-11T06:50:58.427 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027083599999999999 2024-09-11T06:50:58.428 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.428 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.428 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:58.428 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.308809+0000", 2024-09-11T06:50:58.428 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0016880759999999999 2024-09-11T06:50:58.428 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.428 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.428 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:58.428 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310382+0000", 2024-09-11T06:50:58.428 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0015732840000000001 2024-09-11T06:50:58.428 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.428 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.428 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:58.429 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310501+0000", 2024-09-11T06:50:58.429 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000118276 2024-09-11T06:50:58.429 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.429 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.429 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.429 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310787+0000", 2024-09-11T06:50:58.429 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00028657500000000001 2024-09-11T06:50:58.429 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.429 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.429 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.429 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.430 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.430 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:70 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 393216~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:50:58.430 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596118+0000", 2024-09-11T06:50:58.430 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 9.7329468900000009, 2024-09-11T06:50:58.430 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012279145, 2024-09-11T06:50:58.430 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.430 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.430 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:58.430 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:58.430 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:50:58.430 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:50:58.430 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 70 2024-09-11T06:50:58.430 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.431 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.431 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.431 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.431 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596118+0000", 2024-09-11T06:50:58.431 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.431 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.431 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.431 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.431 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596118+0000", 2024-09-11T06:50:58.431 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.432 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.432 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.432 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596119+0000", 2024-09-11T06:50:58.432 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0299999999999997e-07 2024-09-11T06:50:58.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.432 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.432 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.432 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596238+0000", 2024-09-11T06:50:58.432 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000118798 2024-09-11T06:50:58.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.432 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.433 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.433 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596250+0000", 2024-09-11T06:50:58.433 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2775999999999999e-05 2024-09-11T06:50:58.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.433 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.433 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.433 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596259+0000", 2024-09-11T06:50:58.433 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3100000000000001e-06 2024-09-11T06:50:58.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.433 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.433 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.433 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.603389+0000", 2024-09-11T06:50:58.433 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0071300759999999999 2024-09-11T06:50:58.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.434 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.434 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:58.434 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.603540+0000", 2024-09-11T06:50:58.434 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015111700000000001 2024-09-11T06:50:58.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.434 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.434 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:58.434 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.603766+0000", 2024-09-11T06:50:58.434 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022570899999999999 2024-09-11T06:50:58.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.434 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.434 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:58.434 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.606676+0000", 2024-09-11T06:50:58.434 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0029106380000000001 2024-09-11T06:50:58.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.435 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.435 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:58.435 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608173+0000", 2024-09-11T06:50:58.435 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014967629999999999 2024-09-11T06:50:58.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.435 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.435 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:58.435 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608277+0000", 2024-09-11T06:50:58.435 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000104398 2024-09-11T06:50:58.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.436 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.436 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.436 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608397+0000", 2024-09-11T06:50:58.436 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000119657 2024-09-11T06:50:58.436 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.436 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.436 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.436 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.436 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:71 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 524288~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:50:58.436 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596272+0000", 2024-09-11T06:50:58.437 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 9.7327926369999993, 2024-09-11T06:50:58.437 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013390681999999999, 2024-09-11T06:50:58.437 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.437 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.437 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:58.437 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:58.437 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:50:58.437 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:50:58.437 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 71 2024-09-11T06:50:58.437 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.437 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.437 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.437 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.438 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596272+0000", 2024-09-11T06:50:58.438 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.438 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.438 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.438 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596272+0000", 2024-09-11T06:50:58.438 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.438 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.438 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.438 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596273+0000", 2024-09-11T06:50:58.438 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.9999999999999996e-07 2024-09-11T06:50:58.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.438 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.439 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.439 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596395+0000", 2024-09-11T06:50:58.439 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012204799999999999 2024-09-11T06:50:58.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.439 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.439 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.439 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596411+0000", 2024-09-11T06:50:58.439 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5746000000000001e-05 2024-09-11T06:50:58.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.439 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.439 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.439 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596419+0000", 2024-09-11T06:50:58.439 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2600000000000005e-06 2024-09-11T06:50:58.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.440 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.440 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.440 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.605311+0000", 2024-09-11T06:50:58.440 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0088915179999999993 2024-09-11T06:50:58.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.440 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.440 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:58.440 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.605436+0000", 2024-09-11T06:50:58.440 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000125203 2024-09-11T06:50:58.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.440 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.440 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:58.441 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.605625+0000", 2024-09-11T06:50:58.441 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018961299999999999 2024-09-11T06:50:58.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.441 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.441 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:58.441 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609477+0000", 2024-09-11T06:50:58.441 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0038513549999999999 2024-09-11T06:50:58.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.441 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.441 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:58.441 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609499+0000", 2024-09-11T06:50:58.441 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2215e-05 2024-09-11T06:50:58.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.442 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.442 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:58.442 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609589+0000", 2024-09-11T06:50:58.442 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0029000000000001e-05 2024-09-11T06:50:58.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.442 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.442 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.442 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609663+0000", 2024-09-11T06:50:58.442 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3894999999999996e-05 2024-09-11T06:50:58.442 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.442 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.442 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.442 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.443 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:72 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 655360~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:50:58.443 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596432+0000", 2024-09-11T06:50:58.443 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 9.7326331499999998, 2024-09-11T06:50:58.443 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01414727, 2024-09-11T06:50:58.443 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.443 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.443 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:58.443 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:58.443 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:50:58.443 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:50:58.443 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 72 2024-09-11T06:50:58.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.443 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.444 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.444 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.444 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596432+0000", 2024-09-11T06:50:58.444 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.444 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.444 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.444 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596432+0000", 2024-09-11T06:50:58.444 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.444 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.444 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.444 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596433+0000", 2024-09-11T06:50:58.444 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1999999999999998e-07 2024-09-11T06:50:58.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.445 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.445 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596556+0000", 2024-09-11T06:50:58.445 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012379000000000001 2024-09-11T06:50:58.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.445 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.445 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596571+0000", 2024-09-11T06:50:58.445 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4749999999999999e-05 2024-09-11T06:50:58.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.446 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.446 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596579+0000", 2024-09-11T06:50:58.446 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2500000000000006e-06 2024-09-11T06:50:58.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.446 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.446 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.446 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.606784+0000", 2024-09-11T06:50:58.446 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010204757 2024-09-11T06:50:58.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.446 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.446 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:58.446 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.606950+0000", 2024-09-11T06:50:58.446 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000166229 2024-09-11T06:50:58.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.447 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:58.447 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.607176+0000", 2024-09-11T06:50:58.447 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022549899999999999 2024-09-11T06:50:58.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.447 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:58.447 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610363+0000", 2024-09-11T06:50:58.447 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031874690000000001 2024-09-11T06:50:58.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.447 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:58.448 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610490+0000", 2024-09-11T06:50:58.448 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012639899999999999 2024-09-11T06:50:58.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.448 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:58.448 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610529+0000", 2024-09-11T06:50:58.448 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.8936000000000001e-05 2024-09-11T06:50:58.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.448 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.448 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610579+0000", 2024-09-11T06:50:58.448 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0371000000000003e-05 2024-09-11T06:50:58.448 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.448 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.449 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.449 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:73 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 786432~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:50:58.449 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596592+0000", 2024-09-11T06:50:58.449 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 9.7324728969999992, 2024-09-11T06:50:58.449 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014914779, 2024-09-11T06:50:58.449 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.449 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.449 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:58.449 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:58.449 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:50:58.449 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:50:58.450 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 73 2024-09-11T06:50:58.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.450 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.450 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.450 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.450 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596592+0000", 2024-09-11T06:50:58.450 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.450 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.450 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.450 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596592+0000", 2024-09-11T06:50:58.450 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.451 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.451 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.451 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596593+0000", 2024-09-11T06:50:58.451 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.0699999999999996e-07 2024-09-11T06:50:58.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.451 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.451 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.451 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596786+0000", 2024-09-11T06:50:58.451 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019278200000000001 2024-09-11T06:50:58.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.451 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.451 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.451 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596801+0000", 2024-09-11T06:50:58.451 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5690000000000001e-05 2024-09-11T06:50:58.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.452 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.452 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596815+0000", 2024-09-11T06:50:58.452 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3753e-05 2024-09-11T06:50:58.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.452 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.452 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608442+0000", 2024-09-11T06:50:58.452 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011627385 2024-09-11T06:50:58.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.452 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:58.453 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608570+0000", 2024-09-11T06:50:58.453 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012740399999999999 2024-09-11T06:50:58.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.453 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:58.453 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608754+0000", 2024-09-11T06:50:58.453 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018392899999999999 2024-09-11T06:50:58.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.453 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:58.453 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610370+0000", 2024-09-11T06:50:58.453 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001615998 2024-09-11T06:50:58.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.454 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:58.454 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611420+0000", 2024-09-11T06:50:58.454 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010502790000000001 2024-09-11T06:50:58.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.454 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:58.454 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611458+0000", 2024-09-11T06:50:58.454 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7749000000000001e-05 2024-09-11T06:50:58.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.454 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.454 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611507+0000", 2024-09-11T06:50:58.454 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9002999999999998e-05 2024-09-11T06:50:58.454 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.455 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.455 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.455 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:74 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 917504~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:50:58.455 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596849+0000", 2024-09-11T06:50:58.455 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 9.7322162589999994, 2024-09-11T06:50:58.455 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015202963999999999, 2024-09-11T06:50:58.455 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.455 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.455 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:58.455 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:58.455 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:50:58.456 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:50:58.456 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 74 2024-09-11T06:50:58.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.456 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.456 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.456 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596849+0000", 2024-09-11T06:50:58.456 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.456 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.456 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596849+0000", 2024-09-11T06:50:58.456 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.457 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.457 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.457 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596850+0000", 2024-09-11T06:50:58.457 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3170000000000001e-06 2024-09-11T06:50:58.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.457 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.457 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.457 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.597118+0000", 2024-09-11T06:50:58.457 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026772999999999999 2024-09-11T06:50:58.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.457 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.457 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.457 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.597136+0000", 2024-09-11T06:50:58.458 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8467000000000001e-05 2024-09-11T06:50:58.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.458 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.458 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.458 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.597148+0000", 2024-09-11T06:50:58.458 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1486e-05 2024-09-11T06:50:58.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.458 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.458 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.458 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609714+0000", 2024-09-11T06:50:58.458 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012566124 2024-09-11T06:50:58.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.458 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.458 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:58.458 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609792+0000", 2024-09-11T06:50:58.459 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.8579999999999996e-05 2024-09-11T06:50:58.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.459 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:58.459 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609899+0000", 2024-09-11T06:50:58.459 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00010716499999999999 2024-09-11T06:50:58.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.459 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:58.459 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610865+0000", 2024-09-11T06:50:58.459 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00096540299999999999 2024-09-11T06:50:58.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.459 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:58.460 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611970+0000", 2024-09-11T06:50:58.460 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001105118 2024-09-11T06:50:58.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.460 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.460 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:58.460 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.612005+0000", 2024-09-11T06:50:58.460 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4789000000000002e-05 2024-09-11T06:50:58.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.460 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.460 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.460 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.612052+0000", 2024-09-11T06:50:58.460 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6785e-05 2024-09-11T06:50:58.460 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.460 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.461 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.461 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.461 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.461 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:58.463 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:50:58.463 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:50:58.458+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:50:58.563 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:50:58.562+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:50:58.634 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:58.634 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:58.634 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:50:58.634 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:58.648 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:50:58.696 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:58.694+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:58.765 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:58.762+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:58.772 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:58.772 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:58.772 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:50:58.772 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:50:58.772 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:58.787 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:50:58.847 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T06:50:58.876 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:58.874+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:58.956 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:58.956 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:58.956 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:58.957 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:50:58.957 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.957 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4558.0:1 3.3 3:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-11T06:50:58.957 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:50:58.957 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 41.865101527, 2024-09-11T06:50:58.957 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0085520600000000002, 2024-09-11T06:50:58.957 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.957 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.957 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:58.957 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:58.957 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4558", 2024-09-11T06:50:58.957 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1478388898", 2024-09-11T06:50:58.957 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:50:58.958 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.958 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.958 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.958 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.958 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:50:58.958 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.958 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.958 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.958 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.958 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:50:58.958 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.958 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.958 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.958 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.959 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090533+0000", 2024-09-11T06:50:58.959 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2629999999999999e-06 2024-09-11T06:50:58.959 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.959 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.959 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.959 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090546+0000", 2024-09-11T06:50:58.959 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2918999999999999e-05 2024-09-11T06:50:58.959 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.959 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.959 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.959 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090572+0000", 2024-09-11T06:50:58.959 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6412000000000001e-05 2024-09-11T06:50:58.960 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.960 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.960 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.960 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090594+0000", 2024-09-11T06:50:58.960 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2476e-05 2024-09-11T06:50:58.960 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.960 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.960 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.960 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090938+0000", 2024-09-11T06:50:58.960 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034325099999999998 2024-09-11T06:50:58.960 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.960 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.960 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:58.961 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.091149+0000", 2024-09-11T06:50:58.961 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021151300000000001 2024-09-11T06:50:58.961 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.961 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.961 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:50:58.961 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.091377+0000", 2024-09-11T06:50:58.961 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022785399999999999 2024-09-11T06:50:58.961 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.961 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.961 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:58.961 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.092912+0000", 2024-09-11T06:50:58.961 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0015346089999999999 2024-09-11T06:50:58.961 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.961 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.962 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:58.962 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098919+0000", 2024-09-11T06:50:58.962 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0060076799999999996 2024-09-11T06:50:58.962 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.962 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.962 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:58.962 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.099009+0000", 2024-09-11T06:50:58.962 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0056000000000006e-05 2024-09-11T06:50:58.962 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.962 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.962 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.962 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.099081+0000", 2024-09-11T06:50:58.962 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2026999999999993e-05 2024-09-11T06:50:58.963 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.963 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.963 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.963 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.963 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.963 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:50:58.963 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:50:58.963 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 35.823612447000002, 2024-09-11T06:50:58.963 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0071924390000000001, 2024-09-11T06:50:58.963 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.963 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.963 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:58.963 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.964 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.964 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.964 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:50:58.964 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.964 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.964 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.964 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.964 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:50:58.964 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.964 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.964 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.964 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.964 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132020+0000", 2024-09-11T06:50:58.964 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.846e-06 2024-09-11T06:50:58.965 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.965 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.965 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.965 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135688+0000", 2024-09-11T06:50:58.965 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036677350000000001 2024-09-11T06:50:58.965 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.965 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.965 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.965 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135747+0000", 2024-09-11T06:50:58.965 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.9314999999999997e-05 2024-09-11T06:50:58.965 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.965 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.965 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.965 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135797+0000", 2024-09-11T06:50:58.966 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9984000000000001e-05 2024-09-11T06:50:58.966 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.966 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.966 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.966 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136143+0000", 2024-09-11T06:50:58.966 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034594299999999998 2024-09-11T06:50:58.966 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.966 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.966 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:58.966 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136179+0000", 2024-09-11T06:50:58.966 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5636000000000002e-05 2024-09-11T06:50:58.966 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.966 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.967 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.967 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.139211+0000", 2024-09-11T06:50:58.967 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030319800000000001 2024-09-11T06:50:58.967 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.967 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.967 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.967 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.967 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.967 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:50:58.967 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:50:58.967 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 35.821549193000003, 2024-09-11T06:50:58.967 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064954139999999997, 2024-09-11T06:50:58.967 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.967 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.968 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:58.968 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.968 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.968 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.968 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:50:58.968 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.968 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.968 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.968 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.968 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:50:58.968 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.968 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.968 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.969 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.969 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134084+0000", 2024-09-11T06:50:58.969 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.894e-06 2024-09-11T06:50:58.969 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.969 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.969 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.969 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136513+0000", 2024-09-11T06:50:58.969 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024290969999999999 2024-09-11T06:50:58.969 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.969 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.969 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.969 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136554+0000", 2024-09-11T06:50:58.969 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.0932e-05 2024-09-11T06:50:58.970 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.970 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.970 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.970 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136584+0000", 2024-09-11T06:50:58.970 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0056000000000002e-05 2024-09-11T06:50:58.970 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.970 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.970 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.970 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136805+0000", 2024-09-11T06:50:58.970 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022086100000000001 2024-09-11T06:50:58.970 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.970 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.970 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:58.970 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136864+0000", 2024-09-11T06:50:58.971 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.9907000000000001e-05 2024-09-11T06:50:58.971 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.971 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.971 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.971 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.140577+0000", 2024-09-11T06:50:58.971 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0037126669999999998 2024-09-11T06:50:58.971 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.971 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.971 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.971 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.971 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.971 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:50:58.971 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:50:58.971 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 35.813258392999998, 2024-09-11T06:50:58.972 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035763180000000002, 2024-09-11T06:50:58.972 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.972 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.972 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:58.972 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.972 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.972 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.972 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:50:58.972 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.972 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.972 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.972 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.972 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:50:58.973 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.973 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.973 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.973 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.973 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142374+0000", 2024-09-11T06:50:58.973 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5460000000000001e-06 2024-09-11T06:50:58.973 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.973 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.973 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.973 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143902+0000", 2024-09-11T06:50:58.973 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00152808 2024-09-11T06:50:58.973 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.973 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.973 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.974 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143946+0000", 2024-09-11T06:50:58.974 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3874e-05 2024-09-11T06:50:58.974 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.974 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.974 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.974 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143978+0000", 2024-09-11T06:50:58.974 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2425999999999998e-05 2024-09-11T06:50:58.974 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.974 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.974 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.974 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.144177+0000", 2024-09-11T06:50:58.974 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019837500000000001 2024-09-11T06:50:58.974 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.975 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.975 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:58.975 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.144198+0000", 2024-09-11T06:50:58.975 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1639000000000001e-05 2024-09-11T06:50:58.975 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.975 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.975 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.975 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.145949+0000", 2024-09-11T06:50:58.975 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0017503779999999999 2024-09-11T06:50:58.975 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.975 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.975 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.975 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.975 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.976 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:50:58.976 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:50:58.976 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 35.808942690999999, 2024-09-11T06:50:58.976 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0057693220000000003, 2024-09-11T06:50:58.976 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.976 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.976 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:58.976 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.976 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.976 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.976 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:50:58.976 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.976 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.977 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.977 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.977 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:50:58.977 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.977 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.977 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.977 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.977 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146690+0000", 2024-09-11T06:50:58.977 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.607e-06 2024-09-11T06:50:58.977 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.977 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.977 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.977 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148693+0000", 2024-09-11T06:50:58.977 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0020029739999999998 2024-09-11T06:50:58.978 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.978 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.978 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.978 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148735+0000", 2024-09-11T06:50:58.978 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1958e-05 2024-09-11T06:50:58.978 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.978 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.978 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.978 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148764+0000", 2024-09-11T06:50:58.978 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8999000000000001e-05 2024-09-11T06:50:58.978 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.978 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.978 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.978 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149069+0000", 2024-09-11T06:50:58.979 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000305032 2024-09-11T06:50:58.979 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.979 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.979 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:58.979 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149204+0000", 2024-09-11T06:50:58.979 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013495 2024-09-11T06:50:58.979 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.979 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.979 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.979 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.152458+0000", 2024-09-11T06:50:58.979 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0032538020000000001 2024-09-11T06:50:58.979 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.979 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.980 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.980 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.980 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.980 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:50:58.980 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:50:58.980 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 35.808044150000001, 2024-09-11T06:50:58.980 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0042425329999999997, 2024-09-11T06:50:58.980 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.980 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.980 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:58.980 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.980 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.980 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.981 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:50:58.981 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.981 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.981 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.981 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.981 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:50:58.981 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.981 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.981 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.981 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.981 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147588+0000", 2024-09-11T06:50:58.981 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.277e-06 2024-09-11T06:50:58.981 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.982 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.982 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.982 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149051+0000", 2024-09-11T06:50:58.982 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001462669 2024-09-11T06:50:58.982 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.982 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.982 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.982 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149090+0000", 2024-09-11T06:50:58.982 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.8967999999999997e-05 2024-09-11T06:50:58.982 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.982 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.982 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.982 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149120+0000", 2024-09-11T06:50:58.982 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0766000000000003e-05 2024-09-11T06:50:58.983 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.983 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.983 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.983 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149395+0000", 2024-09-11T06:50:58.983 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027484900000000002 2024-09-11T06:50:58.983 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.983 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.983 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:58.983 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149439+0000", 2024-09-11T06:50:58.983 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3940999999999998e-05 2024-09-11T06:50:58.983 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.983 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.983 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.983 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.151829+0000", 2024-09-11T06:50:58.984 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002390063 2024-09-11T06:50:58.984 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.984 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.984 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.984 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.984 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.984 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:50:58.984 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:50:58.984 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 35.801903519, 2024-09-11T06:50:58.984 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0046568219999999997, 2024-09-11T06:50:58.984 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.984 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.984 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:58.985 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.985 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.985 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.985 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:50:58.985 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.985 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.985 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.985 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.985 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:50:58.985 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.985 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.985 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.985 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.985 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153729+0000", 2024-09-11T06:50:58.986 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5629999999999999e-06 2024-09-11T06:50:58.986 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.986 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.986 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.986 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155179+0000", 2024-09-11T06:50:58.986 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001450469 2024-09-11T06:50:58.986 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.986 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.986 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.986 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155221+0000", 2024-09-11T06:50:58.986 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1448e-05 2024-09-11T06:50:58.986 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.986 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.986 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.987 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155249+0000", 2024-09-11T06:50:58.987 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8433e-05 2024-09-11T06:50:58.987 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.987 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.987 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.987 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155420+0000", 2024-09-11T06:50:58.987 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017070599999999999 2024-09-11T06:50:58.987 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.987 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.987 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:58.987 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155445+0000", 2024-09-11T06:50:58.987 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4813000000000001e-05 2024-09-11T06:50:58.987 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.988 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.988 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.988 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.158384+0000", 2024-09-11T06:50:58.988 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0029393900000000001 2024-09-11T06:50:58.988 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.988 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.988 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.988 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.988 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.988 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:50:58.988 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:50:58.988 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 35.799876769999997, 2024-09-11T06:50:58.988 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0046826020000000001, 2024-09-11T06:50:58.988 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.989 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.989 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:58.989 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.989 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.989 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.989 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:50:58.989 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.989 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.989 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.989 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.989 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:50:58.989 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.989 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.990 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.990 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.990 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155756+0000", 2024-09-11T06:50:58.990 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.384e-06 2024-09-11T06:50:58.990 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.990 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.990 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.990 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157205+0000", 2024-09-11T06:50:58.990 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014496260000000001 2024-09-11T06:50:58.990 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.990 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.990 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.990 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157252+0000", 2024-09-11T06:50:58.990 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6368000000000001e-05 2024-09-11T06:50:58.991 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.991 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.991 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.991 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157286+0000", 2024-09-11T06:50:58.991 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4944999999999998e-05 2024-09-11T06:50:58.991 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.991 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.991 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.991 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157540+0000", 2024-09-11T06:50:58.991 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00025307999999999998 2024-09-11T06:50:58.991 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.991 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.991 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:58.991 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157576+0000", 2024-09-11T06:50:58.991 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.6878999999999997e-05 2024-09-11T06:50:58.992 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.992 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.992 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.992 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.160437+0000", 2024-09-11T06:50:58.992 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00286032 2024-09-11T06:50:58.992 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.992 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.992 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.992 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.992 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.992 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:50:58.992 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.280679+0000", 2024-09-11T06:50:58.993 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 23.674951781000001, 2024-09-11T06:50:58.993 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0040934309999999998, 2024-09-11T06:50:58.993 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.993 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.993 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:58.993 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.993 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.993 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.993 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280679+0000", 2024-09-11T06:50:58.993 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.993 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.993 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.993 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.993 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280679+0000", 2024-09-11T06:50:58.994 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.994 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.994 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.994 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.994 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280681+0000", 2024-09-11T06:50:58.994 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2529999999999999e-06 2024-09-11T06:50:58.994 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.994 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.994 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.994 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280713+0000", 2024-09-11T06:50:58.994 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.1936000000000001e-05 2024-09-11T06:50:58.994 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.994 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.995 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.995 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280758+0000", 2024-09-11T06:50:58.995 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5124999999999999e-05 2024-09-11T06:50:58.995 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.995 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.995 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.995 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280779+0000", 2024-09-11T06:50:58.995 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0936000000000001e-05 2024-09-11T06:50:58.995 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.995 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.995 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.995 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280932+0000", 2024-09-11T06:50:58.995 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015220899999999999 2024-09-11T06:50:58.995 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.996 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.996 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.996 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.284773+0000", 2024-09-11T06:50:58.996 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0038409719999999998 2024-09-11T06:50:58.996 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.996 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:58.996 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:58.996 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.996 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.996 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:50:58.996 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.292943+0000", 2024-09-11T06:50:58.996 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 23.662687587000001, 2024-09-11T06:50:58.996 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0028686010000000001, 2024-09-11T06:50:58.996 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:58.997 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:58.997 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:58.997 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:58.997 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.997 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:58.997 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.292943+0000", 2024-09-11T06:50:58.997 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.997 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.997 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.997 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:58.997 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.292943+0000", 2024-09-11T06:50:58.997 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:58.997 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.997 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.998 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:58.998 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.292945+0000", 2024-09-11T06:50:58.998 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7799999999999999e-06 2024-09-11T06:50:58.998 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.998 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.998 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:58.998 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.292968+0000", 2024-09-11T06:50:58.998 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2586e-05 2024-09-11T06:50:58.998 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.998 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.998 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:58.998 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.292998+0000", 2024-09-11T06:50:58.998 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9893e-05 2024-09-11T06:50:58.998 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.999 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.999 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:58.999 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293012+0000", 2024-09-11T06:50:58.999 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4504999999999999e-05 2024-09-11T06:50:58.999 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.999 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.999 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:58.999 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293155+0000", 2024-09-11T06:50:58.999 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000142993 2024-09-11T06:50:58.999 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:58.999 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:58.999 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:58.999 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.295812+0000", 2024-09-11T06:50:58.999 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0026568439999999998 2024-09-11T06:50:59.000 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.000 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.000 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.000 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.000 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.000 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:50:59.000 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.293438+0000", 2024-09-11T06:50:59.000 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 23.662193044999999, 2024-09-11T06:50:59.000 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0061479350000000002, 2024-09-11T06:50:59.000 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.000 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.000 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:59.000 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.001 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.001 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.001 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293438+0000", 2024-09-11T06:50:59.001 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.001 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.001 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.001 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.001 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293438+0000", 2024-09-11T06:50:59.001 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.001 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.001 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.001 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.001 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293439+0000", 2024-09-11T06:50:59.001 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:50:59.002 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.002 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.002 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.002 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293459+0000", 2024-09-11T06:50:59.002 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9922e-05 2024-09-11T06:50:59.002 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.002 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.002 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.002 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293482+0000", 2024-09-11T06:50:59.002 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2969e-05 2024-09-11T06:50:59.002 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.002 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.002 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.002 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293494+0000", 2024-09-11T06:50:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.131e-05 2024-09-11T06:50:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.295901+0000", 2024-09-11T06:50:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024077949999999999 2024-09-11T06:50:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.299586+0000", 2024-09-11T06:50:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036845020000000001 2024-09-11T06:50:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.004 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.004 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:50:59.004 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.309418+0000", 2024-09-11T06:50:59.004 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 23.646213014000001, 2024-09-11T06:50:59.004 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031185879999999998, 2024-09-11T06:50:59.004 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.004 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.004 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:59.004 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.004 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.004 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.004 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.309418+0000", 2024-09-11T06:50:59.004 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.005 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.005 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.005 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.005 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.309418+0000", 2024-09-11T06:50:59.005 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.005 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.005 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.005 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.005 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.309419+0000", 2024-09-11T06:50:59.005 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4699999999999999e-06 2024-09-11T06:50:59.005 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.005 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.005 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.005 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.309441+0000", 2024-09-11T06:50:59.006 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1668999999999999e-05 2024-09-11T06:50:59.006 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.006 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.006 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.006 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.309463+0000", 2024-09-11T06:50:59.006 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2106000000000001e-05 2024-09-11T06:50:59.006 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.006 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.006 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.006 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.309477+0000", 2024-09-11T06:50:59.006 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3482999999999999e-05 2024-09-11T06:50:59.006 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.006 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.006 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.007 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310475+0000", 2024-09-11T06:50:59.007 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00099831099999999999 2024-09-11T06:50:59.007 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.007 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.007 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.007 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.312536+0000", 2024-09-11T06:50:59.007 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0020615490000000002 2024-09-11T06:50:59.007 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.007 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.007 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.007 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.007 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.007 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:50:59.007 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.315262+0000", 2024-09-11T06:50:59.008 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 23.640369206999999, 2024-09-11T06:50:59.008 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0028909550000000002, 2024-09-11T06:50:59.008 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.008 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.008 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:59.008 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.008 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.008 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.008 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.315262+0000", 2024-09-11T06:50:59.008 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.008 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.008 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.008 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.009 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.315262+0000", 2024-09-11T06:50:59.009 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.009 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.009 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.009 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.009 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.315264+0000", 2024-09-11T06:50:59.009 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1629999999999999e-06 2024-09-11T06:50:59.009 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.009 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.009 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.009 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.315291+0000", 2024-09-11T06:50:59.009 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7489000000000001e-05 2024-09-11T06:50:59.009 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.009 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.010 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.010 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.315329+0000", 2024-09-11T06:50:59.010 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7175999999999998e-05 2024-09-11T06:50:59.010 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.010 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.010 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.010 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.315348+0000", 2024-09-11T06:50:59.010 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9596e-05 2024-09-11T06:50:59.010 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.010 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.010 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.010 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.315478+0000", 2024-09-11T06:50:59.010 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129975 2024-09-11T06:50:59.011 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.011 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.011 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.011 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.318153+0000", 2024-09-11T06:50:59.011 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0026745559999999998 2024-09-11T06:50:59.011 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.011 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.011 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.011 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.011 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.011 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:50:59.011 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.324823+0000", 2024-09-11T06:50:59.011 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 23.630808146, 2024-09-11T06:50:59.011 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033143130000000002, 2024-09-11T06:50:59.012 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.012 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.012 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:59.012 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.012 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.012 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.012 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.324823+0000", 2024-09-11T06:50:59.012 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.012 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.012 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.012 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.012 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.324823+0000", 2024-09-11T06:50:59.012 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.013 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.013 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.013 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.013 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.324825+0000", 2024-09-11T06:50:59.013 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.88e-06 2024-09-11T06:50:59.013 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.013 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.013 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.013 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.324851+0000", 2024-09-11T06:50:59.013 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6126e-05 2024-09-11T06:50:59.013 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.013 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.013 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.013 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.324882+0000", 2024-09-11T06:50:59.014 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0778999999999998e-05 2024-09-11T06:50:59.014 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.014 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.014 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.014 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.324902+0000", 2024-09-11T06:50:59.014 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0046000000000001e-05 2024-09-11T06:50:59.014 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.014 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.014 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.014 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.325015+0000", 2024-09-11T06:50:59.014 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000113739 2024-09-11T06:50:59.014 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.014 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.014 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.015 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.328137+0000", 2024-09-11T06:50:59.015 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031217430000000002 2024-09-11T06:50:59.015 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.015 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.015 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.015 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.015 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.015 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:50:59.015 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.327802+0000", 2024-09-11T06:50:59.015 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 23.627829130999999, 2024-09-11T06:50:59.015 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027834639999999998, 2024-09-11T06:50:59.015 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.015 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.015 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:59.016 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.016 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.016 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.016 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.327802+0000", 2024-09-11T06:50:59.016 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.016 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.016 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.016 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.016 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.327802+0000", 2024-09-11T06:50:59.016 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.016 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.016 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.016 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.017 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.327804+0000", 2024-09-11T06:50:59.017 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7370000000000001e-06 2024-09-11T06:50:59.017 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.017 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.017 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.017 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.327828+0000", 2024-09-11T06:50:59.017 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4831999999999999e-05 2024-09-11T06:50:59.017 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.017 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.017 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.017 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.327854+0000", 2024-09-11T06:50:59.017 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5525999999999999e-05 2024-09-11T06:50:59.017 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.017 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.018 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.018 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.327870+0000", 2024-09-11T06:50:59.018 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6330000000000001e-05 2024-09-11T06:50:59.018 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.018 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.018 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.018 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.328230+0000", 2024-09-11T06:50:59.018 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036026800000000001 2024-09-11T06:50:59.018 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.018 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.018 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.018 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.330585+0000", 2024-09-11T06:50:59.018 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0023547709999999999 2024-09-11T06:50:59.018 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.019 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.019 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.019 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.019 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.019 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.7 e70/69 cost 1048576)", 2024-09-11T06:50:59.019 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.343042+0000", 2024-09-11T06:50:59.019 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 23.612588528, 2024-09-11T06:50:59.019 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031815010000000002, 2024-09-11T06:50:59.019 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.019 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.019 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:59.019 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.019 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.020 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.020 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.343042+0000", 2024-09-11T06:50:59.020 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.020 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.020 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.020 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.020 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.343042+0000", 2024-09-11T06:50:59.020 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.020 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.020 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.020 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.020 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.343044+0000", 2024-09-11T06:50:59.020 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.716e-06 2024-09-11T06:50:59.020 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.021 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.021 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.021 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.343071+0000", 2024-09-11T06:50:59.021 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.654e-05 2024-09-11T06:50:59.021 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.021 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.021 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.021 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.343098+0000", 2024-09-11T06:50:59.021 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7529e-05 2024-09-11T06:50:59.021 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.021 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.021 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.021 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.343115+0000", 2024-09-11T06:50:59.021 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6629e-05 2024-09-11T06:50:59.022 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.022 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.022 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.022 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.343229+0000", 2024-09-11T06:50:59.022 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011443900000000001 2024-09-11T06:50:59.022 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.022 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.022 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.022 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.346224+0000", 2024-09-11T06:50:59.022 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0029946479999999999 2024-09-11T06:50:59.022 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.022 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.022 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.022 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.023 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.023 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.7 e70/69 cost 1048576)", 2024-09-11T06:50:59.023 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.352608+0000", 2024-09-11T06:50:59.023 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 23.603023243999999, 2024-09-11T06:50:59.023 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0032189060000000001, 2024-09-11T06:50:59.023 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.023 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.023 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:59.023 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.023 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.023 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.023 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.352608+0000", 2024-09-11T06:50:59.023 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.024 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.024 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.024 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.024 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.352608+0000", 2024-09-11T06:50:59.024 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.024 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.024 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.024 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.024 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.352610+0000", 2024-09-11T06:50:59.024 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.627e-06 2024-09-11T06:50:59.024 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.024 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.025 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.025 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.352641+0000", 2024-09-11T06:50:59.025 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0442000000000001e-05 2024-09-11T06:50:59.025 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.025 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.025 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.025 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.352673+0000", 2024-09-11T06:50:59.025 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2493000000000002e-05 2024-09-11T06:50:59.025 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.025 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.025 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.025 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.352693+0000", 2024-09-11T06:50:59.025 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9911999999999998e-05 2024-09-11T06:50:59.025 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.026 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.026 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.026 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.352785+0000", 2024-09-11T06:50:59.026 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.1539999999999994e-05 2024-09-11T06:50:59.026 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.026 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.026 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.026 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.355827+0000", 2024-09-11T06:50:59.026 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003041892 2024-09-11T06:50:59.026 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.026 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.026 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.026 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.027 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.027 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.7 e70/69 cost 1048576)", 2024-09-11T06:50:59.027 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.379470+0000", 2024-09-11T06:50:59.027 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 23.576160451, 2024-09-11T06:50:59.027 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0029267730000000001, 2024-09-11T06:50:59.027 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.027 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.027 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:59.027 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.027 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.027 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.027 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.379470+0000", 2024-09-11T06:50:59.027 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.028 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.028 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.028 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.028 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.379470+0000", 2024-09-11T06:50:59.028 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.028 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.028 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.028 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.028 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.379472+0000", 2024-09-11T06:50:59.028 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6729999999999999e-06 2024-09-11T06:50:59.028 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.028 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.028 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.029 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.379494+0000", 2024-09-11T06:50:59.029 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1823000000000001e-05 2024-09-11T06:50:59.029 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.029 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.029 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.029 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.379521+0000", 2024-09-11T06:50:59.029 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7161999999999998e-05 2024-09-11T06:50:59.029 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.029 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.029 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.029 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.379538+0000", 2024-09-11T06:50:59.029 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7212999999999999e-05 2024-09-11T06:50:59.029 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.030 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.030 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.030 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.379662+0000", 2024-09-11T06:50:59.030 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000123776 2024-09-11T06:50:59.030 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.030 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.030 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.030 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.382397+0000", 2024-09-11T06:50:59.030 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027351260000000001 2024-09-11T06:50:59.030 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.030 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.030 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.030 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.030 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.031 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.7 e70/69 cost 1048576)", 2024-09-11T06:50:59.031 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.466322+0000", 2024-09-11T06:50:59.031 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 23.489308765000001, 2024-09-11T06:50:59.031 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002802531, 2024-09-11T06:50:59.031 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.031 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.031 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:59.031 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.031 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.031 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.031 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.466322+0000", 2024-09-11T06:50:59.031 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.031 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.031 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.032 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.032 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.466322+0000", 2024-09-11T06:50:59.032 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.032 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.032 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.032 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.032 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.466324+0000", 2024-09-11T06:50:59.032 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7770000000000001e-06 2024-09-11T06:50:59.032 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.032 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.032 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.032 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.466345+0000", 2024-09-11T06:50:59.032 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0716e-05 2024-09-11T06:50:59.033 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.033 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.033 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.033 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.466372+0000", 2024-09-11T06:50:59.033 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7583e-05 2024-09-11T06:50:59.033 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.033 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.033 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.033 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.466386+0000", 2024-09-11T06:50:59.033 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3532e-05 2024-09-11T06:50:59.033 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.033 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.033 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.033 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.466545+0000", 2024-09-11T06:50:59.034 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000159333 2024-09-11T06:50:59.034 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.034 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.034 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.034 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.469125+0000", 2024-09-11T06:50:59.034 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025795900000000001 2024-09-11T06:50:59.034 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.034 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.034 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.034 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.034 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.034 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.7 e70/69 cost 1048576)", 2024-09-11T06:50:59.034 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.497220+0000", 2024-09-11T06:50:59.034 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 23.458411111, 2024-09-11T06:50:59.035 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027665979999999999, 2024-09-11T06:50:59.035 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.035 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.035 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:50:59.035 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.035 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.035 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.035 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.497220+0000", 2024-09-11T06:50:59.035 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.035 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.035 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.035 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.035 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.497220+0000", 2024-09-11T06:50:59.035 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.036 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.036 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.036 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.036 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.497222+0000", 2024-09-11T06:50:59.036 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.883e-06 2024-09-11T06:50:59.036 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.036 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.036 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.036 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.497250+0000", 2024-09-11T06:50:59.036 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8293000000000002e-05 2024-09-11T06:50:59.036 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.036 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.036 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.037 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.497286+0000", 2024-09-11T06:50:59.037 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5651999999999997e-05 2024-09-11T06:50:59.037 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.037 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.037 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.037 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.497303+0000", 2024-09-11T06:50:59.037 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7767000000000001e-05 2024-09-11T06:50:59.037 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.037 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.037 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.037 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.497424+0000", 2024-09-11T06:50:59.037 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000120432 2024-09-11T06:50:59.037 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.037 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.037 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.038 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.499986+0000", 2024-09-11T06:50:59.038 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002562571 2024-09-11T06:50:59.038 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.038 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.038 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.038 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.038 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.038 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:59.040 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:50:59.040 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:59.034+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:59.140 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:59.138+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:59.214 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:59.214 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:59.214 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:50:59.214 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:59.228 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:50:59.272 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:59.270+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:59.342 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:59.338+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:59.389 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:59.389 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:50:59.389 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:50:59.389 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:50:59.389 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:59.403 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:50:59.417 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:50:59.448 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:59.446+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:59.524 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:50:59.524 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:50:59.524 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:50:59.525 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:50:59.525 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.525 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:50:59.525 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:50:59.525 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 108.29298559199999, 2024-09-11T06:50:59.525 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010553425999999999, 2024-09-11T06:50:59.525 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.525 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.525 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:59.525 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:59.525 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:59.525 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:59.525 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:50:59.526 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.526 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.526 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.526 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.526 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:50:59.526 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.526 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.526 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.526 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.526 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:50:59.526 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.526 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.526 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.527 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.527 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230274+0000", 2024-09-11T06:50:59.527 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.533e-06 2024-09-11T06:50:59.527 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.527 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.527 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.527 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230285+0000", 2024-09-11T06:50:59.527 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0540000000000001e-05 2024-09-11T06:50:59.527 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.527 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.527 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.527 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230309+0000", 2024-09-11T06:50:59.527 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4223000000000002e-05 2024-09-11T06:50:59.527 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.528 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.528 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.528 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230327+0000", 2024-09-11T06:50:59.528 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:50:59.528 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.528 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.528 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.528 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230478+0000", 2024-09-11T06:50:59.528 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000151459 2024-09-11T06:50:59.528 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.528 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.528 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:59.528 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230490+0000", 2024-09-11T06:50:59.529 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.147e-05 2024-09-11T06:50:59.529 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.529 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.529 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.529 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231803+0000", 2024-09-11T06:50:59.529 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001313158 2024-09-11T06:50:59.529 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.529 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.529 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:50:59.529 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231817+0000", 2024-09-11T06:50:59.529 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4197e-05 2024-09-11T06:50:59.529 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.529 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.529 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.529 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.235945+0000", 2024-09-11T06:50:59.530 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0041282480000000002 2024-09-11T06:50:59.530 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.530 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.530 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:59.530 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236312+0000", 2024-09-11T06:50:59.530 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036652700000000001 2024-09-11T06:50:59.530 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.530 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.530 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:50:59.530 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236689+0000", 2024-09-11T06:50:59.530 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000377117 2024-09-11T06:50:59.532 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.532 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.532 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:59.532 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238460+0000", 2024-09-11T06:50:59.533 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001770918 2024-09-11T06:50:59.533 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.533 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.533 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:59.533 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240627+0000", 2024-09-11T06:50:59.533 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021670779999999998 2024-09-11T06:50:59.533 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.533 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.533 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:59.533 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240751+0000", 2024-09-11T06:50:59.533 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012384900000000001 2024-09-11T06:50:59.533 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.533 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.533 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240821+0000", 2024-09-11T06:50:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0516e-05 2024-09-11T06:50:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-11T06:50:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:50:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 107.29081474900001, 2024-09-11T06:50:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069904770000000001, 2024-09-11T06:50:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.534 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 5 2024-09-11T06:50:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:50:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.535 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:50:59.536 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.536 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.536 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.536 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232442+0000", 2024-09-11T06:50:59.536 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.044e-06 2024-09-11T06:50:59.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.536 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.536 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.536 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232460+0000", 2024-09-11T06:50:59.536 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7863000000000002e-05 2024-09-11T06:50:59.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.536 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.537 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.537 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232492+0000", 2024-09-11T06:50:59.537 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2812999999999999e-05 2024-09-11T06:50:59.537 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.537 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.537 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.537 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232510+0000", 2024-09-11T06:50:59.537 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7552999999999999e-05 2024-09-11T06:50:59.537 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.537 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.537 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.537 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232640+0000", 2024-09-11T06:50:59.537 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013029900000000001 2024-09-11T06:50:59.537 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.538 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.538 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:59.538 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232650+0000", 2024-09-11T06:50:59.538 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0023999999999999e-05 2024-09-11T06:50:59.538 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.538 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.538 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.538 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235641+0000", 2024-09-11T06:50:59.538 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00299089 2024-09-11T06:50:59.538 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.538 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.538 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:59.538 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235980+0000", 2024-09-11T06:50:59.538 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000338907 2024-09-11T06:50:59.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.539 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.539 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:59.539 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.236351+0000", 2024-09-11T06:50:59.539 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037087699999999998 2024-09-11T06:50:59.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.539 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.539 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:59.539 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238074+0000", 2024-09-11T06:50:59.539 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001722859 2024-09-11T06:50:59.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.539 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.539 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:59.539 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239160+0000", 2024-09-11T06:50:59.540 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001086074 2024-09-11T06:50:59.540 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.540 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.540 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:59.540 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239299+0000", 2024-09-11T06:50:59.540 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013933500000000001 2024-09-11T06:50:59.540 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.540 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.540 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.540 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239429+0000", 2024-09-11T06:50:59.540 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129939 2024-09-11T06:50:59.540 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.540 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.541 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.541 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.541 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.541 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-11T06:50:59.541 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:50:59.541 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 106.27672679699999, 2024-09-11T06:50:59.541 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069253250000000004, 2024-09-11T06:50:59.541 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.541 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.541 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:59.541 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:59.541 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:50:59.541 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:50:59.541 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 7 2024-09-11T06:50:59.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.542 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.542 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.542 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.542 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:50:59.542 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.542 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.542 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.542 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:50:59.542 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.542 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.543 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.543 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246530+0000", 2024-09-11T06:50:59.543 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9469999999999999e-06 2024-09-11T06:50:59.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.543 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.543 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.543 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246545+0000", 2024-09-11T06:50:59.543 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5973000000000001e-05 2024-09-11T06:50:59.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.543 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.543 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.543 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246578+0000", 2024-09-11T06:50:59.543 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2759999999999998e-05 2024-09-11T06:50:59.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.544 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.544 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.544 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246595+0000", 2024-09-11T06:50:59.544 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6939000000000002e-05 2024-09-11T06:50:59.544 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.544 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.544 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.544 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246770+0000", 2024-09-11T06:50:59.544 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000174723 2024-09-11T06:50:59.544 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.544 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.544 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:59.544 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246789+0000", 2024-09-11T06:50:59.544 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9159000000000001e-05 2024-09-11T06:50:59.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.545 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.545 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.545 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250337+0000", 2024-09-11T06:50:59.545 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035480759999999998 2024-09-11T06:50:59.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.545 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.545 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:59.545 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250549+0000", 2024-09-11T06:50:59.545 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021192199999999999 2024-09-11T06:50:59.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.545 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.545 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:50:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250750+0000", 2024-09-11T06:50:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020104199999999999 2024-09-11T06:50:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:50:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.252455+0000", 2024-09-11T06:50:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001705249 2024-09-11T06:50:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:50:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253305+0000", 2024-09-11T06:50:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00084941200000000002 2024-09-11T06:50:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.546 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.547 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:59.547 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253388+0000", 2024-09-11T06:50:59.547 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3642999999999999e-05 2024-09-11T06:50:59.547 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.547 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.547 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.547 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253452+0000", 2024-09-11T06:50:59.547 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3479999999999995e-05 2024-09-11T06:50:59.547 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.547 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.547 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.547 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.547 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.548 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(client.4558.0:5 3.2 e57/55 3:5cea7035:::rbd_info:head v 57'1, mlcod=0'0)", 2024-09-11T06:50:59.548 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:50:59.548 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 41.428622435000001, 2024-09-11T06:50:59.548 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011016419999999999, 2024-09-11T06:50:59.548 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.548 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.548 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:50:59.548 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.548 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.548 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.548 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:50:59.548 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.548 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.549 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.549 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.549 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:50:59.549 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.549 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.549 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.549 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.549 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:50:59.549 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4000000000000002e-07 2024-09-11T06:50:59.549 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.549 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.549 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.549 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094640+0000", 2024-09-11T06:50:59.549 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6230000000000003e-06 2024-09-11T06:50:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094669+0000", 2024-09-11T06:50:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8462e-05 2024-09-11T06:50:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094684+0000", 2024-09-11T06:50:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5536000000000002e-05 2024-09-11T06:50:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.550 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094798+0000", 2024-09-11T06:50:59.551 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011439599999999999 2024-09-11T06:50:59.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.551 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.551 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:59.551 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094819+0000", 2024-09-11T06:50:59.551 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0296e-05 2024-09-11T06:50:59.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.551 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.551 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:50:59.551 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.095687+0000", 2024-09-11T06:50:59.551 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086851799999999998 2024-09-11T06:50:59.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.551 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.095733+0000", 2024-09-11T06:50:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5271000000000001e-05 2024-09-11T06:50:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:160 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 4096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:50:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 38.489507428000003, 2024-09-11T06:50:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001122215, 2024-09-11T06:50:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.552 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.553 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:59.553 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:59.553 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:59.553 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:59.553 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 160 2024-09-11T06:50:59.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.553 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.553 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.553 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:50:59.553 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.554 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.554 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:50:59.554 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.554 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.554 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.554 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:50:59.554 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0999999999999999e-07 2024-09-11T06:50:59.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.554 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.554 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.554 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033749+0000", 2024-09-11T06:50:59.554 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8930000000000001e-06 2024-09-11T06:50:59.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033758+0000", 2024-09-11T06:50:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.242e-06 2024-09-11T06:50:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033764+0000", 2024-09-11T06:50:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0510000000000002e-06 2024-09-11T06:50:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.034207+0000", 2024-09-11T06:50:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00044320600000000003 2024-09-11T06:50:59.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.556 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.556 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:59.556 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.034280+0000", 2024-09-11T06:50:59.556 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3094000000000002e-05 2024-09-11T06:50:59.556 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.556 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.556 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.556 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.034868+0000", 2024-09-11T06:50:59.556 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00058821899999999996 2024-09-11T06:50:59.556 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.556 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.556 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.556 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.556 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.557 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:162 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:59.557 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.039519+0000", 2024-09-11T06:50:59.557 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 38.483734310000003, 2024-09-11T06:50:59.557 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001298485, 2024-09-11T06:50:59.557 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.557 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.557 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:59.557 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:59.557 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:59.557 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:59.557 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 162 2024-09-11T06:50:59.557 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.557 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.558 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.558 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.558 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039519+0000", 2024-09-11T06:50:59.558 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.558 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.558 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.558 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039519+0000", 2024-09-11T06:50:59.558 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.558 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.558 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.558 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039520+0000", 2024-09-11T06:50:59.559 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9299999999999998e-07 2024-09-11T06:50:59.559 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.559 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.559 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.559 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039523+0000", 2024-09-11T06:50:59.559 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.247e-06 2024-09-11T06:50:59.559 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.559 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.559 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.559 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039529+0000", 2024-09-11T06:50:59.559 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8300000000000001e-06 2024-09-11T06:50:59.559 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.560 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.560 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039533+0000", 2024-09-11T06:50:59.560 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.2379999999999999e-06 2024-09-11T06:50:59.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.560 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.560 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.040236+0000", 2024-09-11T06:50:59.560 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00070326500000000001 2024-09-11T06:50:59.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.560 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:59.560 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.040299+0000", 2024-09-11T06:50:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3077999999999995e-05 2024-09-11T06:50:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.040818+0000", 2024-09-11T06:50:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00051833399999999996 2024-09-11T06:50:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:164 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 528384~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.041536+0000", 2024-09-11T06:50:59.561 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 38.481717588000002, 2024-09-11T06:50:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00119962, 2024-09-11T06:50:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 164 2024-09-11T06:50:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041536+0000", 2024-09-11T06:50:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.563 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.563 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041536+0000", 2024-09-11T06:50:59.563 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.563 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.563 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.563 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041537+0000", 2024-09-11T06:50:59.563 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3e-06 2024-09-11T06:50:59.563 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.563 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.563 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041543+0000", 2024-09-11T06:50:59.563 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2360000000000001e-06 2024-09-11T06:50:59.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.564 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.564 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041558+0000", 2024-09-11T06:50:59.564 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4909000000000001e-05 2024-09-11T06:50:59.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.564 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.564 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041569+0000", 2024-09-11T06:50:59.564 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0946e-05 2024-09-11T06:50:59.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.564 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.564 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.042138+0000", 2024-09-11T06:50:59.565 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000568996 2024-09-11T06:50:59.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.565 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.565 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:59.565 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.042190+0000", 2024-09-11T06:50:59.565 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1609e-05 2024-09-11T06:50:59.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.565 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.565 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.565 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.042735+0000", 2024-09-11T06:50:59.565 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00054562399999999998 2024-09-11T06:50:59.565 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.565 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.566 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.566 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.566 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:166 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 790528~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:59.566 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.043564+0000", 2024-09-11T06:50:59.566 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 38.479689190999999, 2024-09-11T06:50:59.566 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011042000000000001, 2024-09-11T06:50:59.566 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.566 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.566 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:59.566 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:59.566 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:59.566 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:59.566 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 166 2024-09-11T06:50:59.567 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.567 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.567 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.567 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.567 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043564+0000", 2024-09-11T06:50:59.567 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.567 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.567 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.567 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.567 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043564+0000", 2024-09-11T06:50:59.567 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.567 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.567 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.567 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.568 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043565+0000", 2024-09-11T06:50:59.568 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0449999999999999e-06 2024-09-11T06:50:59.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.568 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.568 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.568 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043571+0000", 2024-09-11T06:50:59.568 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.6849999999999998e-06 2024-09-11T06:50:59.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.568 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.568 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.568 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043586+0000", 2024-09-11T06:50:59.568 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4902000000000001e-05 2024-09-11T06:50:59.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.569 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.569 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043593+0000", 2024-09-11T06:50:59.569 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9240000000000001e-06 2024-09-11T06:50:59.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.569 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.569 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.044073+0000", 2024-09-11T06:50:59.569 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000479813 2024-09-11T06:50:59.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.569 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:59.569 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.044123+0000", 2024-09-11T06:50:59.569 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0908000000000001e-05 2024-09-11T06:50:59.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.570 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.570 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.044668+0000", 2024-09-11T06:50:59.570 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000544923 2024-09-11T06:50:59.570 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.570 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.570 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.570 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:178 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 2363392~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:59.570 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.061599+0000", 2024-09-11T06:50:59.570 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 38.461654476, 2024-09-11T06:50:59.570 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012194529999999999, 2024-09-11T06:50:59.571 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.571 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.571 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:59.571 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:59.571 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:59.571 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:59.571 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 178 2024-09-11T06:50:59.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.571 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.571 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.571 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061599+0000", 2024-09-11T06:50:59.571 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.572 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.572 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061599+0000", 2024-09-11T06:50:59.572 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.572 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.572 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061600+0000", 2024-09-11T06:50:59.572 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3600000000000003e-07 2024-09-11T06:50:59.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.572 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.572 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061603+0000", 2024-09-11T06:50:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.6550000000000002e-06 2024-09-11T06:50:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061618+0000", 2024-09-11T06:50:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4377e-05 2024-09-11T06:50:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061626+0000", 2024-09-11T06:50:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.0679999999999997e-06 2024-09-11T06:50:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.573 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.062098+0000", 2024-09-11T06:50:59.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00047251299999999998 2024-09-11T06:50:59.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.574 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:59.574 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.062253+0000", 2024-09-11T06:50:59.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015533100000000001 2024-09-11T06:50:59.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.574 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.574 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.062818+0000", 2024-09-11T06:50:59.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00056487300000000002 2024-09-11T06:50:59.574 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.574 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:184 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 3149824~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:50:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 38.457044773, 2024-09-11T06:50:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010685969999999999, 2024-09-11T06:50:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 184 2024-09-11T06:50:59.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:50:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:50:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.576 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:50:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.6700000000000003e-07 2024-09-11T06:50:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066212+0000", 2024-09-11T06:50:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.1669999999999999e-06 2024-09-11T06:50:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066219+0000", 2024-09-11T06:50:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5699999999999998e-06 2024-09-11T06:50:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066223+0000", 2024-09-11T06:50:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3540000000000002e-06 2024-09-11T06:50:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066713+0000", 2024-09-11T06:50:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00048985399999999996 2024-09-11T06:50:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066813+0000", 2024-09-11T06:50:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00010003 2024-09-11T06:50:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.579 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.067277+0000", 2024-09-11T06:50:59.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000463955 2024-09-11T06:50:59.579 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.579 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.579 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.579 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:186 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 3411968~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:59.579 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.071198+0000", 2024-09-11T06:50:59.579 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 38.452055428999998, 2024-09-11T06:50:59.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012515779999999999, 2024-09-11T06:50:59.579 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.579 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 186 2024-09-11T06:50:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071198+0000", 2024-09-11T06:50:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.580 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071198+0000", 2024-09-11T06:50:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071199+0000", 2024-09-11T06:50:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0299999999999999e-07 2024-09-11T06:50:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071201+0000", 2024-09-11T06:50:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8540000000000001e-06 2024-09-11T06:50:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071209+0000", 2024-09-11T06:50:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3629999999999996e-06 2024-09-11T06:50:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071214+0000", 2024-09-11T06:50:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2109999999999997e-06 2024-09-11T06:50:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071915+0000", 2024-09-11T06:50:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00070101799999999998 2024-09-11T06:50:59.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.072011+0000", 2024-09-11T06:50:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.5755999999999996e-05 2024-09-11T06:50:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.072450+0000", 2024-09-11T06:50:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000438773 2024-09-11T06:50:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.584 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:190 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 3936256~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:59.584 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.074875+0000", 2024-09-11T06:50:59.584 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 38.448378513999998, 2024-09-11T06:50:59.584 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0013119620000000001, 2024-09-11T06:50:59.584 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.584 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.584 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:59.584 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:59.584 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:59.584 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:59.584 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 190 2024-09-11T06:50:59.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.584 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074875+0000", 2024-09-11T06:50:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074875+0000", 2024-09-11T06:50:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074876+0000", 2024-09-11T06:50:59.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7400000000000003e-07 2024-09-11T06:50:59.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074887+0000", 2024-09-11T06:50:59.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1817999999999999e-05 2024-09-11T06:50:59.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074896+0000", 2024-09-11T06:50:59.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9120000000000004e-06 2024-09-11T06:50:59.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074900+0000", 2024-09-11T06:50:59.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.2220000000000004e-06 2024-09-11T06:50:59.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.587 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.587 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.075198+0000", 2024-09-11T06:50:59.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00029750399999999998 2024-09-11T06:50:59.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.587 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:59.587 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.075430+0000", 2024-09-11T06:50:59.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023200400000000001 2024-09-11T06:50:59.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.588 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.588 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.076187+0000", 2024-09-11T06:50:59.588 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00075682799999999997 2024-09-11T06:50:59.588 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.588 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.588 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.588 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:359 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 135168~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:59.588 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.298832+0000", 2024-09-11T06:50:59.588 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 38.224420879999997, 2024-09-11T06:50:59.588 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001078575, 2024-09-11T06:50:59.588 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.588 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.589 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:59.589 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:59.589 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:59.589 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:59.589 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 359 2024-09-11T06:50:59.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.589 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.589 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.589 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298832+0000", 2024-09-11T06:50:59.589 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.589 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.590 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298832+0000", 2024-09-11T06:50:59.590 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.590 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298834+0000", 2024-09-11T06:50:59.590 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.646e-06 2024-09-11T06:50:59.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.590 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298845+0000", 2024-09-11T06:50:59.590 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.081e-05 2024-09-11T06:50:59.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.591 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.591 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298872+0000", 2024-09-11T06:50:59.591 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7382e-05 2024-09-11T06:50:59.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.591 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.591 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298884+0000", 2024-09-11T06:50:59.591 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1956e-05 2024-09-11T06:50:59.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.591 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.591 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298959+0000", 2024-09-11T06:50:59.591 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5214000000000002e-05 2024-09-11T06:50:59.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.592 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:59.592 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.299090+0000", 2024-09-11T06:50:59.592 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000130247 2024-09-11T06:50:59.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.592 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.592 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.299911+0000", 2024-09-11T06:50:59.592 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00082131999999999995 2024-09-11T06:50:59.592 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.592 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.592 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.593 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:360 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:59.593 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.298914+0000", 2024-09-11T06:50:59.593 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 38.224339209999997, 2024-09-11T06:50:59.593 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0023571690000000001, 2024-09-11T06:50:59.593 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.593 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.593 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:59.593 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:59.593 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:59.593 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:59.593 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 360 2024-09-11T06:50:59.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.593 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.594 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.594 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298914+0000", 2024-09-11T06:50:59.594 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.594 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.594 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298914+0000", 2024-09-11T06:50:59.594 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.594 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.594 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298915+0000", 2024-09-11T06:50:59.594 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9300000000000006e-07 2024-09-11T06:50:59.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.595 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.595 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298921+0000", 2024-09-11T06:50:59.595 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.6030000000000004e-06 2024-09-11T06:50:59.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.595 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.595 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298932+0000", 2024-09-11T06:50:59.595 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.082e-05 2024-09-11T06:50:59.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.595 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.596 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298939+0000", 2024-09-11T06:50:59.596 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7919999999999993e-06 2024-09-11T06:50:59.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.596 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.596 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.300011+0000", 2024-09-11T06:50:59.596 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001071179 2024-09-11T06:50:59.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.596 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:59.596 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.300245+0000", 2024-09-11T06:50:59.596 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000234434 2024-09-11T06:50:59.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.597 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.597 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.301271+0000", 2024-09-11T06:50:59.597 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010263480000000001 2024-09-11T06:50:59.597 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.597 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.597 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.597 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:362 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 528384~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:59.597 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.302376+0000", 2024-09-11T06:50:59.597 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 38.220876887000003, 2024-09-11T06:50:59.597 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014132909999999999, 2024-09-11T06:50:59.597 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.598 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.598 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:59.598 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:59.598 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:59.598 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:59.598 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 362 2024-09-11T06:50:59.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.598 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.598 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.598 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302376+0000", 2024-09-11T06:50:59.598 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.599 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.599 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302376+0000", 2024-09-11T06:50:59.599 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.599 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.599 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302378+0000", 2024-09-11T06:50:59.599 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1629999999999999e-06 2024-09-11T06:50:59.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.599 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.599 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302383+0000", 2024-09-11T06:50:59.599 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.463e-06 2024-09-11T06:50:59.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.600 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.600 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302401+0000", 2024-09-11T06:50:59.600 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8219e-05 2024-09-11T06:50:59.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.600 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.600 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302411+0000", 2024-09-11T06:50:59.600 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9000000000000001e-06 2024-09-11T06:50:59.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.600 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.600 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302524+0000", 2024-09-11T06:50:59.601 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000113199 2024-09-11T06:50:59.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.601 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:59.601 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302745+0000", 2024-09-11T06:50:59.601 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022057399999999999 2024-09-11T06:50:59.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.601 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.601 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.303790+0000", 2024-09-11T06:50:59.601 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010447729999999999 2024-09-11T06:50:59.601 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.601 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.602 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.602 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:363 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 659456~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:59.602 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.302938+0000", 2024-09-11T06:50:59.602 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 38.220315294000002, 2024-09-11T06:50:59.602 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00185012, 2024-09-11T06:50:59.602 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.602 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.602 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:59.602 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:59.602 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:59.602 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:59.602 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 363 2024-09-11T06:50:59.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.603 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.603 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.603 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302938+0000", 2024-09-11T06:50:59.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.603 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.603 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302938+0000", 2024-09-11T06:50:59.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.603 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.604 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302939+0000", 2024-09-11T06:50:59.604 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2300000000000001e-06 2024-09-11T06:50:59.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.604 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.604 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302944+0000", 2024-09-11T06:50:59.604 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.493e-06 2024-09-11T06:50:59.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.604 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.604 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302960+0000", 2024-09-11T06:50:59.604 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.632e-05 2024-09-11T06:50:59.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.604 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.605 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302972+0000", 2024-09-11T06:50:59.605 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1885000000000001e-05 2024-09-11T06:50:59.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.605 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.605 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.303835+0000", 2024-09-11T06:50:59.605 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000863141 2024-09-11T06:50:59.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.605 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:59.605 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.303965+0000", 2024-09-11T06:50:59.605 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013032199999999999 2024-09-11T06:50:59.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.606 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.606 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.304788+0000", 2024-09-11T06:50:59.606 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00082272900000000002 2024-09-11T06:50:59.606 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.606 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.606 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.606 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.606 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:367 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 1183744~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:59.606 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.312392+0000", 2024-09-11T06:50:59.606 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 38.210861653999999, 2024-09-11T06:50:59.606 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011343460000000001, 2024-09-11T06:50:59.606 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.607 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.607 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:59.607 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:59.607 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:59.607 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:59.607 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 367 2024-09-11T06:50:59.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.607 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.607 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.607 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312392+0000", 2024-09-11T06:50:59.607 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.608 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.608 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312392+0000", 2024-09-11T06:50:59.608 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.608 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.608 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312393+0000", 2024-09-11T06:50:59.608 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6199999999999999e-06 2024-09-11T06:50:59.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.608 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.608 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312400+0000", 2024-09-11T06:50:59.608 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0700000000000001e-06 2024-09-11T06:50:59.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.609 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.609 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312420+0000", 2024-09-11T06:50:59.609 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9375999999999999e-05 2024-09-11T06:50:59.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.609 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.609 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312430+0000", 2024-09-11T06:50:59.609 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0209e-05 2024-09-11T06:50:59.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.609 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.609 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312549+0000", 2024-09-11T06:50:59.609 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000118779 2024-09-11T06:50:59.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.610 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:59.610 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312767+0000", 2024-09-11T06:50:59.610 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021786400000000001 2024-09-11T06:50:59.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.610 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.610 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313526+0000", 2024-09-11T06:50:59.610 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00075942800000000003 2024-09-11T06:50:59.610 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.610 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.610 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.611 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:368 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 1314816~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:59.611 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.312454+0000", 2024-09-11T06:50:59.611 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 38.210799129999998, 2024-09-11T06:50:59.611 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014704550000000001, 2024-09-11T06:50:59.611 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.611 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.611 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:59.611 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:59.611 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:59.611 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:59.611 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 368 2024-09-11T06:50:59.611 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.611 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.612 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.612 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.612 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312454+0000", 2024-09-11T06:50:59.612 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.612 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.612 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.612 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.612 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312454+0000", 2024-09-11T06:50:59.612 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.612 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.612 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.612 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.612 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312455+0000", 2024-09-11T06:50:59.612 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4399999999999999e-07 2024-09-11T06:50:59.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.613 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.613 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.613 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312474+0000", 2024-09-11T06:50:59.613 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8899e-05 2024-09-11T06:50:59.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.613 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.613 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.613 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312485+0000", 2024-09-11T06:50:59.613 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0793e-05 2024-09-11T06:50:59.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.613 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.613 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.614 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312492+0000", 2024-09-11T06:50:59.614 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.593e-06 2024-09-11T06:50:59.614 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.614 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.614 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.614 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313576+0000", 2024-09-11T06:50:59.614 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010837970000000001 2024-09-11T06:50:59.614 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.614 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.614 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:59.614 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313620+0000", 2024-09-11T06:50:59.614 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3640000000000002e-05 2024-09-11T06:50:59.614 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.614 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.615 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.615 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313925+0000", 2024-09-11T06:50:59.615 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00030498899999999998 2024-09-11T06:50:59.615 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.615 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.615 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.615 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.615 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.615 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:370 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 1576960~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:50:59.615 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.314817+0000", 2024-09-11T06:50:59.615 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 38.208436421000002, 2024-09-11T06:50:59.615 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011606190000000001, 2024-09-11T06:50:59.615 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.615 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.616 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:59.616 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:59.616 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:50:59.616 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:50:59.616 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 370 2024-09-11T06:50:59.616 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.616 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.616 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.616 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.616 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314817+0000", 2024-09-11T06:50:59.616 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.616 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.616 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.617 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.617 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314817+0000", 2024-09-11T06:50:59.617 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.617 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.617 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.617 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.617 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314818+0000", 2024-09-11T06:50:59.617 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.463e-06 2024-09-11T06:50:59.617 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.617 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.617 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.617 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314826+0000", 2024-09-11T06:50:59.617 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3860000000000001e-06 2024-09-11T06:50:59.617 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.617 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.618 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.618 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314842+0000", 2024-09-11T06:50:59.618 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6220000000000001e-05 2024-09-11T06:50:59.618 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.618 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.618 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.618 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314854+0000", 2024-09-11T06:50:59.618 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1562e-05 2024-09-11T06:50:59.618 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.618 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.618 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.618 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.315212+0000", 2024-09-11T06:50:59.618 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000358012 2024-09-11T06:50:59.618 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.619 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.619 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:59.619 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.315433+0000", 2024-09-11T06:50:59.619 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022103800000000001 2024-09-11T06:50:59.619 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.619 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.619 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.619 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.315978+0000", 2024-09-11T06:50:59.619 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00054493800000000002 2024-09-11T06:50:59.619 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.619 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.619 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.619 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.620 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.620 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:57 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:50:59.620 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.568529+0000", 2024-09-11T06:50:59.620 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 10.954723876999999, 2024-09-11T06:50:59.620 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004228229, 2024-09-11T06:50:59.620 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:50:59.620 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:50:59.620 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:50:59.620 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:50:59.620 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:50:59.620 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:50:59.620 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 57 2024-09-11T06:50:59.620 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.620 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:50:59.621 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.621 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:50:59.621 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568529+0000", 2024-09-11T06:50:59.621 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.621 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.621 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:50:59.621 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568529+0000", 2024-09-11T06:50:59.621 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:50:59.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.621 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.621 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:50:59.621 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568532+0000", 2024-09-11T06:50:59.621 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0000000000000001e-06 2024-09-11T06:50:59.622 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.622 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.622 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:50:59.622 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568547+0000", 2024-09-11T06:50:59.622 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4219999999999999e-05 2024-09-11T06:50:59.622 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.622 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.622 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:50:59.622 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568580+0000", 2024-09-11T06:50:59.622 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.3328999999999999e-05 2024-09-11T06:50:59.622 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.622 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.622 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:50:59.622 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568596+0000", 2024-09-11T06:50:59.622 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5990000000000001e-05 2024-09-11T06:50:59.623 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.623 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.623 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.623 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568728+0000", 2024-09-11T06:50:59.623 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013175400000000001 2024-09-11T06:50:59.623 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.623 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.623 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:50:59.623 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568738+0000", 2024-09-11T06:50:59.623 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0196e-05 2024-09-11T06:50:59.623 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.623 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.623 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:50:59.623 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.572464+0000", 2024-09-11T06:50:59.624 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0037264160000000002 2024-09-11T06:50:59.624 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.624 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.624 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:50:59.624 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.572584+0000", 2024-09-11T06:50:59.624 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000119438 2024-09-11T06:50:59.624 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:50:59.624 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:50:59.624 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:50:59.624 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.572758+0000", 2024-09-11T06:50:59.624 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017388599999999999 2024-09-11T06:50:59.624 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.624 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.624 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.625 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:50:59.625 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:50:59.625 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:50:59.626 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:50:59.626 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:50:59.627 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:50:59.627 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:50:59.627 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:59.622+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:50:59.627 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:50:59.628 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:50:59.628 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:50:59.628 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:50:59.629 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:50:59.639 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:50:59.639 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:50:59.660 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:50:59.660 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:50:59.727 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:50:59.722+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:50:59.744 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:59.744 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:59.758 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:50:59.827 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:50:59.822+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:50:59.840 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is set 2024-09-11T06:50:59.875 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:50:59.875 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:50:59.890 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:50:59.929 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:50:59.926+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:00.029 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:00.026+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:00.052 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:00.053 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:00.067 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:51:00.157 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:00.154+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:00.231 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:00.226+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:00.234 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:00.234 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:00.249 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:51:00.332 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:00.330+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:00.413 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:00.413 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:00.427 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:51:00.473 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:00.470+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:00.533 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:00.530+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:00.544 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:00.544 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:00.558 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:51:00.634 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:00.630+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:00.726 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:00.727 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:00.735 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:00.730+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:00.742 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:51:00.836 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:00.830+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:00.910 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:00.910 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:00.925 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:51:00.969 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:00.966+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:01.038 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:01.034+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:01.086 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:01.086 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:01.100 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:51:01.148 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:01.146+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:01.231 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:01.231 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:01.240 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:01.238+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:01.244 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:51:01.341 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:01.338+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:01.413 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:01.413 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:01.427 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:51:01.442 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:01.438+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:01.543 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:01.538+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:01.592 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:01.593 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:01.607 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:51:01.643 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:01.638+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:01.678 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:51:01.744 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:01.738+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:01.773 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:01.773 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:01.789 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:51:01.830 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:51:01.830 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:51:01.844 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:01.838+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:01.864 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T06:51:01.918 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:01.918 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:01.932 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:51:01.976 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:01.974+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:02.046 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:02.042+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:02.051 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:02.051 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:02.054 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:51:02.054 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:51:02.066 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:51:02.078 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:51:02.078 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:51:02.102 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:51:02.102 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:51:02.133 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:51:02.134 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:51:02.147 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:02.142+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:02.163 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:51:02.163 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:51:02.237 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:02.237 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:02.237 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:51:02.237 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:02.248 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:02.242+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:02.251 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:51:02.348 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:02.346+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:02.374 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:02.374 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:02.375 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:51:02.375 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:51:02.375 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:02.388 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:51:02.449 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:02.446+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:02.550 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:02.546+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:02.559 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:02.559 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:51:02.559 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:51:02.559 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:51:02.559 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.559 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:51:02.559 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:51:02.559 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 111.325929493, 2024-09-11T06:51:02.559 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010553425999999999, 2024-09-11T06:51:02.559 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:02.559 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:02.559 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:02.559 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:51:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:51:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:51:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:51:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:51:02.560 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.561 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.561 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:02.561 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230274+0000", 2024-09-11T06:51:02.561 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.533e-06 2024-09-11T06:51:02.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.561 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.561 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:02.561 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230285+0000", 2024-09-11T06:51:02.561 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0540000000000001e-05 2024-09-11T06:51:02.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.561 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.561 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:02.562 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230309+0000", 2024-09-11T06:51:02.562 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4223000000000002e-05 2024-09-11T06:51:02.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.562 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:02.562 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230327+0000", 2024-09-11T06:51:02.562 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:51:02.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.562 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:02.562 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230478+0000", 2024-09-11T06:51:02.562 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000151459 2024-09-11T06:51:02.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230490+0000", 2024-09-11T06:51:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.147e-05 2024-09-11T06:51:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231803+0000", 2024-09-11T06:51:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001313158 2024-09-11T06:51:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:51:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231817+0000", 2024-09-11T06:51:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4197e-05 2024-09-11T06:51:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.235945+0000", 2024-09-11T06:51:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0041282480000000002 2024-09-11T06:51:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236312+0000", 2024-09-11T06:51:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036652700000000001 2024-09-11T06:51:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236689+0000", 2024-09-11T06:51:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000377117 2024-09-11T06:51:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.565 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:02.565 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238460+0000", 2024-09-11T06:51:02.565 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001770918 2024-09-11T06:51:02.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.565 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.565 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:02.565 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240627+0000", 2024-09-11T06:51:02.565 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021670779999999998 2024-09-11T06:51:02.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.565 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.565 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:02.565 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240751+0000", 2024-09-11T06:51:02.565 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012384900000000001 2024-09-11T06:51:02.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.566 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.566 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:02.566 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240821+0000", 2024-09-11T06:51:02.566 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0516e-05 2024-09-11T06:51:02.566 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.566 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:02.566 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.566 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.566 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-11T06:51:02.566 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:51:02.566 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 110.32375865, 2024-09-11T06:51:02.566 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069904770000000001, 2024-09-11T06:51:02.567 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:02.567 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:02.567 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:02.567 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:02.567 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:51:02.567 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:51:02.567 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 5 2024-09-11T06:51:02.567 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.567 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:02.567 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.567 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:02.567 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:51:02.567 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.567 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.568 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.568 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:02.568 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:51:02.568 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.568 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.568 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:02.568 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232442+0000", 2024-09-11T06:51:02.568 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.044e-06 2024-09-11T06:51:02.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.568 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.568 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:02.568 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232460+0000", 2024-09-11T06:51:02.568 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7863000000000002e-05 2024-09-11T06:51:02.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.569 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:02.569 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232492+0000", 2024-09-11T06:51:02.569 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2812999999999999e-05 2024-09-11T06:51:02.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.569 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:02.569 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232510+0000", 2024-09-11T06:51:02.569 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7552999999999999e-05 2024-09-11T06:51:02.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.569 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:02.569 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232640+0000", 2024-09-11T06:51:02.570 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013029900000000001 2024-09-11T06:51:02.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.570 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:02.570 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232650+0000", 2024-09-11T06:51:02.570 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0023999999999999e-05 2024-09-11T06:51:02.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.570 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:02.570 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235641+0000", 2024-09-11T06:51:02.570 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00299089 2024-09-11T06:51:02.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.570 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:02.570 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235980+0000", 2024-09-11T06:51:02.571 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000338907 2024-09-11T06:51:02.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.571 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:02.571 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.236351+0000", 2024-09-11T06:51:02.571 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037087699999999998 2024-09-11T06:51:02.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.571 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:02.571 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238074+0000", 2024-09-11T06:51:02.571 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001722859 2024-09-11T06:51:02.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.571 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:02.572 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239160+0000", 2024-09-11T06:51:02.572 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001086074 2024-09-11T06:51:02.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.572 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:02.572 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239299+0000", 2024-09-11T06:51:02.572 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013933500000000001 2024-09-11T06:51:02.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.572 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:02.572 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239429+0000", 2024-09-11T06:51:02.572 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129939 2024-09-11T06:51:02.572 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.572 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:02.573 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.573 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-11T06:51:02.573 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:51:02.573 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 109.30967069800001, 2024-09-11T06:51:02.573 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069253250000000004, 2024-09-11T06:51:02.573 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:02.573 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:02.573 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:02.573 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:02.573 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:51:02.574 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:51:02.574 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 7 2024-09-11T06:51:02.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.574 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:02.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.574 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:02.574 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:51:02.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.575 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:02.575 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:51:02.575 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.575 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:02.575 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246530+0000", 2024-09-11T06:51:02.575 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9469999999999999e-06 2024-09-11T06:51:02.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.575 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:02.575 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246545+0000", 2024-09-11T06:51:02.575 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5973000000000001e-05 2024-09-11T06:51:02.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.576 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:02.576 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246578+0000", 2024-09-11T06:51:02.576 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2759999999999998e-05 2024-09-11T06:51:02.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.576 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:02.576 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246595+0000", 2024-09-11T06:51:02.576 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6939000000000002e-05 2024-09-11T06:51:02.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.576 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:02.576 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246770+0000", 2024-09-11T06:51:02.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000174723 2024-09-11T06:51:02.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:02.577 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246789+0000", 2024-09-11T06:51:02.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9159000000000001e-05 2024-09-11T06:51:02.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:02.577 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250337+0000", 2024-09-11T06:51:02.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035480759999999998 2024-09-11T06:51:02.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250549+0000", 2024-09-11T06:51:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021192199999999999 2024-09-11T06:51:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250750+0000", 2024-09-11T06:51:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020104199999999999 2024-09-11T06:51:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.252455+0000", 2024-09-11T06:51:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001705249 2024-09-11T06:51:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.578 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:02.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253305+0000", 2024-09-11T06:51:02.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00084941200000000002 2024-09-11T06:51:02.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.579 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:02.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253388+0000", 2024-09-11T06:51:02.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3642999999999999e-05 2024-09-11T06:51:02.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.579 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:02.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253452+0000", 2024-09-11T06:51:02.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3479999999999995e-05 2024-09-11T06:51:02.579 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.579 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:02.579 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.580 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(client.4558.0:5 3.2 e57/55 3:5cea7035:::rbd_info:head v 57'1, mlcod=0'0)", 2024-09-11T06:51:02.580 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:51:02.580 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 44.461566335999997, 2024-09-11T06:51:02.580 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011016419999999999, 2024-09-11T06:51:02.580 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:02.580 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:02.580 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:02.580 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:02.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.580 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:02.580 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:51:02.580 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:51:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094631+0000", 2024-09-11T06:51:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4000000000000002e-07 2024-09-11T06:51:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094640+0000", 2024-09-11T06:51:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6230000000000003e-06 2024-09-11T06:51:02.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094669+0000", 2024-09-11T06:51:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8462e-05 2024-09-11T06:51:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094684+0000", 2024-09-11T06:51:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5536000000000002e-05 2024-09-11T06:51:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094798+0000", 2024-09-11T06:51:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011439599999999999 2024-09-11T06:51:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:02.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.094819+0000", 2024-09-11T06:51:02.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0296e-05 2024-09-11T06:51:02.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:02.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.095687+0000", 2024-09-11T06:51:02.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086851799999999998 2024-09-11T06:51:02.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:02.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:18.095733+0000", 2024-09-11T06:51:02.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5271000000000001e-05 2024-09-11T06:51:02.583 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.583 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:160 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 4096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:51:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:51:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 41.522451328999999, 2024-09-11T06:51:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001122215, 2024-09-11T06:51:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:51:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:51:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 160 2024-09-11T06:51:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.584 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:02.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.585 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:02.585 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:51:02.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.585 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:02.585 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:51:02.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.585 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:02.585 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033746+0000", 2024-09-11T06:51:02.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0999999999999999e-07 2024-09-11T06:51:02.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033749+0000", 2024-09-11T06:51:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8930000000000001e-06 2024-09-11T06:51:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033758+0000", 2024-09-11T06:51:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.242e-06 2024-09-11T06:51:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.033764+0000", 2024-09-11T06:51:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0510000000000002e-06 2024-09-11T06:51:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.587 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:02.587 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.034207+0000", 2024-09-11T06:51:02.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00044320600000000003 2024-09-11T06:51:02.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.587 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:02.587 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.034280+0000", 2024-09-11T06:51:02.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3094000000000002e-05 2024-09-11T06:51:02.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.587 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:02.587 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.034868+0000", 2024-09-11T06:51:02.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00058821899999999996 2024-09-11T06:51:02.587 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.588 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:02.588 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.588 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:162 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:51:02.588 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.039519+0000", 2024-09-11T06:51:02.588 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 41.516678210999999, 2024-09-11T06:51:02.588 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001298485, 2024-09-11T06:51:02.588 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:02.588 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:02.588 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:02.588 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:02.588 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:51:02.588 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:51:02.588 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 162 2024-09-11T06:51:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039519+0000", 2024-09-11T06:51:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039519+0000", 2024-09-11T06:51:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:02.589 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039520+0000", 2024-09-11T06:51:02.590 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9299999999999998e-07 2024-09-11T06:51:02.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:02.590 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039523+0000", 2024-09-11T06:51:02.590 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.247e-06 2024-09-11T06:51:02.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:02.590 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039529+0000", 2024-09-11T06:51:02.590 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8300000000000001e-06 2024-09-11T06:51:02.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.591 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:02.591 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.039533+0000", 2024-09-11T06:51:02.591 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.2379999999999999e-06 2024-09-11T06:51:02.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.591 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:02.591 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.040236+0000", 2024-09-11T06:51:02.591 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00070326500000000001 2024-09-11T06:51:02.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.591 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:02.591 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.040299+0000", 2024-09-11T06:51:02.591 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3077999999999995e-05 2024-09-11T06:51:02.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.592 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:02.592 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.040818+0000", 2024-09-11T06:51:02.592 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00051833399999999996 2024-09-11T06:51:02.592 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.592 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:02.592 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.592 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:164 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 528384~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:51:02.592 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.041536+0000", 2024-09-11T06:51:02.592 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 41.514661488999998, 2024-09-11T06:51:02.592 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00119962, 2024-09-11T06:51:02.592 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:02.593 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:02.593 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:02.593 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:02.593 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:51:02.593 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:51:02.593 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 164 2024-09-11T06:51:02.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.593 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:02.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.593 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:02.593 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041536+0000", 2024-09-11T06:51:02.593 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.593 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:02.594 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041536+0000", 2024-09-11T06:51:02.594 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.594 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:02.594 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041537+0000", 2024-09-11T06:51:02.594 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3e-06 2024-09-11T06:51:02.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.594 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:02.594 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041543+0000", 2024-09-11T06:51:02.594 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2360000000000001e-06 2024-09-11T06:51:02.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.595 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:02.595 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041558+0000", 2024-09-11T06:51:02.595 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4909000000000001e-05 2024-09-11T06:51:02.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.595 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:02.595 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.041569+0000", 2024-09-11T06:51:02.595 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0946e-05 2024-09-11T06:51:02.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.595 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:02.595 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.042138+0000", 2024-09-11T06:51:02.595 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000568996 2024-09-11T06:51:02.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.596 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:02.596 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.042190+0000", 2024-09-11T06:51:02.596 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1609e-05 2024-09-11T06:51:02.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.596 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:02.596 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.042735+0000", 2024-09-11T06:51:02.596 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00054562399999999998 2024-09-11T06:51:02.596 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.596 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:02.596 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.597 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:166 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 790528~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:51:02.597 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.043564+0000", 2024-09-11T06:51:02.597 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 41.512633092000002, 2024-09-11T06:51:02.597 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011042000000000001, 2024-09-11T06:51:02.597 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:02.597 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:02.597 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:02.597 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:02.597 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:51:02.597 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:51:02.597 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 166 2024-09-11T06:51:02.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.597 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:02.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.597 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:02.598 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043564+0000", 2024-09-11T06:51:02.598 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.598 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:02.598 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043564+0000", 2024-09-11T06:51:02.598 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.598 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:02.598 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043565+0000", 2024-09-11T06:51:02.598 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0449999999999999e-06 2024-09-11T06:51:02.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.599 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:02.599 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043571+0000", 2024-09-11T06:51:02.599 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.6849999999999998e-06 2024-09-11T06:51:02.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.599 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:02.599 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043586+0000", 2024-09-11T06:51:02.599 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4902000000000001e-05 2024-09-11T06:51:02.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.599 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:02.599 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.043593+0000", 2024-09-11T06:51:02.599 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9240000000000001e-06 2024-09-11T06:51:02.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.600 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:02.600 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.044073+0000", 2024-09-11T06:51:02.600 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000479813 2024-09-11T06:51:02.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.600 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:02.600 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.044123+0000", 2024-09-11T06:51:02.600 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0908000000000001e-05 2024-09-11T06:51:02.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.600 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:02.600 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.044668+0000", 2024-09-11T06:51:02.600 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000544923 2024-09-11T06:51:02.601 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.601 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:02.601 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.601 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:178 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 2363392~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:51:02.601 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.061599+0000", 2024-09-11T06:51:02.601 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 41.494598377000003, 2024-09-11T06:51:02.601 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012194529999999999, 2024-09-11T06:51:02.601 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:02.601 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:02.601 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:02.601 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:02.601 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:51:02.602 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:51:02.602 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 178 2024-09-11T06:51:02.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.602 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:02.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.602 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:02.602 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061599+0000", 2024-09-11T06:51:02.602 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.602 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:02.602 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061599+0000", 2024-09-11T06:51:02.602 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061600+0000", 2024-09-11T06:51:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3600000000000003e-07 2024-09-11T06:51:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061603+0000", 2024-09-11T06:51:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.6550000000000002e-06 2024-09-11T06:51:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061618+0000", 2024-09-11T06:51:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4377e-05 2024-09-11T06:51:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.604 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:02.604 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.061626+0000", 2024-09-11T06:51:02.604 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.0679999999999997e-06 2024-09-11T06:51:02.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.604 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:02.604 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.062098+0000", 2024-09-11T06:51:02.604 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00047251299999999998 2024-09-11T06:51:02.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.604 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:02.604 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.062253+0000", 2024-09-11T06:51:02.604 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015533100000000001 2024-09-11T06:51:02.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.605 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:02.605 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.062818+0000", 2024-09-11T06:51:02.605 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00056487300000000002 2024-09-11T06:51:02.605 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.605 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:02.605 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.605 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:184 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 3149824~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:51:02.605 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:51:02.605 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 41.489988674000003, 2024-09-11T06:51:02.605 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010685969999999999, 2024-09-11T06:51:02.605 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:02.606 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:02.606 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:02.606 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:02.606 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:51:02.606 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:51:02.606 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 184 2024-09-11T06:51:02.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.606 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:02.606 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.606 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:02.606 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:51:02.606 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.606 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:51:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066209+0000", 2024-09-11T06:51:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.6700000000000003e-07 2024-09-11T06:51:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066212+0000", 2024-09-11T06:51:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.1669999999999999e-06 2024-09-11T06:51:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.608 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:02.608 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066219+0000", 2024-09-11T06:51:02.608 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5699999999999998e-06 2024-09-11T06:51:02.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.608 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:02.608 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066223+0000", 2024-09-11T06:51:02.608 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3540000000000002e-06 2024-09-11T06:51:02.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.608 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:02.608 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066713+0000", 2024-09-11T06:51:02.608 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00048985399999999996 2024-09-11T06:51:02.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.066813+0000", 2024-09-11T06:51:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00010003 2024-09-11T06:51:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.067277+0000", 2024-09-11T06:51:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000463955 2024-09-11T06:51:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.609 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:186 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 3411968~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:51:02.610 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.071198+0000", 2024-09-11T06:51:02.610 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 41.484999330000001, 2024-09-11T06:51:02.610 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012515779999999999, 2024-09-11T06:51:02.610 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:02.610 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:02.610 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:02.610 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:02.610 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:51:02.610 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:51:02.610 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 186 2024-09-11T06:51:02.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.610 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:02.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.611 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:02.611 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071198+0000", 2024-09-11T06:51:02.611 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.611 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.611 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.611 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:02.611 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071198+0000", 2024-09-11T06:51:02.611 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.611 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.611 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.611 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:02.611 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071199+0000", 2024-09-11T06:51:02.611 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0299999999999999e-07 2024-09-11T06:51:02.611 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.612 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.612 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:02.612 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071201+0000", 2024-09-11T06:51:02.612 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8540000000000001e-06 2024-09-11T06:51:02.612 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.612 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.612 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:02.612 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071209+0000", 2024-09-11T06:51:02.612 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3629999999999996e-06 2024-09-11T06:51:02.612 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.612 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.612 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:02.612 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071214+0000", 2024-09-11T06:51:02.612 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2109999999999997e-06 2024-09-11T06:51:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.071915+0000", 2024-09-11T06:51:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00070101799999999998 2024-09-11T06:51:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.072011+0000", 2024-09-11T06:51:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.5755999999999996e-05 2024-09-11T06:51:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:02.613 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.072450+0000", 2024-09-11T06:51:02.614 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000438773 2024-09-11T06:51:02.614 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.614 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:02.614 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.614 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.614 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.614 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:190 2.3 2:c3278e6e:::rbd_data.1314f0a2bfa2.0000000000000003:head [sparse-read 3936256~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:51:02.614 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.074875+0000", 2024-09-11T06:51:02.614 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 41.481322415000001, 2024-09-11T06:51:02.614 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0013119620000000001, 2024-09-11T06:51:02.614 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:02.614 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:02.614 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:02.614 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:02.615 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:51:02.615 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:51:02.615 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 190 2024-09-11T06:51:02.615 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.615 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:02.615 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.615 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:02.615 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074875+0000", 2024-09-11T06:51:02.615 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.615 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.615 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.615 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:02.615 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074875+0000", 2024-09-11T06:51:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074876+0000", 2024-09-11T06:51:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7400000000000003e-07 2024-09-11T06:51:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074887+0000", 2024-09-11T06:51:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1817999999999999e-05 2024-09-11T06:51:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:02.616 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074896+0000", 2024-09-11T06:51:02.617 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9120000000000004e-06 2024-09-11T06:51:02.617 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.617 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.617 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:02.617 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.074900+0000", 2024-09-11T06:51:02.617 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.2220000000000004e-06 2024-09-11T06:51:02.617 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.617 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.617 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:02.617 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.075198+0000", 2024-09-11T06:51:02.617 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00029750399999999998 2024-09-11T06:51:02.617 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.075430+0000", 2024-09-11T06:51:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023200400000000001 2024-09-11T06:51:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.076187+0000", 2024-09-11T06:51:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00075682799999999997 2024-09-11T06:51:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.618 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.619 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:359 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 135168~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:51:02.619 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.298832+0000", 2024-09-11T06:51:02.619 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 41.257364781, 2024-09-11T06:51:02.619 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001078575, 2024-09-11T06:51:02.619 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:02.619 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:02.619 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:02.619 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:02.619 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:51:02.619 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:51:02.619 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 359 2024-09-11T06:51:02.619 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.620 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:02.620 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.620 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:02.620 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298832+0000", 2024-09-11T06:51:02.620 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.620 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.620 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.620 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:02.620 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298832+0000", 2024-09-11T06:51:02.620 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.620 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.620 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.620 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:02.620 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298834+0000", 2024-09-11T06:51:02.621 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.646e-06 2024-09-11T06:51:02.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.621 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.621 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:02.621 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298845+0000", 2024-09-11T06:51:02.621 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.081e-05 2024-09-11T06:51:02.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.621 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.621 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:02.621 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298872+0000", 2024-09-11T06:51:02.621 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7382e-05 2024-09-11T06:51:02.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298884+0000", 2024-09-11T06:51:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1956e-05 2024-09-11T06:51:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298959+0000", 2024-09-11T06:51:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5214000000000002e-05 2024-09-11T06:51:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.299090+0000", 2024-09-11T06:51:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000130247 2024-09-11T06:51:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.299911+0000", 2024-09-11T06:51:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00082131999999999995 2024-09-11T06:51:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:360 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:51:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.298914+0000", 2024-09-11T06:51:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 41.257283111, 2024-09-11T06:51:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0023571690000000001, 2024-09-11T06:51:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:51:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:51:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 360 2024-09-11T06:51:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298914+0000", 2024-09-11T06:51:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298914+0000", 2024-09-11T06:51:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298915+0000", 2024-09-11T06:51:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9300000000000006e-07 2024-09-11T06:51:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298921+0000", 2024-09-11T06:51:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.6030000000000004e-06 2024-09-11T06:51:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298932+0000", 2024-09-11T06:51:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.082e-05 2024-09-11T06:51:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298939+0000", 2024-09-11T06:51:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7919999999999993e-06 2024-09-11T06:51:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.300011+0000", 2024-09-11T06:51:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001071179 2024-09-11T06:51:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.300245+0000", 2024-09-11T06:51:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000234434 2024-09-11T06:51:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.301271+0000", 2024-09-11T06:51:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010263480000000001 2024-09-11T06:51:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:362 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 528384~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:51:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.302376+0000", 2024-09-11T06:51:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 41.253820787999999, 2024-09-11T06:51:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014132909999999999, 2024-09-11T06:51:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:51:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:51:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 362 2024-09-11T06:51:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302376+0000", 2024-09-11T06:51:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302376+0000", 2024-09-11T06:51:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302378+0000", 2024-09-11T06:51:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1629999999999999e-06 2024-09-11T06:51:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302383+0000", 2024-09-11T06:51:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.463e-06 2024-09-11T06:51:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302401+0000", 2024-09-11T06:51:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8219e-05 2024-09-11T06:51:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302411+0000", 2024-09-11T06:51:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9000000000000001e-06 2024-09-11T06:51:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302524+0000", 2024-09-11T06:51:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000113199 2024-09-11T06:51:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302745+0000", 2024-09-11T06:51:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022057399999999999 2024-09-11T06:51:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.303790+0000", 2024-09-11T06:51:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010447729999999999 2024-09-11T06:51:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:363 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 659456~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:51:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.302938+0000", 2024-09-11T06:51:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 41.253259194999998, 2024-09-11T06:51:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00185012, 2024-09-11T06:51:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:51:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:51:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 363 2024-09-11T06:51:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302938+0000", 2024-09-11T06:51:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302938+0000", 2024-09-11T06:51:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302939+0000", 2024-09-11T06:51:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2300000000000001e-06 2024-09-11T06:51:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302944+0000", 2024-09-11T06:51:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.493e-06 2024-09-11T06:51:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302960+0000", 2024-09-11T06:51:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.632e-05 2024-09-11T06:51:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302972+0000", 2024-09-11T06:51:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1885000000000001e-05 2024-09-11T06:51:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.303835+0000", 2024-09-11T06:51:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000863141 2024-09-11T06:51:02.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.636 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.636 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:02.636 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.303965+0000", 2024-09-11T06:51:02.636 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013032199999999999 2024-09-11T06:51:02.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.636 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.636 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:02.636 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.304788+0000", 2024-09-11T06:51:02.636 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00082272900000000002 2024-09-11T06:51:02.636 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.636 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:02.636 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:367 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 1183744~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:51:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.312392+0000", 2024-09-11T06:51:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 41.243805555000002, 2024-09-11T06:51:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011343460000000001, 2024-09-11T06:51:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:51:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:51:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 367 2024-09-11T06:51:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:02.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.638 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:02.638 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312392+0000", 2024-09-11T06:51:02.638 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.638 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:02.638 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312392+0000", 2024-09-11T06:51:02.638 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.638 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:02.638 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312393+0000", 2024-09-11T06:51:02.638 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6199999999999999e-06 2024-09-11T06:51:02.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.639 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:02.639 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312400+0000", 2024-09-11T06:51:02.639 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0700000000000001e-06 2024-09-11T06:51:02.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.639 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:02.639 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312420+0000", 2024-09-11T06:51:02.639 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9375999999999999e-05 2024-09-11T06:51:02.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.639 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:02.639 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312430+0000", 2024-09-11T06:51:02.640 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0209e-05 2024-09-11T06:51:02.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.640 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.640 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:02.640 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312549+0000", 2024-09-11T06:51:02.640 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000118779 2024-09-11T06:51:02.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.640 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.640 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:02.640 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312767+0000", 2024-09-11T06:51:02.640 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021786400000000001 2024-09-11T06:51:02.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.640 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.640 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:02.641 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313526+0000", 2024-09-11T06:51:02.641 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00075942800000000003 2024-09-11T06:51:02.641 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.641 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:02.641 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.641 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:368 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 1314816~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:51:02.641 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.312454+0000", 2024-09-11T06:51:02.641 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 41.243743031000001, 2024-09-11T06:51:02.641 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014704550000000001, 2024-09-11T06:51:02.641 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:02.641 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:02.641 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:02.642 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:02.642 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:51:02.642 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:51:02.642 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 368 2024-09-11T06:51:02.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.642 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:02.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.642 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:02.642 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312454+0000", 2024-09-11T06:51:02.642 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.642 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:02.642 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312454+0000", 2024-09-11T06:51:02.643 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.643 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:02.643 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312455+0000", 2024-09-11T06:51:02.643 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4399999999999999e-07 2024-09-11T06:51:02.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.643 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:02.643 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312474+0000", 2024-09-11T06:51:02.643 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8899e-05 2024-09-11T06:51:02.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.643 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:02.644 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312485+0000", 2024-09-11T06:51:02.644 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0793e-05 2024-09-11T06:51:02.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.644 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:02.644 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.312492+0000", 2024-09-11T06:51:02.644 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.593e-06 2024-09-11T06:51:02.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.644 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:02.644 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313576+0000", 2024-09-11T06:51:02.644 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010837970000000001 2024-09-11T06:51:02.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:02.645 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313620+0000", 2024-09-11T06:51:02.645 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3640000000000002e-05 2024-09-11T06:51:02.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.645 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:02.645 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.313925+0000", 2024-09-11T06:51:02.645 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00030498899999999998 2024-09-11T06:51:02.645 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.645 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:02.645 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.645 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:370 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 1576960~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:51:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.314817+0000", 2024-09-11T06:51:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 41.241380321999998, 2024-09-11T06:51:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011606190000000001, 2024-09-11T06:51:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:51:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:51:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 370 2024-09-11T06:51:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:02.647 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314817+0000", 2024-09-11T06:51:02.647 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.647 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:02.647 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314817+0000", 2024-09-11T06:51:02.647 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.647 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:02.647 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314818+0000", 2024-09-11T06:51:02.647 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.463e-06 2024-09-11T06:51:02.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.648 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:02.648 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314826+0000", 2024-09-11T06:51:02.648 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3860000000000001e-06 2024-09-11T06:51:02.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.648 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:02.648 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314842+0000", 2024-09-11T06:51:02.648 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6220000000000001e-05 2024-09-11T06:51:02.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.648 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:02.648 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.314854+0000", 2024-09-11T06:51:02.648 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1562e-05 2024-09-11T06:51:02.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.649 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:02.649 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.315212+0000", 2024-09-11T06:51:02.649 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000358012 2024-09-11T06:51:02.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.649 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:02.649 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.315433+0000", 2024-09-11T06:51:02.649 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022103800000000001 2024-09-11T06:51:02.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.649 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:02.649 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.315978+0000", 2024-09-11T06:51:02.649 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00054493800000000002 2024-09-11T06:51:02.649 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.649 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:02.650 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.650 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:57 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:02.650 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.568529+0000", 2024-09-11T06:51:02.650 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 13.987667778, 2024-09-11T06:51:02.650 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004228229, 2024-09-11T06:51:02.650 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:02.650 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:02.650 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:02.650 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:02.650 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:02.650 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:02.650 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 57 2024-09-11T06:51:02.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.650 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:02.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.651 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:02.651 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568529+0000", 2024-09-11T06:51:02.651 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.651 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:02.651 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568529+0000", 2024-09-11T06:51:02.651 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:02.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.651 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:02.651 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568532+0000", 2024-09-11T06:51:02.651 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0000000000000001e-06 2024-09-11T06:51:02.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.652 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:02.652 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568547+0000", 2024-09-11T06:51:02.652 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4219999999999999e-05 2024-09-11T06:51:02.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.652 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:02.652 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568580+0000", 2024-09-11T06:51:02.652 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.3328999999999999e-05 2024-09-11T06:51:02.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.652 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:02.652 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568596+0000", 2024-09-11T06:51:02.653 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5990000000000001e-05 2024-09-11T06:51:02.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.653 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:02.653 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568728+0000", 2024-09-11T06:51:02.653 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013175400000000001 2024-09-11T06:51:02.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.653 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:02.653 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568738+0000", 2024-09-11T06:51:02.653 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0196e-05 2024-09-11T06:51:02.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.653 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:02.653 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.572464+0000", 2024-09-11T06:51:02.654 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0037264160000000002 2024-09-11T06:51:02.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.654 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:02.654 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.572584+0000", 2024-09-11T06:51:02.654 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000119438 2024-09-11T06:51:02.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:02.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:02.654 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:02.654 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.572758+0000", 2024-09-11T06:51:02.654 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017388599999999999 2024-09-11T06:51:02.654 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.654 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:02.654 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.654 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:02.654 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:02.655 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:02.656 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:51:02.657 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:02.650+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:02.757 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:02.754+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:02.786 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:02.786 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:02.786 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:51:02.786 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:02.800 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:51:02.852 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is unset 2024-09-11T06:51:02.858 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:02.854+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:02.871 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 1, 3, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T06:51:02.871 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T06:51:02.872 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-11T06:51:02.872 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-11T06:51:02.928 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:02.928 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:02.928 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:51:02.928 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:51:02.928 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:02.943 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:51:02.989 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:02.986+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:03.059 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:03.054+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:03.109 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:03.109 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:51:03.109 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:51:03.109 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:51:03.109 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.109 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:54 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'32, mlcod=13'29)", 2024-09-11T06:51:03.110 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:51:03.110 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 116.895336951, 2024-09-11T06:51:03.110 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034271670000000001, 2024-09-11T06:51:03.110 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.110 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.110 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.110 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.110 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.110 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.110 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:51:03.110 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.110 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.110 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.110 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.111 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:51:03.111 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.111 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.111 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.111 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.111 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210361+0000", 2024-09-11T06:51:03.111 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1260000000000001e-06 2024-09-11T06:51:03.111 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.111 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.111 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.111 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210370+0000", 2024-09-11T06:51:03.111 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8200000000000003e-06 2024-09-11T06:51:03.111 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.111 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.112 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.112 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210390+0000", 2024-09-11T06:51:03.112 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0126e-05 2024-09-11T06:51:03.112 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.112 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.112 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.112 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210404+0000", 2024-09-11T06:51:03.112 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4589999999999999e-05 2024-09-11T06:51:03.112 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.112 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.112 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.112 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210538+0000", 2024-09-11T06:51:03.112 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000133719 2024-09-11T06:51:03.112 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.113 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.113 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.113 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210593+0000", 2024-09-11T06:51:03.113 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5016999999999999e-05 2024-09-11T06:51:03.113 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.113 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.113 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.113 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213695+0000", 2024-09-11T06:51:03.113 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003101866 2024-09-11T06:51:03.113 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.113 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.113 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.113 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213787+0000", 2024-09-11T06:51:03.113 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1903000000000004e-05 2024-09-11T06:51:03.114 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.114 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.114 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.114 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.114 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.114 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:55 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'33, mlcod=13'29)", 2024-09-11T06:51:03.114 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:51:03.114 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 116.89431771300001, 2024-09-11T06:51:03.114 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.005086091, 2024-09-11T06:51:03.114 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.114 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.114 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.114 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.114 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.115 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.115 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:51:03.115 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.115 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.115 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.115 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.115 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:51:03.115 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.115 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.115 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.115 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.115 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211380+0000", 2024-09-11T06:51:03.115 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.097e-06 2024-09-11T06:51:03.116 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.116 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.116 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.116 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211537+0000", 2024-09-11T06:51:03.116 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000157369 2024-09-11T06:51:03.116 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.116 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.116 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.116 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211559+0000", 2024-09-11T06:51:03.116 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000001e-05 2024-09-11T06:51:03.116 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.116 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.116 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.116 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211573+0000", 2024-09-11T06:51:03.117 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4946000000000001e-05 2024-09-11T06:51:03.117 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.117 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.117 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.117 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211676+0000", 2024-09-11T06:51:03.117 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0001026 2024-09-11T06:51:03.117 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.117 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.117 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.117 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211709+0000", 2024-09-11T06:51:03.117 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3071999999999998e-05 2024-09-11T06:51:03.117 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.117 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.117 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.118 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216379+0000", 2024-09-11T06:51:03.118 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0046693780000000001 2024-09-11T06:51:03.118 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.118 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.118 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.118 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216465+0000", 2024-09-11T06:51:03.118 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6469000000000002e-05 2024-09-11T06:51:03.118 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.118 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.118 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.118 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.118 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.118 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:56 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'34, mlcod=13'29)", 2024-09-11T06:51:03.118 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:51:03.119 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 116.892883854, 2024-09-11T06:51:03.119 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0060310629999999997, 2024-09-11T06:51:03.119 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.119 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.119 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.119 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.119 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.119 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.119 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:51:03.119 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.119 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.119 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.119 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.119 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:51:03.120 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.120 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.120 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.120 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.120 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212814+0000", 2024-09-11T06:51:03.120 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0869999999999999e-06 2024-09-11T06:51:03.120 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.120 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.120 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.120 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212822+0000", 2024-09-11T06:51:03.120 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0870000000000007e-06 2024-09-11T06:51:03.120 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.120 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.120 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.121 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212840+0000", 2024-09-11T06:51:03.121 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:51:03.121 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.121 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.121 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.121 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212854+0000", 2024-09-11T06:51:03.121 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4003e-05 2024-09-11T06:51:03.121 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.121 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.121 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.121 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213026+0000", 2024-09-11T06:51:03.121 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017207499999999999 2024-09-11T06:51:03.121 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.122 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.122 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.122 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213063+0000", 2024-09-11T06:51:03.122 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7187000000000002e-05 2024-09-11T06:51:03.122 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.122 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.122 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.122 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218762+0000", 2024-09-11T06:51:03.122 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0056993290000000004 2024-09-11T06:51:03.122 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.122 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.122 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.122 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218844+0000", 2024-09-11T06:51:03.122 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1701999999999998e-05 2024-09-11T06:51:03.123 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.123 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.123 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.123 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.123 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.123 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:60 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'38, mlcod=13'36)", 2024-09-11T06:51:03.123 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:51:03.123 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 116.878942505, 2024-09-11T06:51:03.123 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039254270000000004, 2024-09-11T06:51:03.123 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.123 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.123 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.123 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.123 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.124 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.124 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:51:03.124 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.124 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.124 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.124 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.124 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:51:03.124 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.124 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.124 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.124 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.124 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226756+0000", 2024-09-11T06:51:03.124 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-11T06:51:03.124 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.125 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.125 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.125 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226768+0000", 2024-09-11T06:51:03.125 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1919999999999999e-05 2024-09-11T06:51:03.125 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.125 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.125 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.125 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226797+0000", 2024-09-11T06:51:03.125 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9573e-05 2024-09-11T06:51:03.125 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.125 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.125 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.126 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226818+0000", 2024-09-11T06:51:03.126 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0432999999999999e-05 2024-09-11T06:51:03.126 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.126 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.126 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.126 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226909+0000", 2024-09-11T06:51:03.126 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0920000000000004e-05 2024-09-11T06:51:03.126 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.126 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.126 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.126 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226936+0000", 2024-09-11T06:51:03.126 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7472999999999999e-05 2024-09-11T06:51:03.126 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.126 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.127 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.127 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230596+0000", 2024-09-11T06:51:03.127 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0036602520000000001 2024-09-11T06:51:03.127 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.127 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.127 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.127 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230680+0000", 2024-09-11T06:51:03.127 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3106e-05 2024-09-11T06:51:03.127 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.127 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.127 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.127 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.127 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.128 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:107 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'69, mlcod=13'64)", 2024-09-11T06:51:03.128 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:51:03.128 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 116.662495048, 2024-09-11T06:51:03.128 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040728830000000002, 2024-09-11T06:51:03.128 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.128 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.128 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.128 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.128 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.128 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.128 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:51:03.128 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.128 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.129 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.129 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.129 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:51:03.129 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.129 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.129 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.129 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.129 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443203+0000", 2024-09-11T06:51:03.129 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1480000000000001e-06 2024-09-11T06:51:03.129 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.129 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.129 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.129 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443222+0000", 2024-09-11T06:51:03.130 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9040000000000001e-05 2024-09-11T06:51:03.130 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.130 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.130 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.130 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443246+0000", 2024-09-11T06:51:03.130 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4606000000000001e-05 2024-09-11T06:51:03.130 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.130 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.130 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.130 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443277+0000", 2024-09-11T06:51:03.130 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0207999999999999e-05 2024-09-11T06:51:03.130 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.130 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.130 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.131 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443340+0000", 2024-09-11T06:51:03.131 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3602000000000005e-05 2024-09-11T06:51:03.131 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.131 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.131 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.131 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443363+0000", 2024-09-11T06:51:03.131 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2331e-05 2024-09-11T06:51:03.131 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.131 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.131 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.131 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447216+0000", 2024-09-11T06:51:03.131 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0038539690000000001 2024-09-11T06:51:03.131 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.131 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.132 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.132 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447274+0000", 2024-09-11T06:51:03.132 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7979000000000002e-05 2024-09-11T06:51:03.132 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.132 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.132 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.132 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.132 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.132 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:111 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'73, mlcod=13'67)", 2024-09-11T06:51:03.132 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:51:03.132 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 116.65598341800001, 2024-09-11T06:51:03.132 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034372360000000002, 2024-09-11T06:51:03.132 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.132 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.133 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.133 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.133 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.133 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.133 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:51:03.133 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.133 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.133 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.133 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.133 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:51:03.133 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.133 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.133 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.133 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.133 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449714+0000", 2024-09-11T06:51:03.134 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5000000000000001e-07 2024-09-11T06:51:03.134 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.134 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.134 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.134 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449721+0000", 2024-09-11T06:51:03.134 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8530000000000003e-06 2024-09-11T06:51:03.134 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.134 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.134 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.134 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449736+0000", 2024-09-11T06:51:03.134 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5114e-05 2024-09-11T06:51:03.134 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.134 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.134 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.135 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449743+0000", 2024-09-11T06:51:03.135 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0210000000000002e-06 2024-09-11T06:51:03.135 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.135 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.135 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.135 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449804+0000", 2024-09-11T06:51:03.135 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0479999999999997e-05 2024-09-11T06:51:03.135 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.135 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.135 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.135 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449826+0000", 2024-09-11T06:51:03.135 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1922000000000001e-05 2024-09-11T06:51:03.135 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.135 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.136 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.136 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453104+0000", 2024-09-11T06:51:03.136 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003278652 2024-09-11T06:51:03.136 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.136 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.136 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.136 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453150+0000", 2024-09-11T06:51:03.136 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6244e-05 2024-09-11T06:51:03.136 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.136 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.136 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.136 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.136 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.136 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:114 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'76, mlcod=13'71)", 2024-09-11T06:51:03.136 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:51:03.137 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 116.651305347, 2024-09-11T06:51:03.137 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0042576660000000002, 2024-09-11T06:51:03.137 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.137 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.137 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.137 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.137 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.137 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.137 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:51:03.137 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.137 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.137 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.137 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.138 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:51:03.138 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.138 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.138 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.138 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.138 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454392+0000", 2024-09-11T06:51:03.138 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5999999999999991e-07 2024-09-11T06:51:03.138 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.138 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.138 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.138 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454399+0000", 2024-09-11T06:51:03.138 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.878e-06 2024-09-11T06:51:03.138 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.138 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.139 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.139 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454415+0000", 2024-09-11T06:51:03.139 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5659999999999999e-05 2024-09-11T06:51:03.139 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.139 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.139 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.139 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454422+0000", 2024-09-11T06:51:03.139 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:51:03.139 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.139 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.139 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.139 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454474+0000", 2024-09-11T06:51:03.139 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1860999999999997e-05 2024-09-11T06:51:03.139 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.139 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.140 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.140 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454497+0000", 2024-09-11T06:51:03.140 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3686e-05 2024-09-11T06:51:03.140 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.140 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.140 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.140 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458552+0000", 2024-09-11T06:51:03.140 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040544550000000002 2024-09-11T06:51:03.140 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.140 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.140 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.140 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458649+0000", 2024-09-11T06:51:03.140 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7005999999999999e-05 2024-09-11T06:51:03.141 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.141 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.141 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.141 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.141 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.141 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:115 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'77, mlcod=13'72)", 2024-09-11T06:51:03.141 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:51:03.141 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 116.649964735, 2024-09-11T06:51:03.141 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003355584, 2024-09-11T06:51:03.141 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.141 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.141 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.142 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.142 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.142 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.142 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:51:03.142 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.142 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.142 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.142 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.142 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:51:03.142 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.142 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.142 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.142 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.142 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:51:03.143 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5999999999999999e-07 2024-09-11T06:51:03.143 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.143 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.143 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.143 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455803+0000", 2024-09-11T06:51:03.143 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0310999999999998e-05 2024-09-11T06:51:03.143 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.143 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.143 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.143 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455815+0000", 2024-09-11T06:51:03.143 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2787e-05 2024-09-11T06:51:03.143 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.143 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.143 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.144 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455823+0000", 2024-09-11T06:51:03.144 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5959999999999998e-06 2024-09-11T06:51:03.144 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.144 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.144 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.144 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455875+0000", 2024-09-11T06:51:03.144 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2054e-05 2024-09-11T06:51:03.144 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.144 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.144 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.144 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455896+0000", 2024-09-11T06:51:03.144 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0554999999999999e-05 2024-09-11T06:51:03.144 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.144 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.145 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.145 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459000+0000", 2024-09-11T06:51:03.145 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003104842 2024-09-11T06:51:03.145 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.145 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.145 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.145 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459087+0000", 2024-09-11T06:51:03.145 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6978999999999995e-05 2024-09-11T06:51:03.145 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.145 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.145 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.145 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.145 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.145 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:120 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'82, mlcod=13'78)", 2024-09-11T06:51:03.146 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:51:03.146 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 116.63928745, 2024-09-11T06:51:03.146 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0064612549999999999, 2024-09-11T06:51:03.146 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.146 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.146 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.146 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.146 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.146 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.146 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:51:03.146 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.146 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.146 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.146 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.147 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:51:03.147 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.147 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.147 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.147 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.147 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466410+0000", 2024-09-11T06:51:03.147 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0389999999999999e-06 2024-09-11T06:51:03.147 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.147 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.147 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.147 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466496+0000", 2024-09-11T06:51:03.147 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5519000000000006e-05 2024-09-11T06:51:03.147 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.148 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.148 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.148 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466510+0000", 2024-09-11T06:51:03.148 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3842999999999999e-05 2024-09-11T06:51:03.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.148 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.148 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.148 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466517+0000", 2024-09-11T06:51:03.148 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5229999999999998e-06 2024-09-11T06:51:03.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.148 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.148 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.148 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466595+0000", 2024-09-11T06:51:03.148 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8019e-05 2024-09-11T06:51:03.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.148 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466612+0000", 2024-09-11T06:51:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7013000000000001e-05 2024-09-11T06:51:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472787+0000", 2024-09-11T06:51:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0061750269999999996 2024-09-11T06:51:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472870+0000", 2024-09-11T06:51:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3271999999999998e-05 2024-09-11T06:51:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.150 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.150 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.150 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.150 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:121 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'83, mlcod=13'78)", 2024-09-11T06:51:03.150 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:51:03.150 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 116.638062654, 2024-09-11T06:51:03.150 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055613069999999997, 2024-09-11T06:51:03.150 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.150 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.150 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.150 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.150 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:51:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:51:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467635+0000", 2024-09-11T06:51:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6899999999999996e-07 2024-09-11T06:51:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.152 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.152 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467727+0000", 2024-09-11T06:51:03.152 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1800999999999997e-05 2024-09-11T06:51:03.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.152 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.152 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.152 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467740+0000", 2024-09-11T06:51:03.152 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3747999999999999e-05 2024-09-11T06:51:03.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.152 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.152 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.152 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467753+0000", 2024-09-11T06:51:03.152 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2807e-05 2024-09-11T06:51:03.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.153 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.153 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.153 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467812+0000", 2024-09-11T06:51:03.153 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8381999999999998e-05 2024-09-11T06:51:03.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.153 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.153 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.153 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467828+0000", 2024-09-11T06:51:03.153 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6728e-05 2024-09-11T06:51:03.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.153 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.153 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473112+0000", 2024-09-11T06:51:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052833619999999998 2024-09-11T06:51:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473195+0000", 2024-09-11T06:51:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3609999999999994e-05 2024-09-11T06:51:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:131 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'89, mlcod=13'85)", 2024-09-11T06:51:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:51:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 116.286656939, 2024-09-11T06:51:03.155 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052492260000000001, 2024-09-11T06:51:03.155 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.155 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.155 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.155 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.155 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.155 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.155 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:51:03.155 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.155 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.155 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.155 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:51:03.155 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.156 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.156 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.156 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.156 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819041+0000", 2024-09-11T06:51:03.156 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4360000000000001e-06 2024-09-11T06:51:03.156 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.156 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.156 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.156 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819100+0000", 2024-09-11T06:51:03.156 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9225999999999998e-05 2024-09-11T06:51:03.156 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.156 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.156 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.156 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819128+0000", 2024-09-11T06:51:03.157 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.779e-05 2024-09-11T06:51:03.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.157 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.157 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.157 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819142+0000", 2024-09-11T06:51:03.157 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3573e-05 2024-09-11T06:51:03.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.157 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.157 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.157 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819263+0000", 2024-09-11T06:51:03.157 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000121086 2024-09-11T06:51:03.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.157 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.157 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.158 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819292+0000", 2024-09-11T06:51:03.158 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9363e-05 2024-09-11T06:51:03.158 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.158 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.158 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.158 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824185+0000", 2024-09-11T06:51:03.158 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0048924620000000002 2024-09-11T06:51:03.158 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.158 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.158 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.158 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824289+0000", 2024-09-11T06:51:03.158 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010429000000000001 2024-09-11T06:51:03.158 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.159 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.159 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.159 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:132 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'90, mlcod=13'85)", 2024-09-11T06:51:03.159 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:51:03.159 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 116.285882249, 2024-09-11T06:51:03.159 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.004532342, 2024-09-11T06:51:03.159 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.159 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.159 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.160 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.160 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.160 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.160 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:51:03.160 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.160 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.160 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.160 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:51:03.160 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.160 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.160 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819816+0000", 2024-09-11T06:51:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.61e-06 2024-09-11T06:51:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819827+0000", 2024-09-11T06:51:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1e-05 2024-09-11T06:51:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819851+0000", 2024-09-11T06:51:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4267e-05 2024-09-11T06:51:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819864+0000", 2024-09-11T06:51:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2767e-05 2024-09-11T06:51:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820071+0000", 2024-09-11T06:51:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000207061 2024-09-11T06:51:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820103+0000", 2024-09-11T06:51:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2326999999999997e-05 2024-09-11T06:51:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.163 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.163 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824302+0000", 2024-09-11T06:51:03.163 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0041986000000000002 2024-09-11T06:51:03.163 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.163 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.163 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.163 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824347+0000", 2024-09-11T06:51:03.163 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4709999999999997e-05 2024-09-11T06:51:03.163 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.163 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.163 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.163 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.163 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.163 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:133 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'91, mlcod=13'86)", 2024-09-11T06:51:03.163 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:51:03.164 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 116.28525059, 2024-09-11T06:51:03.164 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039421459999999997, 2024-09-11T06:51:03.164 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.164 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.164 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.164 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.164 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.164 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.164 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:51:03.164 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.164 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.164 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.164 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.165 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:51:03.165 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.165 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.165 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.165 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820448+0000", 2024-09-11T06:51:03.165 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.84e-06 2024-09-11T06:51:03.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.165 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.165 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.165 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820463+0000", 2024-09-11T06:51:03.165 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5356e-05 2024-09-11T06:51:03.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820499+0000", 2024-09-11T06:51:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.625e-05 2024-09-11T06:51:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820546+0000", 2024-09-11T06:51:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7017000000000001e-05 2024-09-11T06:51:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820882+0000", 2024-09-11T06:51:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033512 2024-09-11T06:51:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820941+0000", 2024-09-11T06:51:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8922999999999998e-05 2024-09-11T06:51:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824351+0000", 2024-09-11T06:51:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034101940000000001 2024-09-11T06:51:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.168 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824388+0000", 2024-09-11T06:51:03.168 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7446e-05 2024-09-11T06:51:03.168 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.168 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.168 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.168 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.168 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:139 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'97, mlcod=13'86)", 2024-09-11T06:51:03.168 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:51:03.168 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 116.279840612, 2024-09-11T06:51:03.168 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003732221, 2024-09-11T06:51:03.168 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.168 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:51:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:51:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825858+0000", 2024-09-11T06:51:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5829999999999999e-06 2024-09-11T06:51:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825874+0000", 2024-09-11T06:51:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5967000000000001e-05 2024-09-11T06:51:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825909+0000", 2024-09-11T06:51:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5692999999999999e-05 2024-09-11T06:51:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825923+0000", 2024-09-11T06:51:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4080000000000001e-05 2024-09-11T06:51:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826010+0000", 2024-09-11T06:51:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6248999999999994e-05 2024-09-11T06:51:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826038+0000", 2024-09-11T06:51:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8010000000000001e-05 2024-09-11T06:51:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829499+0000", 2024-09-11T06:51:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003461783 2024-09-11T06:51:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829588+0000", 2024-09-11T06:51:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8856000000000004e-05 2024-09-11T06:51:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:140 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'98, mlcod=13'93)", 2024-09-11T06:51:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:51:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 116.277989883, 2024-09-11T06:51:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0057995929999999996, 2024-09-11T06:51:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:51:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.174 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.174 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:51:03.174 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.174 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.174 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.174 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827708+0000", 2024-09-11T06:51:03.174 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.547e-06 2024-09-11T06:51:03.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.174 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.174 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.174 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827765+0000", 2024-09-11T06:51:03.174 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6266e-05 2024-09-11T06:51:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827791+0000", 2024-09-11T06:51:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6098999999999999e-05 2024-09-11T06:51:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827806+0000", 2024-09-11T06:51:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5537e-05 2024-09-11T06:51:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827928+0000", 2024-09-11T06:51:03.176 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000122012 2024-09-11T06:51:03.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.176 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.176 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.176 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827961+0000", 2024-09-11T06:51:03.176 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2663999999999998e-05 2024-09-11T06:51:03.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.176 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.176 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.176 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833409+0000", 2024-09-11T06:51:03.176 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054479350000000001 2024-09-11T06:51:03.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.176 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.176 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833506+0000", 2024-09-11T06:51:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7533000000000003e-05 2024-09-11T06:51:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:141 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'99, mlcod=13'93)", 2024-09-11T06:51:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:51:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 116.2770965, 2024-09-11T06:51:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055394709999999998, 2024-09-11T06:51:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:51:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:51:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828602+0000", 2024-09-11T06:51:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.46e-06 2024-09-11T06:51:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828618+0000", 2024-09-11T06:51:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6592999999999999e-05 2024-09-11T06:51:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828646+0000", 2024-09-11T06:51:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7552999999999999e-05 2024-09-11T06:51:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828663+0000", 2024-09-11T06:51:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.719e-05 2024-09-11T06:51:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829023+0000", 2024-09-11T06:51:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036054799999999998 2024-09-11T06:51:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829062+0000", 2024-09-11T06:51:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8865999999999997e-05 2024-09-11T06:51:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.181 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834078+0000", 2024-09-11T06:51:03.181 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050157480000000004 2024-09-11T06:51:03.181 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.181 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.181 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.181 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834140+0000", 2024-09-11T06:51:03.181 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1513000000000005e-05 2024-09-11T06:51:03.181 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.181 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.181 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.181 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.181 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.181 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:145 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'103, mlcod=13'98)", 2024-09-11T06:51:03.182 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:51:03.182 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 116.267144003, 2024-09-11T06:51:03.182 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054455409999999999, 2024-09-11T06:51:03.182 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.182 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.182 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.182 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.182 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.182 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:51:03.182 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:51:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838554+0000", 2024-09-11T06:51:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3430000000000001e-06 2024-09-11T06:51:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838640+0000", 2024-09-11T06:51:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5569999999999996e-05 2024-09-11T06:51:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838660+0000", 2024-09-11T06:51:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0576000000000001e-05 2024-09-11T06:51:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838674+0000", 2024-09-11T06:51:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3529999999999999e-05 2024-09-11T06:51:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838838+0000", 2024-09-11T06:51:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016468799999999999 2024-09-11T06:51:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838885+0000", 2024-09-11T06:51:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6884e-05 2024-09-11T06:51:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843901+0000", 2024-09-11T06:51:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050162310000000003 2024-09-11T06:51:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843998+0000", 2024-09-11T06:51:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.6718999999999993e-05 2024-09-11T06:51:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4558.0:1 3.3 e56/55 3:c4c92e5a:::rbd_trash:head v 56'1, mlcod=0'0)", 2024-09-11T06:51:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:51:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 46.014340941, 2024-09-11T06:51:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0053791810000000002, 2024-09-11T06:51:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:51:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:51:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091357+0000", 2024-09-11T06:51:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.54e-07 2024-09-11T06:51:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091376+0000", 2024-09-11T06:51:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9769e-05 2024-09-11T06:51:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091396+0000", 2024-09-11T06:51:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0152999999999998e-05 2024-09-11T06:51:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091416+0000", 2024-09-11T06:51:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0006000000000001e-05 2024-09-11T06:51:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091506+0000", 2024-09-11T06:51:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9325000000000003e-05 2024-09-11T06:51:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091523+0000", 2024-09-11T06:51:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.77e-05 2024-09-11T06:51:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.096685+0000", 2024-09-11T06:51:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0051618569999999997 2024-09-11T06:51:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.096735+0000", 2024-09-11T06:51:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9517e-05 2024-09-11T06:51:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.5544.0:67 2.5 e82/69 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head v 82'77, mlcod=81'74)", 2024-09-11T06:51:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.595302+0000", 2024-09-11T06:51:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.510394525000001, 2024-09-11T06:51:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054511999999999998, 2024-09-11T06:51:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595302+0000", 2024-09-11T06:51:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595302+0000", 2024-09-11T06:51:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595303+0000", 2024-09-11T06:51:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2699999999999999e-07 2024-09-11T06:51:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595522+0000", 2024-09-11T06:51:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000218946 2024-09-11T06:51:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595544+0000", 2024-09-11T06:51:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2558999999999999e-05 2024-09-11T06:51:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595557+0000", 2024-09-11T06:51:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2653000000000001e-05 2024-09-11T06:51:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595662+0000", 2024-09-11T06:51:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000104862 2024-09-11T06:51:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595699+0000", 2024-09-11T06:51:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7376000000000003e-05 2024-09-11T06:51:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.600688+0000", 2024-09-11T06:51:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0049885279999999999 2024-09-11T06:51:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.600753+0000", 2024-09-11T06:51:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5549000000000005e-05 2024-09-11T06:51:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.5544.0:90 2.5 e82/69 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head v 82'86, mlcod=82'84)", 2024-09-11T06:51:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.725934+0000", 2024-09-11T06:51:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.379762764000001, 2024-09-11T06:51:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0044823220000000004, 2024-09-11T06:51:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.725934+0000", 2024-09-11T06:51:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.725934+0000", 2024-09-11T06:51:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.725936+0000", 2024-09-11T06:51:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9269999999999999e-06 2024-09-11T06:51:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.727188+0000", 2024-09-11T06:51:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012522779999999999 2024-09-11T06:51:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.727217+0000", 2024-09-11T06:51:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9373000000000002e-05 2024-09-11T06:51:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.727231+0000", 2024-09-11T06:51:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3499999999999999e-05 2024-09-11T06:51:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.727335+0000", 2024-09-11T06:51:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000104064 2024-09-11T06:51:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.727374+0000", 2024-09-11T06:51:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8967000000000001e-05 2024-09-11T06:51:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.730326+0000", 2024-09-11T06:51:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0029516519999999999 2024-09-11T06:51:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.730416+0000", 2024-09-11T06:51:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0561000000000002e-05 2024-09-11T06:51:03.199 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.199 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.199 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.199 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.199 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.199 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:03.200 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:51:03.201 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:03.194+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:03.224 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:51:03.225 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 89, 2024-09-11T06:51:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "bd5dcb33-e9f7-4a5f-a26c-a451452e8aa2", 2024-09-11T06:51:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2024-09-11T06:48:52.191706+0000", 2024-09-11T06:51:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2024-09-11T06:51:02.847293+0000", 2024-09-11T06:51:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2024-09-11T06:49:04.007203+0000", 2024-09-11T06:51:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2024-09-11T06:50:46.287457+0000", 2024-09-11T06:51:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-11T06:51:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5804032, 2024-09-11T06:51:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2024-09-11T06:51:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: "nodeep-scrub", 2024-09-11T06:51:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2024-09-11T06:51:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2024-09-11T06:51:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2024-09-11T06:51:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2024-09-11T06:51:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-11T06:51:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2024-09-11T06:51:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.94999998807907104, 2024-09-11T06:51:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-11T06:51:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-11T06:51:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2024-09-11T06:51:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 3, 2024-09-11T06:51:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 6, 2024-09-11T06:51:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2024-09-11T06:51:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "luminous", 2024-09-11T06:51:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "squid", 2024-09-11T06:51:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2024-09-11T06:51:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2024-09-11T06:51:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2024-09-11T06:51:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2024-09-11T06:51:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2024-09-11T06:49:05.084515+0000", 2024-09-11T06:51:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-11T06:51:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2024-09-11T06:51:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2024-09-11T06:51:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2024-09-11T06:51:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2024-09-11T06:51:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2024-09-11T06:51:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2024-09-11T06:51:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2024-09-11T06:51:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T06:51:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T06:51:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: "is_stretch_pool": false, 2024-09-11T06:51:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2024-09-11T06:51:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2024-09-11T06:51:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2024-09-11T06:51:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2024-09-11T06:51:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2024-09-11T06:51:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2024-09-11T06:51:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2024-09-11T06:51:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2024-09-11T06:51:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2024-09-11T06:51:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2024-09-11T06:51:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2024-09-11T06:51:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2024-09-11T06:51:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2024-09-11T06:51:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2024-09-11T06:51:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "14", 2024-09-11T06:51:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2024-09-11T06:51:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T06:51:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T06:51:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2024-09-11T06:51:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2024-09-11T06:51:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2024-09-11T06:51:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2024-09-11T06:51:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2024-09-11T06:51:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2024-09-11T06:51:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2024-09-11T06:51:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2024-09-11T06:51:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2024-09-11T06:51:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2024-09-11T06:51:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2024-09-11T06:51:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2024-09-11T06:51:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2024-09-11T06:51:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2024-09-11T06:51:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2024-09-11T06:51:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T06:51:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T06:51:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T06:51:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2024-09-11T06:51:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2024-09-11T06:51:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2024-09-11T06:51:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2024-09-11T06:51:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2024-09-11T06:51:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2024-09-11T06:51:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2024-09-11T06:51:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2024-09-11T06:51:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2024-09-11T06:51:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2024-09-11T06:51:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T06:51:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2024-09-11T06:51:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2024-09-11T06:51:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2024-09-11T06:51:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2024-09-11T06:51:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2024-09-11T06:51:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2024-09-11T06:51:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2024-09-11T06:51:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2024-09-11T06:51:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2024-09-11T06:51:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2024-09-11T06:51:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2024-09-11T06:51:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "score_type": "Fair distribution", 2024-09-11T06:51:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 6.059999942779541, 2024-09-11T06:51:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 6.059999942779541, 2024-09-11T06:51:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.33000001311302185, 2024-09-11T06:51:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2024-09-11T06:51:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2024-09-11T06:51:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 0.85340124368667603, 2024-09-11T06:51:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 0.85000002384185791, 2024-09-11T06:51:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 0.85000002384185791 2024-09-11T06:51:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 2, 2024-09-11T06:51:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "rbd", 2024-09-11T06:51:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2024-09-11T06:49:09.530167+0000", 2024-09-11T06:51:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 8193, 2024-09-11T06:51:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T06:51:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2024-09-11T06:51:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2024-09-11T06:51:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2024-09-11T06:51:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2024-09-11T06:51:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2024-09-11T06:51:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2024-09-11T06:51:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T06:51:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T06:51:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "is_stretch_pool": false, 2024-09-11T06:51:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2024-09-11T06:51:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2024-09-11T06:51:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 8, 2024-09-11T06:51:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 8, 2024-09-11T06:51:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 8, 2024-09-11T06:51:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 8, 2024-09-11T06:51:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 8, 2024-09-11T06:51:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2024-09-11T06:51:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2024-09-11T06:51:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2024-09-11T06:51:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2024-09-11T06:51:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2024-09-11T06:51:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2024-09-11T06:51:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2024-09-11T06:51:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "85", 2024-09-11T06:51:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2024-09-11T06:51:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T06:51:03.236 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T06:51:03.236 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2024-09-11T06:51:03.236 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2024-09-11T06:51:03.236 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 6, 2024-09-11T06:51:03.236 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 85, 2024-09-11T06:51:03.236 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2024-09-11T06:51:03.236 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2024-09-11T06:51:03.236 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2024-09-11T06:51:03.236 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2024-09-11T06:51:03.236 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2024-09-11T06:51:03.236 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2024-09-11T06:51:03.236 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2024-09-11T06:51:03.236 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2024-09-11T06:51:03.236 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2024-09-11T06:51:03.237 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2024-09-11T06:51:03.237 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2024-09-11T06:51:03.237 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T06:51:03.237 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T06:51:03.237 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T06:51:03.237 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2024-09-11T06:51:03.237 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2024-09-11T06:51:03.237 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2024-09-11T06:51:03.237 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2024-09-11T06:51:03.237 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2024-09-11T06:51:03.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.237 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2024-09-11T06:51:03.237 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2024-09-11T06:51:03.237 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2024-09-11T06:51:03.238 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2024-09-11T06:51:03.238 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2024-09-11T06:51:03.238 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T06:51:03.238 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2024-09-11T06:51:03.238 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2024-09-11T06:51:03.238 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2024-09-11T06:51:03.238 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2024-09-11T06:51:03.238 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2024-09-11T06:51:03.238 INFO:teuthology.orchestra.run.smithi071.stdout: "options": {}, 2024-09-11T06:51:03.238 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2024-09-11T06:51:03.238 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd": {} 2024-09-11T06:51:03.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.238 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2024-09-11T06:51:03.238 INFO:teuthology.orchestra.run.smithi071.stdout: "score_type": "Fair distribution", 2024-09-11T06:51:03.239 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 2.25, 2024-09-11T06:51:03.239 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 2.25, 2024-09-11T06:51:03.239 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2024-09-11T06:51:03.239 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2.25, 2024-09-11T06:51:03.239 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2.25, 2024-09-11T06:51:03.239 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 0.85340124368667603, 2024-09-11T06:51:03.239 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 0.85000002384185791, 2024-09-11T06:51:03.239 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 0.85000002384185791 2024-09-11T06:51:03.239 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.239 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.239 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 3, 2024-09-11T06:51:03.239 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "new_default_pool21607", 2024-09-11T06:51:03.239 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2024-09-11T06:50:15.393063+0000", 2024-09-11T06:51:03.240 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 8193, 2024-09-11T06:51:03.240 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T06:51:03.240 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2024-09-11T06:51:03.240 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2024-09-11T06:51:03.240 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2024-09-11T06:51:03.240 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2024-09-11T06:51:03.240 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2024-09-11T06:51:03.240 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2024-09-11T06:51:03.240 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T06:51:03.240 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T06:51:03.240 INFO:teuthology.orchestra.run.smithi071.stdout: "is_stretch_pool": false, 2024-09-11T06:51:03.240 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2024-09-11T06:51:03.240 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2024-09-11T06:51:03.241 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 8, 2024-09-11T06:51:03.241 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 8, 2024-09-11T06:51:03.241 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 8, 2024-09-11T06:51:03.241 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 8, 2024-09-11T06:51:03.241 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 8, 2024-09-11T06:51:03.241 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2024-09-11T06:51:03.241 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2024-09-11T06:51:03.241 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2024-09-11T06:51:03.241 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2024-09-11T06:51:03.241 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2024-09-11T06:51:03.241 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2024-09-11T06:51:03.242 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2024-09-11T06:51:03.242 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.242 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "58", 2024-09-11T06:51:03.242 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2024-09-11T06:51:03.242 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T06:51:03.242 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T06:51:03.242 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2024-09-11T06:51:03.242 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2024-09-11T06:51:03.242 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 2, 2024-09-11T06:51:03.242 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 58, 2024-09-11T06:51:03.242 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2024-09-11T06:51:03.242 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2024-09-11T06:51:03.242 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2024-09-11T06:51:03.243 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2024-09-11T06:51:03.243 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2024-09-11T06:51:03.243 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2024-09-11T06:51:03.243 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2024-09-11T06:51:03.243 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2024-09-11T06:51:03.243 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2024-09-11T06:51:03.243 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2024-09-11T06:51:03.243 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2024-09-11T06:51:03.243 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T06:51:03.243 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T06:51:03.243 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T06:51:03.243 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2024-09-11T06:51:03.243 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2024-09-11T06:51:03.243 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2024-09-11T06:51:03.244 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2024-09-11T06:51:03.244 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2024-09-11T06:51:03.244 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.244 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2024-09-11T06:51:03.244 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2024-09-11T06:51:03.244 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2024-09-11T06:51:03.244 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2024-09-11T06:51:03.244 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2024-09-11T06:51:03.244 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T06:51:03.244 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2024-09-11T06:51:03.244 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2024-09-11T06:51:03.244 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2024-09-11T06:51:03.244 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2024-09-11T06:51:03.244 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2024-09-11T06:51:03.245 INFO:teuthology.orchestra.run.smithi071.stdout: "options": {}, 2024-09-11T06:51:03.245 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2024-09-11T06:51:03.245 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd": {} 2024-09-11T06:51:03.245 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.245 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2024-09-11T06:51:03.245 INFO:teuthology.orchestra.run.smithi071.stdout: "score_type": "Fair distribution", 2024-09-11T06:51:03.245 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 3, 2024-09-11T06:51:03.245 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 3, 2024-09-11T06:51:03.245 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2024-09-11T06:51:03.245 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 3, 2024-09-11T06:51:03.245 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 3, 2024-09-11T06:51:03.245 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 0.85340124368667603, 2024-09-11T06:51:03.245 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 0.85000002384185791, 2024-09-11T06:51:03.245 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 0.85000002384185791 2024-09-11T06:51:03.246 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.246 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.246 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-11T06:51:03.246 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2024-09-11T06:51:03.246 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.246 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2024-09-11T06:51:03.246 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "3c9e35bf-d3db-49db-b10c-d2cf38c7eb7e", 2024-09-11T06:51:03.246 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-11T06:51:03.246 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-11T06:51:03.246 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-11T06:51:03.246 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 0.1204071044921875, 2024-09-11T06:51:03.246 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-11T06:51:03.246 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-11T06:51:03.246 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 11, 2024-09-11T06:51:03.247 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 71, 2024-09-11T06:51:03.247 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-11T06:51:03.247 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-11T06:51:03.247 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-11T06:51:03.247 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:03.247 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.247 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:03.247 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2024-09-11T06:51:03.247 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1158147777 2024-09-11T06:51:03.247 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.247 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.247 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:03.247 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2024-09-11T06:51:03.247 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1158147777 2024-09-11T06:51:03.248 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.248 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.248 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.248 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-11T06:51:03.248 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:03.248 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.248 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:03.248 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2024-09-11T06:51:03.248 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1158147777 2024-09-11T06:51:03.248 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.248 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.248 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:03.248 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2024-09-11T06:51:03.248 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1158147777 2024-09-11T06:51:03.249 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.249 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.249 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.249 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-11T06:51:03.249 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:03.249 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.249 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:03.249 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2024-09-11T06:51:03.249 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1158147777 2024-09-11T06:51:03.249 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.249 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.249 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:03.249 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2024-09-11T06:51:03.249 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1158147777 2024-09-11T06:51:03.250 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.250 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.250 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.250 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-11T06:51:03.250 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:03.250 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.250 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:03.250 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2024-09-11T06:51:03.250 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1158147777 2024-09-11T06:51:03.250 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.250 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.250 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:03.250 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2024-09-11T06:51:03.250 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1158147777 2024-09-11T06:51:03.250 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.251 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.251 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.251 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/1158147777", 2024-09-11T06:51:03.251 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/1158147777", 2024-09-11T06:51:03.251 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/1158147777", 2024-09-11T06:51:03.251 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/1158147777", 2024-09-11T06:51:03.251 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-11T06:51:03.251 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-11T06:51:03.251 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-11T06:51:03.251 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.251 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.251 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.251 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2024-09-11T06:51:03.251 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "693649dc-db7c-43cf-bdbe-87f6128ef311", 2024-09-11T06:51:03.252 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-11T06:51:03.252 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-11T06:51:03.252 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-11T06:51:03.252 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-11T06:51:03.252 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-11T06:51:03.252 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-11T06:51:03.252 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 11, 2024-09-11T06:51:03.252 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 86, 2024-09-11T06:51:03.252 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-11T06:51:03.252 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-11T06:51:03.252 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-11T06:51:03.252 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:03.252 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.252 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:03.253 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2024-09-11T06:51:03.253 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 620364628 2024-09-11T06:51:03.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.253 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:03.253 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2024-09-11T06:51:03.253 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 620364628 2024-09-11T06:51:03.253 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.253 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.253 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-11T06:51:03.253 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:03.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.253 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:03.254 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2024-09-11T06:51:03.254 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 620364628 2024-09-11T06:51:03.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.254 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.254 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:03.254 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2024-09-11T06:51:03.254 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 620364628 2024-09-11T06:51:03.254 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.254 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.254 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-11T06:51:03.254 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:03.254 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.254 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:03.255 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2024-09-11T06:51:03.255 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 620364628 2024-09-11T06:51:03.255 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.255 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.255 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:03.255 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2024-09-11T06:51:03.255 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 620364628 2024-09-11T06:51:03.255 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.255 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.255 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.255 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-11T06:51:03.255 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:03.255 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.255 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:03.256 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2024-09-11T06:51:03.256 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 620364628 2024-09-11T06:51:03.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.256 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.256 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:03.256 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2024-09-11T06:51:03.256 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 620364628 2024-09-11T06:51:03.256 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.256 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.256 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/620364628", 2024-09-11T06:51:03.256 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6811/620364628", 2024-09-11T06:51:03.256 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6815/620364628", 2024-09-11T06:51:03.257 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6813/620364628", 2024-09-11T06:51:03.257 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-11T06:51:03.257 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-11T06:51:03.257 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-11T06:51:03.257 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.257 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.257 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2024-09-11T06:51:03.257 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "89421ae7-fead-409c-83f7-ede424db6004", 2024-09-11T06:51:03.257 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-11T06:51:03.257 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-11T06:51:03.257 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-11T06:51:03.257 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-11T06:51:03.257 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-11T06:51:03.258 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-11T06:51:03.258 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 11, 2024-09-11T06:51:03.258 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 69, 2024-09-11T06:51:03.258 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-11T06:51:03.258 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-11T06:51:03.258 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-11T06:51:03.258 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:03.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.258 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:03.258 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2024-09-11T06:51:03.258 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926882521 2024-09-11T06:51:03.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.258 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:03.259 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2024-09-11T06:51:03.259 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926882521 2024-09-11T06:51:03.259 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.259 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.259 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-11T06:51:03.259 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:03.259 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.259 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:03.259 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2024-09-11T06:51:03.259 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926882521 2024-09-11T06:51:03.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.260 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.260 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:03.260 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2024-09-11T06:51:03.260 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926882521 2024-09-11T06:51:03.260 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.260 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.260 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-11T06:51:03.260 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:03.260 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.260 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:03.260 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2024-09-11T06:51:03.260 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926882521 2024-09-11T06:51:03.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.261 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.261 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:03.261 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2024-09-11T06:51:03.261 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926882521 2024-09-11T06:51:03.261 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.261 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.261 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.261 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-11T06:51:03.261 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:03.261 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.261 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:03.261 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2024-09-11T06:51:03.261 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926882521 2024-09-11T06:51:03.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.262 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.262 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:03.262 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2024-09-11T06:51:03.262 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926882521 2024-09-11T06:51:03.262 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.262 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.262 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6817/2926882521", 2024-09-11T06:51:03.262 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6819/2926882521", 2024-09-11T06:51:03.262 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/2926882521", 2024-09-11T06:51:03.262 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6821/2926882521", 2024-09-11T06:51:03.262 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-11T06:51:03.262 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-11T06:51:03.263 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-11T06:51:03.263 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.263 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.263 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.263 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2024-09-11T06:51:03.263 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "f4512e48-63f1-42b6-81be-594bd82e3210", 2024-09-11T06:51:03.263 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-11T06:51:03.263 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-11T06:51:03.263 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-11T06:51:03.263 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-11T06:51:03.263 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-11T06:51:03.263 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-11T06:51:03.263 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 11, 2024-09-11T06:51:03.264 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 72, 2024-09-11T06:51:03.264 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-11T06:51:03.264 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-11T06:51:03.264 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-11T06:51:03.264 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:03.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.264 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:03.264 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6808", 2024-09-11T06:51:03.264 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 558942315 2024-09-11T06:51:03.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.264 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:03.264 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6809", 2024-09-11T06:51:03.264 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 558942315 2024-09-11T06:51:03.265 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.265 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.265 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-11T06:51:03.265 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:03.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.265 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:03.265 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6810", 2024-09-11T06:51:03.265 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 558942315 2024-09-11T06:51:03.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.265 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:03.265 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6811", 2024-09-11T06:51:03.266 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 558942315 2024-09-11T06:51:03.266 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.266 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.266 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-11T06:51:03.266 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:03.266 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.266 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:03.266 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6814", 2024-09-11T06:51:03.266 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 558942315 2024-09-11T06:51:03.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.266 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.266 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:03.266 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6815", 2024-09-11T06:51:03.266 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 558942315 2024-09-11T06:51:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-11T06:51:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6812", 2024-09-11T06:51:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 558942315 2024-09-11T06:51:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6813", 2024-09-11T06:51:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 558942315 2024-09-11T06:51:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.268 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.268 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.89:6809/558942315", 2024-09-11T06:51:03.268 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.89:6811/558942315", 2024-09-11T06:51:03.268 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.89:6815/558942315", 2024-09-11T06:51:03.268 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.89:6813/558942315", 2024-09-11T06:51:03.268 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-11T06:51:03.268 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-11T06:51:03.268 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-11T06:51:03.268 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.268 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2024-09-11T06:51:03.269 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "52d47b6f-af30-455e-8326-e53c598911ae", 2024-09-11T06:51:03.269 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-11T06:51:03.269 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-11T06:51:03.269 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-11T06:51:03.269 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-11T06:51:03.269 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-11T06:51:03.269 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-11T06:51:03.269 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 11, 2024-09-11T06:51:03.269 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 60, 2024-09-11T06:51:03.269 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-11T06:51:03.269 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-11T06:51:03.269 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-11T06:51:03.269 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:03.269 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.270 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:03.270 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6816", 2024-09-11T06:51:03.270 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3878488003 2024-09-11T06:51:03.270 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.270 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.270 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:03.270 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6817", 2024-09-11T06:51:03.270 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3878488003 2024-09-11T06:51:03.270 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.270 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.270 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.270 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-11T06:51:03.270 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:03.270 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.271 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:03.271 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6818", 2024-09-11T06:51:03.271 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3878488003 2024-09-11T06:51:03.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.271 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:03.271 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6819", 2024-09-11T06:51:03.271 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3878488003 2024-09-11T06:51:03.271 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.271 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.271 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-11T06:51:03.271 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:03.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.272 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:03.272 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6822", 2024-09-11T06:51:03.272 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3878488003 2024-09-11T06:51:03.272 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.272 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.272 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:03.272 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6823", 2024-09-11T06:51:03.272 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3878488003 2024-09-11T06:51:03.272 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.272 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.272 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.272 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-11T06:51:03.272 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:03.272 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.273 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:03.273 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6820", 2024-09-11T06:51:03.273 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3878488003 2024-09-11T06:51:03.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.273 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:03.273 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6821", 2024-09-11T06:51:03.273 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3878488003 2024-09-11T06:51:03.273 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.273 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.273 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.89:6817/3878488003", 2024-09-11T06:51:03.273 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.89:6819/3878488003", 2024-09-11T06:51:03.274 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.89:6823/3878488003", 2024-09-11T06:51:03.274 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.89:6821/3878488003", 2024-09-11T06:51:03.274 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-11T06:51:03.274 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-11T06:51:03.274 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-11T06:51:03.274 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.274 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.274 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.274 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2024-09-11T06:51:03.274 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "6b4f9416-0ba0-4c58-883e-8cbdb362adcc", 2024-09-11T06:51:03.274 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-11T06:51:03.274 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-11T06:51:03.274 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-11T06:51:03.275 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-11T06:51:03.275 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-11T06:51:03.275 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-11T06:51:03.275 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 11, 2024-09-11T06:51:03.275 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 80, 2024-09-11T06:51:03.275 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-11T06:51:03.275 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-11T06:51:03.275 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-11T06:51:03.275 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:03.275 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.275 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:03.275 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6800", 2024-09-11T06:51:03.275 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2840204107 2024-09-11T06:51:03.275 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.276 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.276 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:03.276 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6801", 2024-09-11T06:51:03.276 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2840204107 2024-09-11T06:51:03.276 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.276 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.276 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-11T06:51:03.276 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:03.276 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.276 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:03.276 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6802", 2024-09-11T06:51:03.276 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2840204107 2024-09-11T06:51:03.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.277 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:03.277 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6803", 2024-09-11T06:51:03.277 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2840204107 2024-09-11T06:51:03.277 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.277 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.277 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-11T06:51:03.277 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:03.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.277 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:03.277 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6806", 2024-09-11T06:51:03.277 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2840204107 2024-09-11T06:51:03.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.278 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.278 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:03.278 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6807", 2024-09-11T06:51:03.278 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2840204107 2024-09-11T06:51:03.278 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.278 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.278 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-11T06:51:03.278 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:03.278 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.278 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:03.278 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6804", 2024-09-11T06:51:03.278 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2840204107 2024-09-11T06:51:03.279 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.279 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.279 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:03.279 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6805", 2024-09-11T06:51:03.279 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2840204107 2024-09-11T06:51:03.279 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.279 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.279 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.279 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.89:6801/2840204107", 2024-09-11T06:51:03.279 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.89:6803/2840204107", 2024-09-11T06:51:03.279 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.89:6807/2840204107", 2024-09-11T06:51:03.279 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.89:6805/2840204107", 2024-09-11T06:51:03.279 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-11T06:51:03.279 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-11T06:51:03.280 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-11T06:51:03.280 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.280 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.280 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-11T06:51:03.280 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2024-09-11T06:51:03.280 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.280 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2024-09-11T06:51:03.280 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-11T06:51:03.280 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-11T06:51:03.280 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-11T06:51:03.280 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540701547738038271, 2024-09-11T06:51:03.280 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-11T06:51:03.280 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-11T06:49:00.282785+0000", 2024-09-11T06:51:03.280 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-11T06:51:03.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.281 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.281 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2024-09-11T06:51:03.281 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-11T06:51:03.281 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-11T06:51:03.281 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-11T06:51:03.281 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540701547738038271, 2024-09-11T06:51:03.281 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-11T06:51:03.281 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-11T06:49:00.381259+0000", 2024-09-11T06:51:03.281 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-11T06:51:03.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.281 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.281 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2024-09-11T06:51:03.281 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-11T06:51:03.282 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-11T06:51:03.282 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-11T06:51:03.282 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540701547738038271, 2024-09-11T06:51:03.282 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-11T06:51:03.282 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-11T06:49:00.986977+0000", 2024-09-11T06:51:03.282 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-11T06:51:03.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.282 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2024-09-11T06:51:03.282 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-11T06:51:03.282 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-11T06:51:03.282 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-11T06:51:03.282 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540701547738038271, 2024-09-11T06:51:03.282 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-11T06:51:03.283 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-11T06:49:00.732329+0000", 2024-09-11T06:51:03.283 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-11T06:51:03.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.283 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2024-09-11T06:51:03.283 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-11T06:51:03.283 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-11T06:51:03.283 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-11T06:51:03.283 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540701547738038271, 2024-09-11T06:51:03.283 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-11T06:51:03.283 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-11T06:49:00.844988+0000", 2024-09-11T06:51:03.283 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-11T06:51:03.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.284 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2024-09-11T06:51:03.284 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-11T06:51:03.284 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-11T06:51:03.284 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-11T06:51:03.284 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540701547738038271, 2024-09-11T06:51:03.284 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-11T06:51:03.284 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-11T06:49:00.639989+0000", 2024-09-11T06:51:03.284 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-11T06:51:03.284 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.284 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-11T06:51:03.284 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2024-09-11T06:51:03.284 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [ 2024-09-11T06:51:03.284 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.284 INFO:teuthology.orchestra.run.smithi071.stdout: "pgid": "3.5", 2024-09-11T06:51:03.285 INFO:teuthology.orchestra.run.smithi071.stdout: "mappings": [ 2024-09-11T06:51:03.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.285 INFO:teuthology.orchestra.run.smithi071.stdout: "from": 5, 2024-09-11T06:51:03.285 INFO:teuthology.orchestra.run.smithi071.stdout: "to": 1 2024-09-11T06:51:03.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.285 INFO:teuthology.orchestra.run.smithi071.stdout: "from": 2, 2024-09-11T06:51:03.285 INFO:teuthology.orchestra.run.smithi071.stdout: "to": 3 2024-09-11T06:51:03.285 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.285 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.285 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.285 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-11T06:51:03.285 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2024-09-11T06:51:03.285 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2024-09-11T06:51:03.286 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2024-09-11T06:51:03.286 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2024-09-11T06:51:03.286 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2024-09-11T06:51:03.286 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2024-09-11T06:51:03.286 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2024-09-11T06:51:03.286 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2024-09-11T06:51:03.286 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2024-09-11T06:51:03.286 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2024-09-11T06:51:03.286 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2024-09-11T06:51:03.286 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2024-09-11T06:51:03.286 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.286 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2024-09-11T06:51:03.286 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2024-09-11T06:51:03.286 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2024-09-11T06:51:03.287 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2024-09-11T06:51:03.287 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2024-09-11T06:51:03.287 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2024-09-11T06:51:03.287 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2024-09-11T06:51:03.287 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2024-09-11T06:51:03.287 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2024-09-11T06:51:03.287 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2024-09-11T06:51:03.287 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2024-09-11T06:51:03.287 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.287 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:03.289 INFO:tasks.thrashosds.thrasher:j is {'epoch': 89, 'fsid': 'bd5dcb33-e9f7-4a5f-a26c-a451452e8aa2', 'created': '2024-09-11T06:48:52.191706+0000', 'modified': '2024-09-11T06:51:02.847293+0000', 'last_up_change': '2024-09-11T06:49:04.007203+0000', 'last_in_change': '2024-09-11T06:50:46.287457+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T06:49:05.084515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.853401243686676, 'average_primary_affinity': 0.8500000238418579, 'average_primary_affinity_weighted': 0.8500000238418579}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-11T06:49:09.530167+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 6, 'snap_epoch': 85, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.853401243686676, 'average_primary_affinity': 0.8500000238418579, 'average_primary_affinity_weighted': 0.8500000238418579}}, {'pool': 3, 'pool_name': 'new_default_pool21607', 'create_time': '2024-09-11T06:50:15.393063+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 58, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 1, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.853401243686676, 'average_primary_affinity': 0.8500000238418579, 'average_primary_affinity_weighted': 0.8500000238418579}}], 'osds': [{'osd': 0, 'uuid': '3c9e35bf-d3db-49db-b10c-d2cf38c7eb7e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.1204071044921875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 71, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6800', 'nonce': 1158147777}, {'type': 'v1', 'addr': '172.21.15.71:6801', 'nonce': 1158147777}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6802', 'nonce': 1158147777}, {'type': 'v1', 'addr': '172.21.15.71:6803', 'nonce': 1158147777}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6806', 'nonce': 1158147777}, {'type': 'v1', 'addr': '172.21.15.71:6807', 'nonce': 1158147777}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6804', 'nonce': 1158147777}, {'type': 'v1', 'addr': '172.21.15.71:6805', 'nonce': 1158147777}]}, 'public_addr': '172.21.15.71:6801/1158147777', 'cluster_addr': '172.21.15.71:6803/1158147777', 'heartbeat_back_addr': '172.21.15.71:6807/1158147777', 'heartbeat_front_addr': '172.21.15.71:6805/1158147777', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '693649dc-db7c-43cf-bdbe-87f6128ef311', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6808', 'nonce': 620364628}, {'type': 'v1', 'addr': '172.21.15.71:6809', 'nonce': 620364628}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6810', 'nonce': 620364628}, {'type': 'v1', 'addr': '172.21.15.71:6811', 'nonce': 620364628}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6814', 'nonce': 620364628}, {'type': 'v1', 'addr': '172.21.15.71:6815', 'nonce': 620364628}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6812', 'nonce': 620364628}, {'type': 'v1', 'addr': '172.21.15.71:6813', 'nonce': 620364628}]}, 'public_addr': '172.21.15.71:6809/620364628', 'cluster_addr': '172.21.15.71:6811/620364628', 'heartbeat_back_addr': '172.21.15.71:6815/620364628', 'heartbeat_front_addr': '172.21.15.71:6813/620364628', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '89421ae7-fead-409c-83f7-ede424db6004', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 69, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6816', 'nonce': 2926882521}, {'type': 'v1', 'addr': '172.21.15.71:6817', 'nonce': 2926882521}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6818', 'nonce': 2926882521}, {'type': 'v1', 'addr': '172.21.15.71:6819', 'nonce': 2926882521}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6822', 'nonce': 2926882521}, {'type': 'v1', 'addr': '172.21.15.71:6823', 'nonce': 2926882521}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6820', 'nonce': 2926882521}, {'type': 'v1', 'addr': '172.21.15.71:6821', 'nonce': 2926882521}]}, 'public_addr': '172.21.15.71:6817/2926882521', 'cluster_addr': '172.21.15.71:6819/2926882521', 'heartbeat_back_addr': '172.21.15.71:6823/2926882521', 'heartbeat_front_addr': '172.21.15.71:6821/2926882521', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'f4512e48-63f1-42b6-81be-594bd82e3210', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 72, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6808', 'nonce': 558942315}, {'type': 'v1', 'addr': '172.21.15.89:6809', 'nonce': 558942315}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6810', 'nonce': 558942315}, {'type': 'v1', 'addr': '172.21.15.89:6811', 'nonce': 558942315}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6814', 'nonce': 558942315}, {'type': 'v1', 'addr': '172.21.15.89:6815', 'nonce': 558942315}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6812', 'nonce': 558942315}, {'type': 'v1', 'addr': '172.21.15.89:6813', 'nonce': 558942315}]}, 'public_addr': '172.21.15.89:6809/558942315', 'cluster_addr': '172.21.15.89:6811/558942315', 'heartbeat_back_addr': '172.21.15.89:6815/558942315', 'heartbeat_front_addr': '172.21.15.89:6813/558942315', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '52d47b6f-af30-455e-8326-e53c598911ae', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 60, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6816', 'nonce': 3878488003}, {'type': 'v1', 'addr': '172.21.15.89:6817', 'nonce': 3878488003}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6818', 'nonce': 3878488003}, {'type': 'v1', 'addr': '172.21.15.89:6819', 'nonce': 3878488003}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6822', 'nonce': 3878488003}, {'type': 'v1', 'addr': '172.21.15.89:6823', 'nonce': 3878488003}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6820', 'nonce': 3878488003}, {'type': 'v1', 'addr': '172.21.15.89:6821', 'nonce': 3878488003}]}, 'public_addr': '172.21.15.89:6817/3878488003', 'cluster_addr': '172.21.15.89:6819/3878488003', 'heartbeat_back_addr': '172.21.15.89:6823/3878488003', 'heartbeat_front_addr': '172.21.15.89:6821/3878488003', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '6b4f9416-0ba0-4c58-883e-8cbdb362adcc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 80, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6800', 'nonce': 2840204107}, {'type': 'v1', 'addr': '172.21.15.89:6801', 'nonce': 2840204107}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6802', 'nonce': 2840204107}, {'type': 'v1', 'addr': '172.21.15.89:6803', 'nonce': 2840204107}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6806', 'nonce': 2840204107}, {'type': 'v1', 'addr': '172.21.15.89:6807', 'nonce': 2840204107}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6804', 'nonce': 2840204107}, {'type': 'v1', 'addr': '172.21.15.89:6805', 'nonce': 2840204107}]}, 'public_addr': '172.21.15.89:6801/2840204107', 'cluster_addr': '172.21.15.89:6803/2840204107', 'heartbeat_back_addr': '172.21.15.89:6807/2840204107', 'heartbeat_front_addr': '172.21.15.89:6805/2840204107', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T06:49:00.282785+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T06:49:00.381259+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T06:49:00.986977+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T06:49:00.732329+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T06:49:00.844988+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T06:49:00.639989+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.5', 'mappings': [{'from': 5, 'to': 1}, {'from': 2, 'to': 3}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-11T06:51:03.289 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:51:03.333 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:03.330+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:03.401 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:03.398+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:03.412 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:03.412 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:03.413 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:51:03.413 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:03.427 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:51:03.513 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:03.510+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:03.581 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:03.581 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:03.581 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:51:03.582 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:51:03.582 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:03.594 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:51:03.630 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:51:03.630 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:51:03.631 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:03.626+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:03.647 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-09-11T06:51:03.038789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":137954173,"num_objects":59,"num_object_clones":3,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":3338,"num_read_kb":110297,"num_write":758,"num_write_kb":268861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":186911175,"num_keys_recovered":22,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":714,"ondisk_log_size":714,"up":34,"acting":34,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":458436,"kb_used_data":275444,"kb_used_omap":9,"kb_used_meta":182838,"kb_avail":565772604,"statfs":{"total":579820584960,"available":579351146496,"internally_reserved":0,"allocated":282054656,"data_stored":297090215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9565,"internal_metadata":187226787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1048614,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":74,"num_write":8,"num_write_kb":1030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.211292"},"pg_stats":[{"pgid":"1.0","version":"14'117","reported_seq":312,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T06:50:58.837214+0000","last_change":"2024-09-11T06:50:37.941212+0000","last_active":"2024-09-11T06:50:58.837214+0000","last_peered":"2024-09-11T06:50:58.837214+0000","last_clean":"2024-09-11T06:50:58.837214+0000","last_became_active":"2024-09-11T06:50:37.940448+0000","last_became_peered":"2024-09-11T06:50:37.940448+0000","last_unstale":"2024-09-11T06:50:58.837214+0000","last_undegraded":"2024-09-11T06:50:58.837214+0000","last_fullsized":"2024-09-11T06:50:58.837214+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_clean_scrub_stamp":"2024-09-11T06:49:05.088839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:48:59.833858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"72'39","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T06:50:58.845296+0000","last_change":"2024-09-11T06:50:47.401354+0000","last_active":"2024-09-11T06:50:58.845296+0000","last_peered":"2024-09-11T06:50:58.845296+0000","last_clean":"2024-09-11T06:50:58.845296+0000","last_became_active":"2024-09-11T06:50:47.306211+0000","last_became_peered":"2024-09-11T06:50:47.306211+0000","last_unstale":"2024-09-11T06:50:58.845296+0000","last_undegraded":"2024-09-11T06:50:58.845296+0000","last_fullsized":"2024-09-11T06:50:58.845296+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:07:23.731938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011212469999999999,"stat_sum":{"num_bytes":12582950,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":70,"num_read_kb":8196,"num_write":38,"num_write_kb":28673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":25165862,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"58'2","reported_seq":110,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T06:50:58.840276+0000","last_change":"2024-09-11T06:50:47.310511+0000","last_active":"2024-09-11T06:50:58.840276+0000","last_peered":"2024-09-11T06:50:58.840276+0000","last_clean":"2024-09-11T06:50:58.840276+0000","last_became_active":"2024-09-11T06:50:47.304328+0000","last_became_peered":"2024-09-11T06:50:47.304328+0000","last_unstale":"2024-09-11T06:50:58.840276+0000","last_undegraded":"2024-09-11T06:50:58.840276+0000","last_fullsized":"2024-09-11T06:50:58.840276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:21:15.806857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011764950000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"68'57","reported_seq":207,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T06:50:58.837294+0000","last_change":"2024-09-11T06:50:35.668519+0000","last_active":"2024-09-11T06:50:58.837294+0000","last_peered":"2024-09-11T06:50:58.837294+0000","last_clean":"2024-09-11T06:50:58.837294+0000","last_became_active":"2024-09-11T06:50:35.238279+0000","last_became_peered":"2024-09-11T06:50:35.238279+0000","last_unstale":"2024-09-11T06:50:58.837294+0000","last_undegraded":"2024-09-11T06:50:58.837294+0000","last_fullsized":"2024-09-11T06:50:58.837294+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:55.964462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077615599999999996,"stat_sum":{"num_bytes":20971520,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":150,"num_read_kb":23060,"num_write":56,"num_write_kb":45056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":41943040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":76,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T06:50:59.019843+0000","last_change":"2024-09-11T06:50:19.703255+0000","last_active":"2024-09-11T06:50:59.019843+0000","last_peered":"2024-09-11T06:50:59.019843+0000","last_clean":"2024-09-11T06:50:59.019843+0000","last_became_active":"2024-09-11T06:50:17.089296+0000","last_became_peered":"2024-09-11T06:50:17.089296+0000","last_unstale":"2024-09-11T06:50:59.019843+0000","last_undegraded":"2024-09-11T06:50:59.019843+0000","last_fullsized":"2024-09-11T06:50:59.019843+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:42:18.684413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001159373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"85'128","reported_seq":1256,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T06:50:58.837133+0000","last_change":"2024-09-11T06:49:55.948809+0000","last_active":"2024-09-11T06:50:58.837133+0000","last_peered":"2024-09-11T06:50:58.837133+0000","last_clean":"2024-09-11T06:50:58.837133+0000","last_became_active":"2024-09-11T06:49:55.948499+0000","last_became_peered":"2024-09-11T06:49:55.948499+0000","last_unstale":"2024-09-11T06:50:58.837133+0000","last_undegraded":"2024-09-11T06:50:58.837133+0000","last_fullsized":"2024-09-11T06:50:58.837133+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:43:13.588972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010163780000000001,"stat_sum":{"num_bytes":25165862,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1694,"num_read_kb":23684,"num_write":69,"num_write_kb":40970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"79'1","reported_seq":78,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T06:50:58.840196+0000","last_change":"2024-09-11T06:50:19.097182+0000","last_active":"2024-09-11T06:50:58.840196+0000","last_peered":"2024-09-11T06:50:58.840196+0000","last_clean":"2024-09-11T06:50:58.840196+0000","last_became_active":"2024-09-11T06:50:17.097783+0000","last_became_peered":"2024-09-11T06:50:17.097783+0000","last_unstale":"2024-09-11T06:50:58.840196+0000","last_undegraded":"2024-09-11T06:50:58.840196+0000","last_fullsized":"2024-09-11T06:50:58.840196+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:35:49.244377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088028100000000001,"stat_sum":{"num_bytes":35,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"54'4","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T06:50:58.845269+0000","last_change":"2024-09-11T06:50:47.453549+0000","last_active":"2024-09-11T06:50:58.845269+0000","last_peered":"2024-09-11T06:50:58.845269+0000","last_clean":"2024-09-11T06:50:58.845269+0000","last_became_active":"2024-09-11T06:50:47.306692+0000","last_became_peered":"2024-09-11T06:50:47.306692+0000","last_unstale":"2024-09-11T06:50:58.845269+0000","last_undegraded":"2024-09-11T06:50:58.845269+0000","last_fullsized":"2024-09-11T06:50:58.845269+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:49:15.322538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001910244,"stat_sum":{"num_bytes":8388627,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":4,"num_write_kb":8194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":8388684,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"56'1","reported_seq":77,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T06:50:59.019844+0000","last_change":"2024-09-11T06:50:19.703261+0000","last_active":"2024-09-11T06:50:59.019844+0000","last_peered":"2024-09-11T06:50:59.019844+0000","last_clean":"2024-09-11T06:50:59.019844+0000","last_became_active":"2024-09-11T06:50:17.089261+0000","last_became_peered":"2024-09-11T06:50:17.089261+0000","last_unstale":"2024-09-11T06:50:59.019844+0000","last_undegraded":"2024-09-11T06:50:59.019844+0000","last_fullsized":"2024-09-11T06:50:59.019844+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:00:38.491173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001141853,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T06:50:58.853162+0000","last_change":"2024-09-11T06:50:58.852200+0000","last_active":"2024-09-11T06:50:58.853162+0000","last_peered":"2024-09-11T06:50:58.853162+0000","last_clean":"2024-09-11T06:50:58.853162+0000","last_became_active":"2024-09-11T06:50:58.843758+0000","last_became_peered":"2024-09-11T06:50:58.843758+0000","last_unstale":"2024-09-11T06:50:58.853162+0000","last_undegraded":"2024-09-11T06:50:58.853162+0000","last_fullsized":"2024-09-11T06:50:58.853162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:08.232430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"85'114","reported_seq":295,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T06:50:58.840466+0000","last_change":"2024-09-11T06:49:56.122331+0000","last_active":"2024-09-11T06:50:58.840466+0000","last_peered":"2024-09-11T06:50:58.840466+0000","last_clean":"2024-09-11T06:50:58.840466+0000","last_became_active":"2024-09-11T06:49:56.121998+0000","last_became_peered":"2024-09-11T06:49:56.121998+0000","last_unstale":"2024-09-11T06:50:58.840466+0000","last_undegraded":"2024-09-11T06:50:58.840466+0000","last_fullsized":"2024-09-11T06:50:58.840466+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:37:13.880382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582950,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":534,"num_read_kb":8607,"num_write":181,"num_write_kb":20622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"79'4","reported_seq":91,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T06:50:58.840418+0000","last_change":"2024-09-11T06:50:19.097504+0000","last_active":"2024-09-11T06:50:58.840418+0000","last_peered":"2024-09-11T06:50:58.840418+0000","last_clean":"2024-09-11T06:50:58.840418+0000","last_became_active":"2024-09-11T06:50:17.096243+0000","last_became_peered":"2024-09-11T06:50:17.096243+0000","last_unstale":"2024-09-11T06:50:58.840418+0000","last_undegraded":"2024-09-11T06:50:58.840418+0000","last_fullsized":"2024-09-11T06:50:58.840418+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:44:45.211005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085501399999999997,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":25,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"85'90","reported_seq":257,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T06:50:58.837143+0000","last_change":"2024-09-11T06:50:35.337013+0000","last_active":"2024-09-11T06:50:58.837143+0000","last_peered":"2024-09-11T06:50:58.837143+0000","last_clean":"2024-09-11T06:50:58.837143+0000","last_became_active":"2024-09-11T06:50:35.236333+0000","last_became_peered":"2024-09-11T06:50:35.236333+0000","last_unstale":"2024-09-11T06:50:58.837143+0000","last_undegraded":"2024-09-11T06:50:58.837143+0000","last_fullsized":"2024-09-11T06:50:58.837143+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:10:00.941790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095172900000000001,"stat_sum":{"num_bytes":24117318,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":340,"num_read_kb":17580,"num_write":87,"num_write_kb":40964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":41943104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":77,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T06:50:58.837192+0000","last_change":"2024-09-11T06:50:19.096344+0000","last_active":"2024-09-11T06:50:58.837192+0000","last_peered":"2024-09-11T06:50:58.837192+0000","last_clean":"2024-09-11T06:50:58.837192+0000","last_became_active":"2024-09-11T06:50:17.096560+0000","last_became_peered":"2024-09-11T06:50:17.096560+0000","last_unstale":"2024-09-11T06:50:58.837192+0000","last_undegraded":"2024-09-11T06:50:58.837192+0000","last_fullsized":"2024-09-11T06:50:58.837192+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:03:19.807744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053029399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"74'43","reported_seq":219,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T06:50:58.840799+0000","last_change":"2024-09-11T06:50:35.320796+0000","last_active":"2024-09-11T06:50:58.840799+0000","last_peered":"2024-09-11T06:50:58.840799+0000","last_clean":"2024-09-11T06:50:58.840799+0000","last_became_active":"2024-09-11T06:50:35.241589+0000","last_became_peered":"2024-09-11T06:50:35.241589+0000","last_unstale":"2024-09-11T06:50:58.840799+0000","last_undegraded":"2024-09-11T06:50:58.840799+0000","last_fullsized":"2024-09-11T06:50:58.840799+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:18.791577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388646,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":118,"num_read_kb":8286,"num_write":59,"num_write_kb":16420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":16777292,"num_keys_recovered":6,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":77,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T06:50:58.836496+0000","last_change":"2024-09-11T06:50:19.688101+0000","last_active":"2024-09-11T06:50:58.836496+0000","last_peered":"2024-09-11T06:50:58.836496+0000","last_clean":"2024-09-11T06:50:58.836496+0000","last_became_active":"2024-09-11T06:50:17.096959+0000","last_became_peered":"2024-09-11T06:50:17.096959+0000","last_unstale":"2024-09-11T06:50:58.836496+0000","last_undegraded":"2024-09-11T06:50:58.836496+0000","last_fullsized":"2024-09-11T06:50:58.836496+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:54:30.491263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059833200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"82'113","reported_seq":165,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T06:50:58.837176+0000","last_change":"2024-09-11T06:50:35.519269+0000","last_active":"2024-09-11T06:50:58.837176+0000","last_peered":"2024-09-11T06:50:58.837176+0000","last_clean":"2024-09-11T06:50:58.837176+0000","last_became_active":"2024-09-11T06:50:35.238011+0000","last_became_peered":"2024-09-11T06:50:35.238011+0000","last_unstale":"2024-09-11T06:50:58.837176+0000","last_undegraded":"2024-09-11T06:50:58.837176+0000","last_fullsized":"2024-09-11T06:50:58.837176+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:16.367234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001403348,"stat_sum":{"num_bytes":25165862,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":267,"num_read_kb":20741,"num_write":99,"num_write_kb":65540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":50331648,"num_keys_recovered":16,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":70,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":31,"num_read_kb":25,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":73,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":16,"acting":16,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":137363735,"num_objects":51,"num_object_clones":3,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":3191,"num_read_kb":110172,"num_write":593,"num_write_kb":266439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":184549630,"num_keys_recovered":22,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":274800640,"data_stored":293601838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":588,"ondisk_log_size":588,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640281,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48524,"kb_used_data":21488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94323316,"statfs":{"total":96636764160,"available":96587075584,"internally_reserved":0,"allocated":22003712,"data_stored":21361173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640282,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56740,"kb_used_data":25552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94315100,"statfs":{"total":96636764160,"available":96578662400,"internally_reserved":0,"allocated":26165248,"data_stored":25545438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640282,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109576,"kb_used_data":78384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94262264,"statfs":{"total":96636764160,"available":96524558336,"internally_reserved":0,"allocated":80265216,"data_stored":89055489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640283,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69056,"kb_used_data":37864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94302784,"statfs":{"total":96636764160,"available":96566050816,"internally_reserved":0,"allocated":38772736,"data_stored":38133454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640283,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105928,"kb_used_data":74736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94265912,"statfs":{"total":96636764160,"available":96528293888,"internally_reserved":0,"allocated":76529664,"data_stored":75882250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640284,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68612,"kb_used_data":37420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94303228,"statfs":{"total":96636764160,"available":96566505472,"internally_reserved":0,"allocated":38318080,"data_stored":47112411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36716544,"data_stored":46137452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":75513856,"data_stored":75497605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":37756928,"data_stored":37748812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":78663680,"data_stored":88080530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":25169920,"data_stored":25165862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20979712,"data_stored":20971577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":35,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:51:03.648 INFO:tasks.thrashosds.thrasher:Setting 3.3 to [2, 0] 2024-09-11T06:51:03.648 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.3', '2', '0'] 2024-09-11T06:51:03.648 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.3 2 0 2024-09-11T06:51:03.741 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:03.738+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:03.747 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:03.747 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:51:03.748 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:51:03.748 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:51:03.748 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.748 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4558.0:5 3.2 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-11T06:51:03.748 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:51:03.748 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.655943204000003, 2024-09-11T06:51:03.748 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0062353959999999998, 2024-09-11T06:51:03.748 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.748 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.748 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.748 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:03.748 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4558", 2024-09-11T06:51:03.748 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1478388898", 2024-09-11T06:51:03.749 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 5 2024-09-11T06:51:03.749 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.749 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.749 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.749 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:51:03.749 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.749 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.749 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.749 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:51:03.749 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.749 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.750 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.750 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088422+0000", 2024-09-11T06:51:03.750 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7859999999999998e-06 2024-09-11T06:51:03.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.750 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.750 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.750 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088440+0000", 2024-09-11T06:51:03.750 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8185999999999998e-05 2024-09-11T06:51:03.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.750 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.750 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.750 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088479+0000", 2024-09-11T06:51:03.750 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8248999999999999e-05 2024-09-11T06:51:03.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.751 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.751 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.751 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088497+0000", 2024-09-11T06:51:03.751 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8889000000000002e-05 2024-09-11T06:51:03.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.751 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.751 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.751 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088658+0000", 2024-09-11T06:51:03.752 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000160271 2024-09-11T06:51:03.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.752 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.752 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op must wait for map", 2024-09-11T06:51:03.752 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088668+0000", 2024-09-11T06:51:03.752 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0409e-05 2024-09-11T06:51:03.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.752 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.752 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.752 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092347+0000", 2024-09-11T06:51:03.752 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00367915 2024-09-11T06:51:03.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.752 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.752 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.753 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092541+0000", 2024-09-11T06:51:03.753 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000193833 2024-09-11T06:51:03.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.753 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-11T06:51:03.753 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092732+0000", 2024-09-11T06:51:03.753 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000190772 2024-09-11T06:51:03.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.753 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:03.753 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094136+0000", 2024-09-11T06:51:03.753 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014040809999999999 2024-09-11T06:51:03.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.754 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.754 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:03.754 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094506+0000", 2024-09-11T06:51:03.754 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000369959 2024-09-11T06:51:03.754 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.754 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.754 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.754 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094590+0000", 2024-09-11T06:51:03.754 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4033e-05 2024-09-11T06:51:03.754 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.754 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.754 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.754 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094654+0000", 2024-09-11T06:51:03.754 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3777999999999998e-05 2024-09-11T06:51:03.755 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.755 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.755 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.755 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:261 2.6 2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:51:03.755 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.154411+0000", 2024-09-11T06:51:03.755 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 42.589950406, 2024-09-11T06:51:03.755 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002245024, 2024-09-11T06:51:03.755 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.755 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.755 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:03.755 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:03.756 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:51:03.756 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:51:03.756 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 261 2024-09-11T06:51:03.756 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.756 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.756 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.756 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.756 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154411+0000", 2024-09-11T06:51:03.756 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.756 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.756 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.756 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.756 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154411+0000", 2024-09-11T06:51:03.757 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.757 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.757 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.757 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154412+0000", 2024-09-11T06:51:03.757 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:51:03.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.757 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.757 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.757 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154416+0000", 2024-09-11T06:51:03.757 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3930000000000001e-06 2024-09-11T06:51:03.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.757 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.758 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.758 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154428+0000", 2024-09-11T06:51:03.758 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2099999999999999e-05 2024-09-11T06:51:03.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.758 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.758 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.758 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154438+0000", 2024-09-11T06:51:03.758 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2590000000000006e-06 2024-09-11T06:51:03.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.758 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.758 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.758 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.155643+0000", 2024-09-11T06:51:03.758 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012048689999999999 2024-09-11T06:51:03.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.759 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.759 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.155838+0000", 2024-09-11T06:51:03.759 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019531299999999999 2024-09-11T06:51:03.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.759 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.759 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.156656+0000", 2024-09-11T06:51:03.759 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00081824700000000003 2024-09-11T06:51:03.759 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.759 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.759 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.760 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.760 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:587 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-11T06:51:03.760 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:51:03.760 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.515110548999999, 2024-09-11T06:51:03.760 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0023436289999999999, 2024-09-11T06:51:03.760 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.760 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.760 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.760 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:03.760 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:51:03.760 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:51:03.761 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 587 2024-09-11T06:51:03.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.761 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.761 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.761 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.761 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:51:03.761 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.761 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.761 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.761 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:51:03.762 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.762 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.762 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229252+0000", 2024-09-11T06:51:03.762 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3799999999999999e-06 2024-09-11T06:51:03.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.762 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.762 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229259+0000", 2024-09-11T06:51:03.762 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4760000000000003e-06 2024-09-11T06:51:03.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.763 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.763 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229271+0000", 2024-09-11T06:51:03.763 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2313000000000001e-05 2024-09-11T06:51:03.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.763 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.763 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.763 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229279+0000", 2024-09-11T06:51:03.763 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1610000000000003e-06 2024-09-11T06:51:03.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.763 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.763 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.763 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229344+0000", 2024-09-11T06:51:03.763 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4375000000000006e-05 2024-09-11T06:51:03.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.764 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.764 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229404+0000", 2024-09-11T06:51:03.764 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0517999999999999e-05 2024-09-11T06:51:03.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.764 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:03.764 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229516+0000", 2024-09-11T06:51:03.764 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011129900000000001 2024-09-11T06:51:03.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.765 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:03.765 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231502+0000", 2024-09-11T06:51:03.765 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001986238 2024-09-11T06:51:03.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.765 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.765 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:03.765 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231537+0000", 2024-09-11T06:51:03.765 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4968000000000001e-05 2024-09-11T06:51:03.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.765 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.765 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.765 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231569+0000", 2024-09-11T06:51:03.765 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2459000000000002e-05 2024-09-11T06:51:03.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.766 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.766 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231595+0000", 2024-09-11T06:51:03.766 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5442000000000001e-05 2024-09-11T06:51:03.766 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.766 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.766 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.766 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:15 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:03.766 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.709418+0000", 2024-09-11T06:51:03.766 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.034943396000003, 2024-09-11T06:51:03.767 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027504280000000001, 2024-09-11T06:51:03.767 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.767 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.767 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.767 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:03.767 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:03.767 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:03.767 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 15 2024-09-11T06:51:03.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.767 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.767 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.768 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709418+0000", 2024-09-11T06:51:03.768 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.768 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.768 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.768 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709418+0000", 2024-09-11T06:51:03.768 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.768 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.768 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.768 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709420+0000", 2024-09-11T06:51:03.768 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2299999999999998e-06 2024-09-11T06:51:03.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.769 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.769 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.769 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709432+0000", 2024-09-11T06:51:03.769 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1897e-05 2024-09-11T06:51:03.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.769 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.769 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.769 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709464+0000", 2024-09-11T06:51:03.769 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1819000000000001e-05 2024-09-11T06:51:03.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.769 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.769 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.770 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709479+0000", 2024-09-11T06:51:03.770 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4853e-05 2024-09-11T06:51:03.770 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.770 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.770 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.770 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709624+0000", 2024-09-11T06:51:03.770 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144912 2024-09-11T06:51:03.770 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.770 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.770 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.770 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709773+0000", 2024-09-11T06:51:03.770 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000149529 2024-09-11T06:51:03.770 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.771 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:03.771 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.710055+0000", 2024-09-11T06:51:03.771 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000281692 2024-09-11T06:51:03.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.771 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:03.771 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.711412+0000", 2024-09-11T06:51:03.771 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001356931 2024-09-11T06:51:03.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.771 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:03.771 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.711890+0000", 2024-09-11T06:51:03.771 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00047768700000000002 2024-09-11T06:51:03.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.772 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.772 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.712050+0000", 2024-09-11T06:51:03.772 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016041499999999999 2024-09-11T06:51:03.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.772 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.772 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.712169+0000", 2024-09-11T06:51:03.772 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011846299999999999 2024-09-11T06:51:03.772 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.772 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.772 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.773 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:18 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:03.773 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.713502+0000", 2024-09-11T06:51:03.773 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.030859970000002, 2024-09-11T06:51:03.773 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032323069999999998, 2024-09-11T06:51:03.773 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.773 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.773 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.773 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:03.773 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:03.773 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:03.773 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18 2024-09-11T06:51:03.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.774 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.774 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.774 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713502+0000", 2024-09-11T06:51:03.774 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.774 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.774 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713502+0000", 2024-09-11T06:51:03.774 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.774 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.774 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713503+0000", 2024-09-11T06:51:03.775 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8199999999999999e-06 2024-09-11T06:51:03.775 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.775 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.775 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713513+0000", 2024-09-11T06:51:03.775 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8539999999999992e-06 2024-09-11T06:51:03.775 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.775 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.775 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713540+0000", 2024-09-11T06:51:03.775 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6594999999999999e-05 2024-09-11T06:51:03.775 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.775 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.775 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713553+0000", 2024-09-11T06:51:03.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2843999999999999e-05 2024-09-11T06:51:03.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713686+0000", 2024-09-11T06:51:03.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013294899999999999 2024-09-11T06:51:03.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713937+0000", 2024-09-11T06:51:03.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025110600000000002 2024-09-11T06:51:03.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:03.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.714412+0000", 2024-09-11T06:51:03.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00047562 2024-09-11T06:51:03.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.777 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:03.777 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.715867+0000", 2024-09-11T06:51:03.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001455004 2024-09-11T06:51:03.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.777 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:03.777 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.716569+0000", 2024-09-11T06:51:03.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070159400000000005 2024-09-11T06:51:03.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.777 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.777 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.716660+0000", 2024-09-11T06:51:03.778 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1403000000000006e-05 2024-09-11T06:51:03.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.778 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.778 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.716734+0000", 2024-09-11T06:51:03.778 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3517999999999996e-05 2024-09-11T06:51:03.778 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.778 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.778 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.778 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:23 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:03.778 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.723392+0000", 2024-09-11T06:51:03.778 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.020969374000003, 2024-09-11T06:51:03.779 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027098489999999999, 2024-09-11T06:51:03.779 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.779 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.779 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.779 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:03.779 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:03.779 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:03.779 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23 2024-09-11T06:51:03.779 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.779 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.779 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.779 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.779 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723392+0000", 2024-09-11T06:51:03.779 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.779 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.780 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.780 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.780 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723392+0000", 2024-09-11T06:51:03.780 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.780 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.780 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.780 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.780 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723394+0000", 2024-09-11T06:51:03.780 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0669999999999999e-06 2024-09-11T06:51:03.780 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.780 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.780 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.780 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723406+0000", 2024-09-11T06:51:03.780 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1367e-05 2024-09-11T06:51:03.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723436+0000", 2024-09-11T06:51:03.781 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0874999999999998e-05 2024-09-11T06:51:03.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723451+0000", 2024-09-11T06:51:03.781 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4192999999999999e-05 2024-09-11T06:51:03.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.782 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.782 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723577+0000", 2024-09-11T06:51:03.782 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000126267 2024-09-11T06:51:03.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.782 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.782 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723718+0000", 2024-09-11T06:51:03.782 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000141206 2024-09-11T06:51:03.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.782 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:03.782 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723982+0000", 2024-09-11T06:51:03.782 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026338499999999999 2024-09-11T06:51:03.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.783 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:03.783 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.725377+0000", 2024-09-11T06:51:03.783 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013956999999999999 2024-09-11T06:51:03.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.783 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:03.783 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.725908+0000", 2024-09-11T06:51:03.783 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00053036200000000004 2024-09-11T06:51:03.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.783 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.783 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.725999+0000", 2024-09-11T06:51:03.784 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1016999999999999e-05 2024-09-11T06:51:03.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.784 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.784 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.784 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726102+0000", 2024-09-11T06:51:03.784 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010341 2024-09-11T06:51:03.784 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.784 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.784 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.784 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.784 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:15 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:03.784 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.868921+0000", 2024-09-11T06:51:03.784 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 32.875440333, 2024-09-11T06:51:03.785 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0026879, 2024-09-11T06:51:03.785 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.785 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.785 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.785 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:03.785 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:03.785 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:03.785 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 15 2024-09-11T06:51:03.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.785 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.785 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.785 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.786 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868921+0000", 2024-09-11T06:51:03.786 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.786 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.786 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868921+0000", 2024-09-11T06:51:03.786 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.786 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.786 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868923+0000", 2024-09-11T06:51:03.786 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8959999999999999e-06 2024-09-11T06:51:03.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.786 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.787 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868934+0000", 2024-09-11T06:51:03.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1083e-05 2024-09-11T06:51:03.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.787 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.787 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868965+0000", 2024-09-11T06:51:03.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0833000000000001e-05 2024-09-11T06:51:03.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.787 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.787 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868990+0000", 2024-09-11T06:51:03.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4545999999999998e-05 2024-09-11T06:51:03.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.788 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.788 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.869124+0000", 2024-09-11T06:51:03.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013489599999999999 2024-09-11T06:51:03.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.788 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.788 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.869270+0000", 2024-09-11T06:51:03.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000145976 2024-09-11T06:51:03.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.788 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:03.789 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.869538+0000", 2024-09-11T06:51:03.789 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026780599999999999 2024-09-11T06:51:03.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.789 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.789 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:03.789 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.870691+0000", 2024-09-11T06:51:03.789 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00115277 2024-09-11T06:51:03.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.789 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.789 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:03.789 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.871456+0000", 2024-09-11T06:51:03.789 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00076473800000000005 2024-09-11T06:51:03.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.790 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.790 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.790 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.871543+0000", 2024-09-11T06:51:03.790 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7040999999999996e-05 2024-09-11T06:51:03.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.790 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.790 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.790 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.871609+0000", 2024-09-11T06:51:03.790 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6315000000000005e-05 2024-09-11T06:51:03.790 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.790 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.790 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.791 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.791 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:20 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:03.791 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.877429+0000", 2024-09-11T06:51:03.791 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 32.866932044000002, 2024-09-11T06:51:03.791 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032382629999999999, 2024-09-11T06:51:03.791 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.791 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.791 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.791 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:03.791 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:03.791 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:03.791 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 20 2024-09-11T06:51:03.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.792 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.792 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.792 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877429+0000", 2024-09-11T06:51:03.792 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.792 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.792 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877429+0000", 2024-09-11T06:51:03.792 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.792 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.792 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877431+0000", 2024-09-11T06:51:03.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9429999999999999e-06 2024-09-11T06:51:03.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.793 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.793 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.793 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877444+0000", 2024-09-11T06:51:03.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2396e-05 2024-09-11T06:51:03.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.793 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.793 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.793 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877474+0000", 2024-09-11T06:51:03.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0239999999999998e-05 2024-09-11T06:51:03.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.793 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.794 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.794 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877496+0000", 2024-09-11T06:51:03.794 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1914999999999999e-05 2024-09-11T06:51:03.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.794 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.794 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877616+0000", 2024-09-11T06:51:03.794 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000119814 2024-09-11T06:51:03.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.794 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.794 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877836+0000", 2024-09-11T06:51:03.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022029600000000001 2024-09-11T06:51:03.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.795 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:03.795 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.878482+0000", 2024-09-11T06:51:03.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00064605200000000004 2024-09-11T06:51:03.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.795 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:03.795 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.879804+0000", 2024-09-11T06:51:03.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013216460000000001 2024-09-11T06:51:03.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.795 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:03.795 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.880519+0000", 2024-09-11T06:51:03.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00071547600000000005 2024-09-11T06:51:03.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.880606+0000", 2024-09-11T06:51:03.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6907e-05 2024-09-11T06:51:03.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.880668+0000", 2024-09-11T06:51:03.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1577999999999999e-05 2024-09-11T06:51:03.796 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.796 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.797 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.797 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:23 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:03.797 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.885067+0000", 2024-09-11T06:51:03.797 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 32.859294390000002, 2024-09-11T06:51:03.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028127320000000001, 2024-09-11T06:51:03.797 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.797 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.797 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.797 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:03.797 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:03.797 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:03.797 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23 2024-09-11T06:51:03.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.798 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885067+0000", 2024-09-11T06:51:03.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885067+0000", 2024-09-11T06:51:03.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.799 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.799 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885069+0000", 2024-09-11T06:51:03.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2000000000000001e-06 2024-09-11T06:51:03.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.799 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.799 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885081+0000", 2024-09-11T06:51:03.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1209999999999999e-05 2024-09-11T06:51:03.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.799 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.799 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885111+0000", 2024-09-11T06:51:03.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0318999999999999e-05 2024-09-11T06:51:03.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.800 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.800 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.800 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885132+0000", 2024-09-11T06:51:03.800 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0649000000000002e-05 2024-09-11T06:51:03.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.800 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.800 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.800 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885243+0000", 2024-09-11T06:51:03.800 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000111284 2024-09-11T06:51:03.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.800 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.801 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.801 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885387+0000", 2024-09-11T06:51:03.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144222 2024-09-11T06:51:03.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.801 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:03.801 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885699+0000", 2024-09-11T06:51:03.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00031239800000000003 2024-09-11T06:51:03.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.801 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:03.801 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.886817+0000", 2024-09-11T06:51:03.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001117191 2024-09-11T06:51:03.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:03.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.887684+0000", 2024-09-11T06:51:03.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00086699499999999996 2024-09-11T06:51:03.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.887772+0000", 2024-09-11T06:51:03.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8224000000000001e-05 2024-09-11T06:51:03.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.887880+0000", 2024-09-11T06:51:03.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010804 2024-09-11T06:51:03.803 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.803 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.803 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.803 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "MOSDPGPull(2.0 e70/69 cost 1048576)", 2024-09-11T06:51:03.803 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:35.528435+0000", 2024-09-11T06:51:03.803 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 28.215926654, 2024-09-11T06:51:03.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027390520000000001, 2024-09-11T06:51:03.803 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.803 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.803 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-11T06:51:03.803 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.804 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.804 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528435+0000", 2024-09-11T06:51:03.804 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.804 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.804 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528435+0000", 2024-09-11T06:51:03.804 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.804 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.804 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528437+0000", 2024-09-11T06:51:03.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5959999999999998e-06 2024-09-11T06:51:03.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.805 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.805 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528465+0000", 2024-09-11T06:51:03.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7436e-05 2024-09-11T06:51:03.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.805 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.805 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528499+0000", 2024-09-11T06:51:03.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3599000000000001e-05 2024-09-11T06:51:03.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528515+0000", 2024-09-11T06:51:03.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.677e-05 2024-09-11T06:51:03.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528652+0000", 2024-09-11T06:51:03.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000136546 2024-09-11T06:51:03.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.807 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.531174+0000", 2024-09-11T06:51:03.807 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0025221050000000002 2024-09-11T06:51:03.807 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.807 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.807 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.807 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "MOSDPGPull(2.0 e70/69 cost 1048576)", 2024-09-11T06:51:03.807 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:35.607413+0000", 2024-09-11T06:51:03.807 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 28.136948499999999, 2024-09-11T06:51:03.807 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0021226639999999998, 2024-09-11T06:51:03.807 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.808 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.808 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-11T06:51:03.808 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.808 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.808 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.808 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.607413+0000", 2024-09-11T06:51:03.808 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.808 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.808 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.808 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.607413+0000", 2024-09-11T06:51:03.808 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.809 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.809 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.607415+0000", 2024-09-11T06:51:03.809 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7069999999999999e-06 2024-09-11T06:51:03.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.809 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.809 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.607436+0000", 2024-09-11T06:51:03.809 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1151000000000001e-05 2024-09-11T06:51:03.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.809 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.809 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.607456+0000", 2024-09-11T06:51:03.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0559000000000001e-05 2024-09-11T06:51:03.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.607466+0000", 2024-09-11T06:51:03.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3419999999999997e-06 2024-09-11T06:51:03.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.607646+0000", 2024-09-11T06:51:03.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017984000000000001 2024-09-11T06:51:03.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.811 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.609536+0000", 2024-09-11T06:51:03.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001890065 2024-09-11T06:51:03.811 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.811 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.811 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.811 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4828.0:15 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-11T06:51:03.811 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:36.253730+0000", 2024-09-11T06:51:03.811 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.490631311000001, 2024-09-11T06:51:03.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0025257589999999998, 2024-09-11T06:51:03.811 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.812 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.812 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.812 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:03.812 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4828", 2024-09-11T06:51:03.812 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2410288650", 2024-09-11T06:51:03.812 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 15 2024-09-11T06:51:03.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.812 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.812 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.812 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253730+0000", 2024-09-11T06:51:03.812 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.813 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.813 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253730+0000", 2024-09-11T06:51:03.813 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.813 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.813 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253732+0000", 2024-09-11T06:51:03.813 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2400000000000002e-06 2024-09-11T06:51:03.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.813 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.813 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253744+0000", 2024-09-11T06:51:03.813 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1316e-05 2024-09-11T06:51:03.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.814 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.814 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253771+0000", 2024-09-11T06:51:03.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.724e-05 2024-09-11T06:51:03.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.814 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.814 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253785+0000", 2024-09-11T06:51:03.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3786e-05 2024-09-11T06:51:03.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.814 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253886+0000", 2024-09-11T06:51:03.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000100981 2024-09-11T06:51:03.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.254029+0000", 2024-09-11T06:51:03.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014311900000000001 2024-09-11T06:51:03.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:03.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.254317+0000", 2024-09-11T06:51:03.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00028809300000000001 2024-09-11T06:51:03.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.816 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:03.816 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.255403+0000", 2024-09-11T06:51:03.816 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001085639 2024-09-11T06:51:03.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.816 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:03.816 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.256107+0000", 2024-09-11T06:51:03.816 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070468799999999997 2024-09-11T06:51:03.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.816 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.817 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.256191+0000", 2024-09-11T06:51:03.817 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3714999999999994e-05 2024-09-11T06:51:03.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.817 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.817 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.817 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.256256+0000", 2024-09-11T06:51:03.817 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4942000000000003e-05 2024-09-11T06:51:03.817 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.817 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.817 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.817 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.818 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5313.0:26 2.6 2:7b712441:ns::rbd_object_map.11adc335555f:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:51:03.818 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.260230+0000", 2024-09-11T06:51:03.818 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.484131903000002, 2024-09-11T06:51:03.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002457201, 2024-09-11T06:51:03.818 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.818 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.818 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.818 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:03.818 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5313", 2024-09-11T06:51:03.818 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/624314229", 2024-09-11T06:51:03.818 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:51:03.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.818 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260230+0000", 2024-09-11T06:51:03.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260230+0000", 2024-09-11T06:51:03.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260232+0000", 2024-09-11T06:51:03.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.956e-06 2024-09-11T06:51:03.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260243+0000", 2024-09-11T06:51:03.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1687e-05 2024-09-11T06:51:03.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260269+0000", 2024-09-11T06:51:03.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6211999999999999e-05 2024-09-11T06:51:03.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260284+0000", 2024-09-11T06:51:03.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4053000000000001e-05 2024-09-11T06:51:03.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.821 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.821 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260391+0000", 2024-09-11T06:51:03.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000107278 2024-09-11T06:51:03.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.821 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.821 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260541+0000", 2024-09-11T06:51:03.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015033000000000001 2024-09-11T06:51:03.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.822 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 3", 2024-09-11T06:51:03.822 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260951+0000", 2024-09-11T06:51:03.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040981299999999997 2024-09-11T06:51:03.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.822 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.822 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:03.822 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262399+0000", 2024-09-11T06:51:03.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014475930000000001 2024-09-11T06:51:03.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.822 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.822 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:03.822 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262531+0000", 2024-09-11T06:51:03.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013239 2024-09-11T06:51:03.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.823 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.823 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262623+0000", 2024-09-11T06:51:03.823 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2361000000000006e-05 2024-09-11T06:51:03.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.823 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.823 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262687+0000", 2024-09-11T06:51:03.823 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3528000000000006e-05 2024-09-11T06:51:03.823 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.823 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.824 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.824 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5313.0:31 2.6 2:7b712441:ns::rbd_object_map.11adc335555f:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:51:03.824 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.266495+0000", 2024-09-11T06:51:03.824 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.477866128999999, 2024-09-11T06:51:03.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0026763870000000001, 2024-09-11T06:51:03.824 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.824 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.824 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.825 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:03.825 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5313", 2024-09-11T06:51:03.825 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/624314229", 2024-09-11T06:51:03.825 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 31 2024-09-11T06:51:03.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.825 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266495+0000", 2024-09-11T06:51:03.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266495+0000", 2024-09-11T06:51:03.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.826 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.826 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266498+0000", 2024-09-11T06:51:03.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2369999999999999e-06 2024-09-11T06:51:03.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.826 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.826 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266509+0000", 2024-09-11T06:51:03.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1459e-05 2024-09-11T06:51:03.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.827 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.827 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266540+0000", 2024-09-11T06:51:03.827 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1189999999999998e-05 2024-09-11T06:51:03.827 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.827 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.827 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.827 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266563+0000", 2024-09-11T06:51:03.827 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2529000000000001e-05 2024-09-11T06:51:03.827 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.827 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.827 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.827 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266708+0000", 2024-09-11T06:51:03.827 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014511 2024-09-11T06:51:03.827 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266962+0000", 2024-09-11T06:51:03.828 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000253897 2024-09-11T06:51:03.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 3", 2024-09-11T06:51:03.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.267430+0000", 2024-09-11T06:51:03.828 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00046799899999999998 2024-09-11T06:51:03.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:03.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.268765+0000", 2024-09-11T06:51:03.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013350359999999999 2024-09-11T06:51:03.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:03.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.269019+0000", 2024-09-11T06:51:03.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025397000000000001 2024-09-11T06:51:03.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.269105+0000", 2024-09-11T06:51:03.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5952000000000006e-05 2024-09-11T06:51:03.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.830 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.269172+0000", 2024-09-11T06:51:03.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7008000000000006e-05 2024-09-11T06:51:03.830 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.830 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.830 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.830 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:5 3.4 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:03.830 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.588582+0000", 2024-09-11T06:51:03.830 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.155779976000002, 2024-09-11T06:51:03.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002889857, 2024-09-11T06:51:03.830 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.830 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.831 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.831 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:03.831 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:51:03.831 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:51:03.831 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 5 2024-09-11T06:51:03.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.831 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588582+0000", 2024-09-11T06:51:03.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.832 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588582+0000", 2024-09-11T06:51:03.832 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.832 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588583+0000", 2024-09-11T06:51:03.832 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.223e-06 2024-09-11T06:51:03.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.832 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588588+0000", 2024-09-11T06:51:03.832 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4299999999999997e-06 2024-09-11T06:51:03.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.833 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588606+0000", 2024-09-11T06:51:03.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7652999999999999e-05 2024-09-11T06:51:03.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.833 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588615+0000", 2024-09-11T06:51:03.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4399999999999994e-06 2024-09-11T06:51:03.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.833 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588725+0000", 2024-09-11T06:51:03.834 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010945799999999999 2024-09-11T06:51:03.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.834 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.834 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.589031+0000", 2024-09-11T06:51:03.834 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030590000000000001 2024-09-11T06:51:03.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.834 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:03.834 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.589521+0000", 2024-09-11T06:51:03.834 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00049061299999999999 2024-09-11T06:51:03.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:03.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.590792+0000", 2024-09-11T06:51:03.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012708039999999999 2024-09-11T06:51:03.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:03.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591339+0000", 2024-09-11T06:51:03.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00054710499999999999 2024-09-11T06:51:03.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591412+0000", 2024-09-11T06:51:03.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2811999999999993e-05 2024-09-11T06:51:03.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.836 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591471+0000", 2024-09-11T06:51:03.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9419000000000001e-05 2024-09-11T06:51:03.836 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.836 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.836 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.836 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:8 3.4 3:3f0de396:::rbd_header.1415bf940188:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:03.836 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.596120+0000", 2024-09-11T06:51:03.836 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.148241091999999, 2024-09-11T06:51:03.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027911020000000002, 2024-09-11T06:51:03.837 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.837 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.837 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.837 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:03.837 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:51:03.837 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:51:03.837 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 8 2024-09-11T06:51:03.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.837 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.837 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.837 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.837 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596120+0000", 2024-09-11T06:51:03.837 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596120+0000", 2024-09-11T06:51:03.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596122+0000", 2024-09-11T06:51:03.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.513e-06 2024-09-11T06:51:03.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596130+0000", 2024-09-11T06:51:03.839 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4999999999999999e-06 2024-09-11T06:51:03.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.840 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.840 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596152+0000", 2024-09-11T06:51:03.840 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1846000000000001e-05 2024-09-11T06:51:03.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596163+0000", 2024-09-11T06:51:03.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1029999999999999e-05 2024-09-11T06:51:03.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596242+0000", 2024-09-11T06:51:03.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8275000000000005e-05 2024-09-11T06:51:03.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.842 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.842 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596439+0000", 2024-09-11T06:51:03.842 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000196999 2024-09-11T06:51:03.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.842 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:03.842 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596755+0000", 2024-09-11T06:51:03.842 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00031626100000000003 2024-09-11T06:51:03.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.842 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:03.842 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.598043+0000", 2024-09-11T06:51:03.842 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001288389 2024-09-11T06:51:03.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.843 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.843 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:03.843 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.598774+0000", 2024-09-11T06:51:03.843 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00073080499999999995 2024-09-11T06:51:03.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.843 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.843 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.843 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.598855+0000", 2024-09-11T06:51:03.843 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0615e-05 2024-09-11T06:51:03.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.843 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.843 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.844 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.598912+0000", 2024-09-11T06:51:03.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6869e-05 2024-09-11T06:51:03.844 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.844 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.844 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.844 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:9 3.0 3:00834abc:::rbd_object_map.1415bf940188:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:03.844 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.599161+0000", 2024-09-11T06:51:03.844 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.145200075000002, 2024-09-11T06:51:03.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002826051, 2024-09-11T06:51:03.845 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.845 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.845 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.845 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:03.845 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:51:03.845 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:51:03.845 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 9 2024-09-11T06:51:03.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.845 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.845 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599161+0000", 2024-09-11T06:51:03.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599161+0000", 2024-09-11T06:51:03.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599163+0000", 2024-09-11T06:51:03.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8700000000000001e-06 2024-09-11T06:51:03.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599174+0000", 2024-09-11T06:51:03.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0416e-05 2024-09-11T06:51:03.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.847 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.847 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599202+0000", 2024-09-11T06:51:03.847 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7946e-05 2024-09-11T06:51:03.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.847 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.847 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599216+0000", 2024-09-11T06:51:03.847 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3934000000000001e-05 2024-09-11T06:51:03.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.847 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.847 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599345+0000", 2024-09-11T06:51:03.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012954000000000001 2024-09-11T06:51:03.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.848 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.848 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599540+0000", 2024-09-11T06:51:03.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019516299999999999 2024-09-11T06:51:03.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.848 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:03.848 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599796+0000", 2024-09-11T06:51:03.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025556500000000002 2024-09-11T06:51:03.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.848 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:03.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.601071+0000", 2024-09-11T06:51:03.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001275396 2024-09-11T06:51:03.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.849 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:03.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.601852+0000", 2024-09-11T06:51:03.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000780524 2024-09-11T06:51:03.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.849 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.601925+0000", 2024-09-11T06:51:03.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3574999999999999e-05 2024-09-11T06:51:03.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.850 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.850 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.601988+0000", 2024-09-11T06:51:03.850 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2121999999999999e-05 2024-09-11T06:51:03.850 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.850 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.850 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.850 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5038.0:12 3.4 3:3f0de396:::rbd_header.1415bf940188:head [watch watch cookie 140693532068064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:03.850 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.718024+0000", 2024-09-11T06:51:03.850 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.026337136999999, 2024-09-11T06:51:03.850 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002724583, 2024-09-11T06:51:03.850 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.850 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.851 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.851 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:03.851 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5038", 2024-09-11T06:51:03.851 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/945190275", 2024-09-11T06:51:03.851 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:51:03.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.851 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718024+0000", 2024-09-11T06:51:03.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.852 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.852 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.852 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718024+0000", 2024-09-11T06:51:03.852 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.852 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.852 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.852 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718027+0000", 2024-09-11T06:51:03.852 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1430000000000001e-06 2024-09-11T06:51:03.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.852 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.852 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.852 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718037+0000", 2024-09-11T06:51:03.853 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0253e-05 2024-09-11T06:51:03.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.853 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.853 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.853 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718068+0000", 2024-09-11T06:51:03.853 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1050000000000003e-05 2024-09-11T06:51:03.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.853 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.853 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.853 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718086+0000", 2024-09-11T06:51:03.853 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8629000000000001e-05 2024-09-11T06:51:03.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.853 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.853 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.853 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718206+0000", 2024-09-11T06:51:03.854 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000119358 2024-09-11T06:51:03.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.854 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.854 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718425+0000", 2024-09-11T06:51:03.854 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000218975 2024-09-11T06:51:03.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.854 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:03.854 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718830+0000", 2024-09-11T06:51:03.854 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040538199999999998 2024-09-11T06:51:03.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.854 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:03.854 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.719849+0000", 2024-09-11T06:51:03.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010187589999999999 2024-09-11T06:51:03.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.855 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:03.855 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.720548+0000", 2024-09-11T06:51:03.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000699112 2024-09-11T06:51:03.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.855 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.855 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.720632+0000", 2024-09-11T06:51:03.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4314999999999995e-05 2024-09-11T06:51:03.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.856 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.856 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.720749+0000", 2024-09-11T06:51:03.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116607 2024-09-11T06:51:03.856 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.856 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.856 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.856 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:47 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:03.856 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.553228+0000", 2024-09-11T06:51:03.856 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.191133046999999, 2024-09-11T06:51:03.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040683100000000003, 2024-09-11T06:51:03.856 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.857 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.857 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.857 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:03.857 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:03.857 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:03.857 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 47 2024-09-11T06:51:03.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.857 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.857 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.857 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553228+0000", 2024-09-11T06:51:03.857 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.858 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.858 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553228+0000", 2024-09-11T06:51:03.858 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.858 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.858 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.858 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553231+0000", 2024-09-11T06:51:03.858 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3130000000000001e-06 2024-09-11T06:51:03.858 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.858 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.858 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553242+0000", 2024-09-11T06:51:03.858 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1345999999999999e-05 2024-09-11T06:51:03.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.859 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.859 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.859 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553272+0000", 2024-09-11T06:51:03.859 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0179999999999999e-05 2024-09-11T06:51:03.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.859 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.859 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.859 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553287+0000", 2024-09-11T06:51:03.859 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4283e-05 2024-09-11T06:51:03.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.859 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.859 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.859 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553415+0000", 2024-09-11T06:51:03.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000128491 2024-09-11T06:51:03.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.860 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.860 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553651+0000", 2024-09-11T06:51:03.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000236328 2024-09-11T06:51:03.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.860 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:03.860 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.554356+0000", 2024-09-11T06:51:03.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070436299999999995 2024-09-11T06:51:03.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.861 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:03.861 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.556064+0000", 2024-09-11T06:51:03.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0017084439999999999 2024-09-11T06:51:03.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.861 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:03.861 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.556993+0000", 2024-09-11T06:51:03.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00092912400000000003 2024-09-11T06:51:03.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.861 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.861 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.557173+0000", 2024-09-11T06:51:03.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017940400000000001 2024-09-11T06:51:03.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.862 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.862 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.557297+0000", 2024-09-11T06:51:03.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000124034 2024-09-11T06:51:03.862 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.862 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.862 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.862 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:66 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:03.862 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.589465+0000", 2024-09-11T06:51:03.862 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.154896242, 2024-09-11T06:51:03.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039855489999999997, 2024-09-11T06:51:03.863 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:03.863 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:03.863 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:03.863 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:03.863 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:03.863 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:03.863 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 66 2024-09-11T06:51:03.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.863 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:03.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:03.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589465+0000", 2024-09-11T06:51:03.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.864 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:03.864 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589465+0000", 2024-09-11T06:51:03.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:03.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.864 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:03.864 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589468+0000", 2024-09-11T06:51:03.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3539999999999998e-06 2024-09-11T06:51:03.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.864 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:03.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589480+0000", 2024-09-11T06:51:03.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1946e-05 2024-09-11T06:51:03.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:03.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589512+0000", 2024-09-11T06:51:03.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2712999999999997e-05 2024-09-11T06:51:03.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:03.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589527+0000", 2024-09-11T06:51:03.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4686e-05 2024-09-11T06:51:03.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.866 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:03.866 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589673+0000", 2024-09-11T06:51:03.866 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014630700000000001 2024-09-11T06:51:03.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.866 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:03.866 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589936+0000", 2024-09-11T06:51:03.866 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026234499999999998 2024-09-11T06:51:03.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.866 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:03.867 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.591001+0000", 2024-09-11T06:51:03.867 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001065023 2024-09-11T06:51:03.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.867 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:03.867 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.592757+0000", 2024-09-11T06:51:03.867 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0017560849999999999 2024-09-11T06:51:03.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.867 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:03.867 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.593297+0000", 2024-09-11T06:51:03.867 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00054007600000000001 2024-09-11T06:51:03.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.868 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:03.868 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.593385+0000", 2024-09-11T06:51:03.868 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8327999999999999e-05 2024-09-11T06:51:03.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:03.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:03.868 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:03.868 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.593451+0000", 2024-09-11T06:51:03.868 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5685999999999994e-05 2024-09-11T06:51:03.868 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.868 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.868 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.868 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:03.869 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:03.869 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:03.870 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:51:03.870 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^23729\b' 2024-09-11T06:51:03.870 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^23729\b' 2024-09-11T06:51:03.871 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:51:03.871 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:03.866+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:03.927 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:51:03.927 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -C rbd-nbd 2024-09-11T06:51:03.928 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^ *23729\b' 2024-09-11T06:51:03.928 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^ *23729\b' 2024-09-11T06:51:03.935 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:51:03.936 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 0.5 2024-09-11T06:51:03.971 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:03.970+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:04.030 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:04.030 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:04.030 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:51:04.031 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:04.044 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:51:04.072 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:04.070+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:04.156 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:04.156 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:04.156 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:51:04.156 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:51:04.156 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:04.169 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:51:04.173 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:04.170+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:04.180 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:51:04.273 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:04.270+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:04.291 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:04.291 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:51:04.291 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:51:04.291 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:51:04.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.291 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:20 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:04.291 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.717149+0000", 2024-09-11T06:51:04.291 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.572072345000002, 2024-09-11T06:51:04.292 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002913297, 2024-09-11T06:51:04.292 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:04.292 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:04.292 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:04.292 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:04.292 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:04.292 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:04.292 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 20 2024-09-11T06:51:04.292 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.292 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:04.292 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.292 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:04.292 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717149+0000", 2024-09-11T06:51:04.293 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.293 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.293 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:04.293 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717149+0000", 2024-09-11T06:51:04.293 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.293 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.293 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:04.293 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717152+0000", 2024-09-11T06:51:04.293 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.21e-06 2024-09-11T06:51:04.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.293 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.293 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:04.294 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717164+0000", 2024-09-11T06:51:04.294 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1912999999999999e-05 2024-09-11T06:51:04.294 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.294 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.294 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:04.294 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717191+0000", 2024-09-11T06:51:04.294 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7526e-05 2024-09-11T06:51:04.294 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.294 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.294 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:04.294 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717206+0000", 2024-09-11T06:51:04.294 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5113000000000001e-05 2024-09-11T06:51:04.294 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.294 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.295 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:04.295 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717355+0000", 2024-09-11T06:51:04.295 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014896500000000001 2024-09-11T06:51:04.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.295 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.295 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:04.295 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717607+0000", 2024-09-11T06:51:04.295 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000252017 2024-09-11T06:51:04.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.295 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.295 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:04.295 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.718128+0000", 2024-09-11T06:51:04.295 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00052114499999999998 2024-09-11T06:51:04.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.296 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.296 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:04.296 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.719432+0000", 2024-09-11T06:51:04.296 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013041629999999999 2024-09-11T06:51:04.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.296 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.296 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:04.296 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.719864+0000", 2024-09-11T06:51:04.296 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00043115299999999998 2024-09-11T06:51:04.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.296 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.296 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:04.296 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.719949+0000", 2024-09-11T06:51:04.296 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4862000000000002e-05 2024-09-11T06:51:04.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.297 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.297 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:04.297 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.720063+0000", 2024-09-11T06:51:04.297 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011423 2024-09-11T06:51:04.297 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.297 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:04.297 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.297 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.297 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:26 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 140250278007648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:04.297 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:51:04.297 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.559589209999999, 2024-09-11T06:51:04.297 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034645140000000001, 2024-09-11T06:51:04.298 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:04.298 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:04.298 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:04.298 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:04.298 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:04.298 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:04.298 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:51:04.298 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.298 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:04.298 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.298 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:04.298 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:51:04.298 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.298 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.299 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.299 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:04.299 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:51:04.299 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.299 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.299 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.299 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:04.299 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729634+0000", 2024-09-11T06:51:04.299 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0470000000000001e-06 2024-09-11T06:51:04.299 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.299 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.299 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:04.299 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729640+0000", 2024-09-11T06:51:04.299 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8860000000000001e-06 2024-09-11T06:51:04.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.300 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.300 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:04.300 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729654+0000", 2024-09-11T06:51:04.300 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:51:04.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.300 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.300 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:04.300 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729664+0000", 2024-09-11T06:51:04.300 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.594e-06 2024-09-11T06:51:04.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.300 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.300 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:04.301 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730422+0000", 2024-09-11T06:51:04.301 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000758648 2024-09-11T06:51:04.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.301 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.301 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:04.301 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730571+0000", 2024-09-11T06:51:04.301 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000148932 2024-09-11T06:51:04.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.301 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.301 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:04.301 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730936+0000", 2024-09-11T06:51:04.301 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036510900000000002 2024-09-11T06:51:04.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.301 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.302 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:04.302 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732159+0000", 2024-09-11T06:51:04.302 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001222302 2024-09-11T06:51:04.302 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.302 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.302 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:04.302 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732763+0000", 2024-09-11T06:51:04.302 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00060421599999999998 2024-09-11T06:51:04.302 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.302 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.302 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:04.302 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732916+0000", 2024-09-11T06:51:04.302 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015264 2024-09-11T06:51:04.302 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.303 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.303 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:04.303 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733097+0000", 2024-09-11T06:51:04.303 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000181504 2024-09-11T06:51:04.303 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.303 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:04.303 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.303 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.303 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.303 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:27 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:04.303 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:51:04.303 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.558615308, 2024-09-11T06:51:04.303 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003072852, 2024-09-11T06:51:04.304 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:04.304 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:04.304 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:04.304 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:04.304 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:04.304 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:04.304 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27 2024-09-11T06:51:04.304 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.304 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:04.304 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.304 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:04.304 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:51:04.304 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.304 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.305 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.305 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:04.305 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:51:04.305 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.305 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.305 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:04.305 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730608+0000", 2024-09-11T06:51:04.305 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8369999999999999e-06 2024-09-11T06:51:04.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.305 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.305 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:04.305 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730617+0000", 2024-09-11T06:51:04.306 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1859999999999992e-06 2024-09-11T06:51:04.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.306 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:04.306 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730636+0000", 2024-09-11T06:51:04.306 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9296e-05 2024-09-11T06:51:04.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.306 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:04.306 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730674+0000", 2024-09-11T06:51:04.306 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8609000000000003e-05 2024-09-11T06:51:04.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.306 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:04.307 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731496+0000", 2024-09-11T06:51:04.307 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00082112299999999999 2024-09-11T06:51:04.307 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.307 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.307 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:04.307 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731722+0000", 2024-09-11T06:51:04.307 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022651400000000001 2024-09-11T06:51:04.307 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.307 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.307 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:04.307 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733215+0000", 2024-09-11T06:51:04.307 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001493257 2024-09-11T06:51:04.307 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.308 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.308 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:04.308 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733452+0000", 2024-09-11T06:51:04.308 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023631600000000001 2024-09-11T06:51:04.308 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.308 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.308 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:04.308 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733679+0000", 2024-09-11T06:51:04.308 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022771400000000001 2024-09-11T06:51:04.308 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.308 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:04.308 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.308 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.308 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.309 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:73 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:04.309 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:51:04.309 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.558531023999997, 2024-09-11T06:51:04.309 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033454439999999999, 2024-09-11T06:51:04.309 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:04.309 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:04.309 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:04.309 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:04.309 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:51:04.309 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:51:04.309 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73 2024-09-11T06:51:04.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.309 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:04.309 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.310 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:04.310 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:51:04.310 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.310 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.310 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:04.310 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:51:04.310 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.310 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.310 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:04.310 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730694+0000", 2024-09-11T06:51:04.310 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5099999999999999e-06 2024-09-11T06:51:04.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.311 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.311 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:04.311 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730708+0000", 2024-09-11T06:51:04.311 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3237e-05 2024-09-11T06:51:04.311 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.311 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.311 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:04.311 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730738+0000", 2024-09-11T06:51:04.311 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0688999999999997e-05 2024-09-11T06:51:04.311 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.311 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.311 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:04.311 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730759+0000", 2024-09-11T06:51:04.311 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0818999999999998e-05 2024-09-11T06:51:04.312 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.312 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.312 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:04.312 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731841+0000", 2024-09-11T06:51:04.312 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010821819999999999 2024-09-11T06:51:04.312 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.312 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.312 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:04.312 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732092+0000", 2024-09-11T06:51:04.312 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025086299999999999 2024-09-11T06:51:04.312 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.312 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.312 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:04.312 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733730+0000", 2024-09-11T06:51:04.313 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0016374600000000001 2024-09-11T06:51:04.313 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.313 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.313 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:04.313 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733866+0000", 2024-09-11T06:51:04.313 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000136666 2024-09-11T06:51:04.313 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.313 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.313 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:04.313 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.734036+0000", 2024-09-11T06:51:04.313 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000170018 2024-09-11T06:51:04.313 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.313 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:04.313 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.314 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.314 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:18 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:04.314 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:51:04.314 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.415113835, 2024-09-11T06:51:04.314 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0031646579999999999, 2024-09-11T06:51:04.314 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:04.314 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:04.314 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:04.314 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:04.314 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:04.314 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:04.314 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18 2024-09-11T06:51:04.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.314 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:04.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.315 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:04.315 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:51:04.315 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.315 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:04.315 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:51:04.315 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.315 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:04.315 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874109+0000", 2024-09-11T06:51:04.316 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.426e-06 2024-09-11T06:51:04.316 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.316 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.316 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:04.316 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874118+0000", 2024-09-11T06:51:04.316 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6929999999999996e-06 2024-09-11T06:51:04.316 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.316 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.316 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:04.316 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874134+0000", 2024-09-11T06:51:04.316 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5503e-05 2024-09-11T06:51:04.316 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.316 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.317 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:04.317 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874144+0000", 2024-09-11T06:51:04.317 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0207000000000001e-05 2024-09-11T06:51:04.317 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.317 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.317 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:04.317 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874545+0000", 2024-09-11T06:51:04.317 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040120800000000001 2024-09-11T06:51:04.317 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.317 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.317 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:04.317 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874686+0000", 2024-09-11T06:51:04.317 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014128599999999999 2024-09-11T06:51:04.317 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.318 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.318 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:04.318 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875023+0000", 2024-09-11T06:51:04.318 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033706600000000001 2024-09-11T06:51:04.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.318 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.318 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:04.318 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.876217+0000", 2024-09-11T06:51:04.318 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011938999999999999 2024-09-11T06:51:04.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.318 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.318 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:04.318 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877042+0000", 2024-09-11T06:51:04.318 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00082499899999999998 2024-09-11T06:51:04.319 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.319 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.319 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:04.319 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877128+0000", 2024-09-11T06:51:04.319 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5903999999999996e-05 2024-09-11T06:51:04.319 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.319 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.319 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:04.319 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877273+0000", 2024-09-11T06:51:04.319 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144466 2024-09-11T06:51:04.319 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.319 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:04.319 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.320 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:24 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:04.320 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:51:04.320 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.400608128999998, 2024-09-11T06:51:04.320 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002980271, 2024-09-11T06:51:04.320 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:04.320 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:04.320 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:04.320 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:04.320 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:04.320 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:04.320 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 24 2024-09-11T06:51:04.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.321 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:04.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.321 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:04.321 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:51:04.321 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.321 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:04.321 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:51:04.321 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.322 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:04.322 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888616+0000", 2024-09-11T06:51:04.322 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0030000000000001e-06 2024-09-11T06:51:04.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.322 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.322 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:04.322 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888626+0000", 2024-09-11T06:51:04.322 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0502999999999999e-05 2024-09-11T06:51:04.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.322 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.322 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:04.322 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888655+0000", 2024-09-11T06:51:04.322 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8419999999999999e-05 2024-09-11T06:51:04.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.323 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.323 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:04.323 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888688+0000", 2024-09-11T06:51:04.323 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3212e-05 2024-09-11T06:51:04.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.323 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.323 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:04.323 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888829+0000", 2024-09-11T06:51:04.323 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014114299999999999 2024-09-11T06:51:04.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.323 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.323 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:04.323 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.889082+0000", 2024-09-11T06:51:04.324 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025279499999999999 2024-09-11T06:51:04.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.324 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:04.324 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.889577+0000", 2024-09-11T06:51:04.324 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00049536299999999997 2024-09-11T06:51:04.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.324 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:04.324 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.890811+0000", 2024-09-11T06:51:04.324 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001233858 2024-09-11T06:51:04.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.324 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:04.325 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891368+0000", 2024-09-11T06:51:04.325 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00055677400000000001 2024-09-11T06:51:04.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.325 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:04.325 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891456+0000", 2024-09-11T06:51:04.325 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8246999999999997e-05 2024-09-11T06:51:04.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.325 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:04.325 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891594+0000", 2024-09-11T06:51:04.325 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013795300000000001 2024-09-11T06:51:04.325 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.325 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:04.326 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.326 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:26 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 139740787512064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:04.326 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:51:04.326 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.396905484999998, 2024-09-11T06:51:04.326 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003613733, 2024-09-11T06:51:04.326 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:04.326 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:04.326 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:04.326 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:04.326 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:04.326 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:04.327 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:51:04.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.327 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:04.327 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.327 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:04.327 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:51:04.327 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.327 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.327 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:04.327 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:51:04.327 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.327 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.328 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:04.328 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892318+0000", 2024-09-11T06:51:04.328 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.623e-06 2024-09-11T06:51:04.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.328 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.328 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:04.328 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892325+0000", 2024-09-11T06:51:04.328 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:51:04.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.328 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.328 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:04.328 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892346+0000", 2024-09-11T06:51:04.328 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1246e-05 2024-09-11T06:51:04.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.329 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.329 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:04.329 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892362+0000", 2024-09-11T06:51:04.329 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5409000000000001e-05 2024-09-11T06:51:04.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.329 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.329 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:04.329 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892883+0000", 2024-09-11T06:51:04.329 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00052087599999999998 2024-09-11T06:51:04.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.329 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.329 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:04.329 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893123+0000", 2024-09-11T06:51:04.330 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000240769 2024-09-11T06:51:04.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.330 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.330 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:04.330 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893526+0000", 2024-09-11T06:51:04.330 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040300200000000002 2024-09-11T06:51:04.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.330 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.330 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:04.330 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894820+0000", 2024-09-11T06:51:04.330 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012934159999999999 2024-09-11T06:51:04.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.331 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.331 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:04.331 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895608+0000", 2024-09-11T06:51:04.331 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00078788100000000004 2024-09-11T06:51:04.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.331 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.331 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:04.331 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895724+0000", 2024-09-11T06:51:04.331 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011595999999999999 2024-09-11T06:51:04.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.331 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.331 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:04.331 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895930+0000", 2024-09-11T06:51:04.332 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000206391 2024-09-11T06:51:04.332 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.332 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:04.332 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.332 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.332 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.332 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:133 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:04.332 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:51:04.332 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.396107467999997, 2024-09-11T06:51:04.332 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035017360000000001, 2024-09-11T06:51:04.332 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:04.332 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:04.332 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:04.332 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:04.333 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:51:04.333 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:51:04.333 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 133 2024-09-11T06:51:04.333 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.333 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:04.333 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.333 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:04.333 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:51:04.333 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.333 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.333 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.333 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:04.333 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:51:04.334 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.334 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.334 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.334 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:04.334 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893116+0000", 2024-09-11T06:51:04.334 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1629999999999999e-06 2024-09-11T06:51:04.334 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.334 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.334 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:04.334 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893128+0000", 2024-09-11T06:51:04.334 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.11e-05 2024-09-11T06:51:04.335 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.335 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.335 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:04.335 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893156+0000", 2024-09-11T06:51:04.335 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8266e-05 2024-09-11T06:51:04.335 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.335 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.335 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:04.335 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893171+0000", 2024-09-11T06:51:04.335 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5426000000000001e-05 2024-09-11T06:51:04.335 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.335 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.335 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:04.336 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894561+0000", 2024-09-11T06:51:04.336 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013899959999999999 2024-09-11T06:51:04.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.336 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.336 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:04.336 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894781+0000", 2024-09-11T06:51:04.336 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021987100000000001 2024-09-11T06:51:04.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.336 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.336 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:04.336 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896349+0000", 2024-09-11T06:51:04.336 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015681779999999999 2024-09-11T06:51:04.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.337 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.337 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:04.337 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896458+0000", 2024-09-11T06:51:04.337 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000108187 2024-09-11T06:51:04.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.337 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.337 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:04.337 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896616+0000", 2024-09-11T06:51:04.337 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015854899999999999 2024-09-11T06:51:04.337 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.337 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:04.337 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.337 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.338 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:27 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:04.338 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:51:04.338 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.396096088, 2024-09-11T06:51:04.338 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003151989, 2024-09-11T06:51:04.338 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:04.338 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:04.338 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:04.338 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:04.338 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:04.338 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:04.338 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27 2024-09-11T06:51:04.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.338 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:04.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.339 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:04.339 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:51:04.339 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.339 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:04.339 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:51:04.339 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.339 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:04.339 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893128+0000", 2024-09-11T06:51:04.339 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.013e-06 2024-09-11T06:51:04.340 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.340 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.340 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:04.340 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893136+0000", 2024-09-11T06:51:04.340 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4190000000000002e-06 2024-09-11T06:51:04.340 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.340 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.340 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:04.340 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893158+0000", 2024-09-11T06:51:04.340 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1506999999999999e-05 2024-09-11T06:51:04.340 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.340 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.341 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:04.341 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893170+0000", 2024-09-11T06:51:04.341 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2741999999999999e-05 2024-09-11T06:51:04.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.341 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.341 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:04.341 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894255+0000", 2024-09-11T06:51:04.341 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001084453 2024-09-11T06:51:04.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.341 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.341 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:04.341 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894461+0000", 2024-09-11T06:51:04.341 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00020586700000000001 2024-09-11T06:51:04.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.342 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.342 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:04.342 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896033+0000", 2024-09-11T06:51:04.342 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001572092 2024-09-11T06:51:04.342 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.342 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.342 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:04.342 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896149+0000", 2024-09-11T06:51:04.342 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116636 2024-09-11T06:51:04.342 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.342 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.342 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:04.342 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896278+0000", 2024-09-11T06:51:04.342 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012826 2024-09-11T06:51:04.343 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.343 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:04.343 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.343 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.343 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:176 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch reconnect cookie 139735284585312 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:04.343 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:51:04.343 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.364729269000001, 2024-09-11T06:51:04.343 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033620960000000002, 2024-09-11T06:51:04.343 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:04.343 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:04.343 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:04.343 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:04.344 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:51:04.344 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:51:04.344 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 176 2024-09-11T06:51:04.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.344 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:04.344 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.344 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:04.344 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:51:04.344 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.344 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.344 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:04.344 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:51:04.345 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.345 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.345 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:04.345 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924494+0000", 2024-09-11T06:51:04.345 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.967e-06 2024-09-11T06:51:04.345 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.345 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:04.345 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924501+0000", 2024-09-11T06:51:04.345 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2659999999999998e-06 2024-09-11T06:51:04.345 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.345 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:04.346 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924520+0000", 2024-09-11T06:51:04.346 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9038999999999998e-05 2024-09-11T06:51:04.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.346 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.346 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:04.346 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924535+0000", 2024-09-11T06:51:04.346 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5359999999999999e-05 2024-09-11T06:51:04.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.346 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.346 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:04.347 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.925544+0000", 2024-09-11T06:51:04.347 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010085109999999999 2024-09-11T06:51:04.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.347 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.347 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:04.347 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.925671+0000", 2024-09-11T06:51:04.347 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012756599999999999 2024-09-11T06:51:04.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.347 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.347 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:04.347 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.927855+0000", 2024-09-11T06:51:04.347 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0021833870000000002 2024-09-11T06:51:04.347 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.348 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:04.348 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.348 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.348 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.348 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4828.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch watch cookie 140587969811264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-11T06:51:04.348 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:51:04.348 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 28.043026310999998, 2024-09-11T06:51:04.348 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032778859999999998, 2024-09-11T06:51:04.348 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:04.348 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:04.348 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:04.348 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:04.348 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4828", 2024-09-11T06:51:04.348 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2410288650", 2024-09-11T06:51:04.349 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:51:04.349 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.349 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:04.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.349 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:04.349 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:51:04.349 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.349 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.349 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:04.349 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:51:04.349 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.350 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:04.350 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246198+0000", 2024-09-11T06:51:04.350 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.667e-06 2024-09-11T06:51:04.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.350 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:04.350 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246209+0000", 2024-09-11T06:51:04.350 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0409e-05 2024-09-11T06:51:04.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.350 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:04.350 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246241+0000", 2024-09-11T06:51:04.351 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2592999999999998e-05 2024-09-11T06:51:04.351 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.351 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.351 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:04.351 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246260+0000", 2024-09-11T06:51:04.351 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9343000000000001e-05 2024-09-11T06:51:04.351 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.351 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.351 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:04.351 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246466+0000", 2024-09-11T06:51:04.351 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000205701 2024-09-11T06:51:04.351 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.351 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.351 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:04.352 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246713+0000", 2024-09-11T06:51:04.352 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024646999999999999 2024-09-11T06:51:04.352 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.352 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.352 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:04.352 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.247133+0000", 2024-09-11T06:51:04.352 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000420536 2024-09-11T06:51:04.352 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.352 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.352 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:04.352 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.248541+0000", 2024-09-11T06:51:04.352 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001407801 2024-09-11T06:51:04.352 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.353 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:04.353 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249241+0000", 2024-09-11T06:51:04.353 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070043500000000001 2024-09-11T06:51:04.353 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.353 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:04.353 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249326+0000", 2024-09-11T06:51:04.353 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5037999999999995e-05 2024-09-11T06:51:04.353 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.353 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:04.353 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249473+0000", 2024-09-11T06:51:04.353 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014689299999999999 2024-09-11T06:51:04.354 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.354 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:04.354 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.354 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.354 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5313.0:25 2.1 2:967e1786:ns::rbd_object_map.11adc335555f.0000000000000004:head [writefull 0~38] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:51:04.354 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.256788+0000", 2024-09-11T06:51:04.354 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 25.032433311999998, 2024-09-11T06:51:04.354 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0030446449999999999, 2024-09-11T06:51:04.354 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:04.354 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:04.354 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:04.354 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:04.354 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5313", 2024-09-11T06:51:04.355 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/624314229", 2024-09-11T06:51:04.355 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 25 2024-09-11T06:51:04.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.355 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:04.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.355 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:04.355 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256788+0000", 2024-09-11T06:51:04.355 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.355 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:04.355 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256788+0000", 2024-09-11T06:51:04.355 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.356 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:04.356 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256791+0000", 2024-09-11T06:51:04.356 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5770000000000001e-06 2024-09-11T06:51:04.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.356 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:04.356 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256802+0000", 2024-09-11T06:51:04.356 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0742e-05 2024-09-11T06:51:04.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.356 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:04.356 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256823+0000", 2024-09-11T06:51:04.356 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0999999999999999e-05 2024-09-11T06:51:04.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.357 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:04.357 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256841+0000", 2024-09-11T06:51:04.357 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8099000000000001e-05 2024-09-11T06:51:04.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.357 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:04.357 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256988+0000", 2024-09-11T06:51:04.357 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014756400000000001 2024-09-11T06:51:04.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.357 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:04.357 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.257234+0000", 2024-09-11T06:51:04.358 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024524399999999999 2024-09-11T06:51:04.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.358 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:04.358 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.257446+0000", 2024-09-11T06:51:04.358 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000212241 2024-09-11T06:51:04.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.358 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:04.358 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259292+0000", 2024-09-11T06:51:04.358 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001845937 2024-09-11T06:51:04.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.358 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:04.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259681+0000", 2024-09-11T06:51:04.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00038940699999999998 2024-09-11T06:51:04.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.359 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:04.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259770+0000", 2024-09-11T06:51:04.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8385000000000002e-05 2024-09-11T06:51:04.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.359 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:04.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259833+0000", 2024-09-11T06:51:04.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3448999999999995e-05 2024-09-11T06:51:04.359 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.360 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:04.360 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.360 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5328.0:14 2.1 2:967e1786:ns::rbd_object_map.11adc335555f.0000000000000004:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:51:04.360 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.396257+0000", 2024-09-11T06:51:04.360 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.892964863, 2024-09-11T06:51:04.360 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032125370000000001, 2024-09-11T06:51:04.360 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:04.360 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:04.360 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:04.360 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:04.360 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5328", 2024-09-11T06:51:04.360 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2386493698", 2024-09-11T06:51:04.361 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 14 2024-09-11T06:51:04.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.361 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:04.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.361 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:04.361 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396257+0000", 2024-09-11T06:51:04.361 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.361 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:04.361 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396257+0000", 2024-09-11T06:51:04.361 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.362 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.362 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:04.362 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396259+0000", 2024-09-11T06:51:04.362 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4830000000000002e-06 2024-09-11T06:51:04.362 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.362 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.362 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:04.362 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396271+0000", 2024-09-11T06:51:04.362 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.115e-05 2024-09-11T06:51:04.362 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.362 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.362 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:04.362 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396292+0000", 2024-09-11T06:51:04.362 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1761999999999999e-05 2024-09-11T06:51:04.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.363 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.363 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:04.363 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396314+0000", 2024-09-11T06:51:04.363 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1982999999999999e-05 2024-09-11T06:51:04.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.363 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.363 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:04.363 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396409+0000", 2024-09-11T06:51:04.364 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4250000000000001e-05 2024-09-11T06:51:04.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.364 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:04.364 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396464+0000", 2024-09-11T06:51:04.364 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5884000000000002e-05 2024-09-11T06:51:04.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.364 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:04.364 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.399469+0000", 2024-09-11T06:51:04.364 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0030050250000000001 2024-09-11T06:51:04.364 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.364 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:04.365 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.365 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:6 3.5 3:b2cc3336:::rbd_id.testrbdnbd21607:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:04.365 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.591899+0000", 2024-09-11T06:51:04.365 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.697322362000001, 2024-09-11T06:51:04.365 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032382999999999999, 2024-09-11T06:51:04.365 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:04.365 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:04.365 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:04.365 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:04.365 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:51:04.365 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:51:04.365 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 6 2024-09-11T06:51:04.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.366 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:04.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.366 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:04.366 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591899+0000", 2024-09-11T06:51:04.366 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.366 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:04.366 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591899+0000", 2024-09-11T06:51:04.366 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.367 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:04.367 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591902+0000", 2024-09-11T06:51:04.367 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0770000000000002e-06 2024-09-11T06:51:04.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.367 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.367 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:04.367 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591913+0000", 2024-09-11T06:51:04.367 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1182999999999999e-05 2024-09-11T06:51:04.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.367 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.367 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:04.367 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591940+0000", 2024-09-11T06:51:04.367 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7233000000000002e-05 2024-09-11T06:51:04.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.368 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.368 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:04.368 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591954+0000", 2024-09-11T06:51:04.368 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4503e-05 2024-09-11T06:51:04.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.368 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.368 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:04.368 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.592071+0000", 2024-09-11T06:51:04.368 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116731 2024-09-11T06:51:04.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.368 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.368 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:04.368 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.592274+0000", 2024-09-11T06:51:04.369 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00020320900000000001 2024-09-11T06:51:04.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.369 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 3", 2024-09-11T06:51:04.369 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.592576+0000", 2024-09-11T06:51:04.369 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030179000000000002 2024-09-11T06:51:04.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.369 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:04.369 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.594072+0000", 2024-09-11T06:51:04.369 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001495496 2024-09-11T06:51:04.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.370 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:04.370 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.594945+0000", 2024-09-11T06:51:04.370 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00087379799999999996 2024-09-11T06:51:04.370 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.370 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:04.370 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.595023+0000", 2024-09-11T06:51:04.370 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7751999999999997e-05 2024-09-11T06:51:04.370 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.370 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:04.370 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.595138+0000", 2024-09-11T06:51:04.370 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000114528 2024-09-11T06:51:04.370 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.371 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:04.371 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.371 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:44 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:04.371 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.540846+0000", 2024-09-11T06:51:04.371 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.74837623, 2024-09-11T06:51:04.371 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032138639999999999, 2024-09-11T06:51:04.371 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:04.371 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:04.371 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:04.371 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:04.371 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 44 2024-09-11T06:51:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540846+0000", 2024-09-11T06:51:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540846+0000", 2024-09-11T06:51:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:04.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540848+0000", 2024-09-11T06:51:04.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5469999999999999e-06 2024-09-11T06:51:04.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:04.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540863+0000", 2024-09-11T06:51:04.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4613000000000001e-05 2024-09-11T06:51:04.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:04.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540895+0000", 2024-09-11T06:51:04.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2153000000000002e-05 2024-09-11T06:51:04.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:04.374 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540912+0000", 2024-09-11T06:51:04.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7405999999999999e-05 2024-09-11T06:51:04.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:04.374 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.541111+0000", 2024-09-11T06:51:04.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019849599999999999 2024-09-11T06:51:04.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:04.375 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.541269+0000", 2024-09-11T06:51:04.375 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015792399999999999 2024-09-11T06:51:04.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.375 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:04.375 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.541853+0000", 2024-09-11T06:51:04.375 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00058443799999999995 2024-09-11T06:51:04.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.375 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:04.375 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.543133+0000", 2024-09-11T06:51:04.375 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012797850000000001 2024-09-11T06:51:04.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.376 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:04.376 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.543905+0000", 2024-09-11T06:51:04.376 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000771655 2024-09-11T06:51:04.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.376 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:04.376 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.543993+0000", 2024-09-11T06:51:04.376 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8168e-05 2024-09-11T06:51:04.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.377 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:04.377 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.544059+0000", 2024-09-11T06:51:04.377 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6679000000000003e-05 2024-09-11T06:51:04.377 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.377 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:04.377 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.377 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.377 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.377 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5553.0:24 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 139650593197904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:04.377 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.566567+0000", 2024-09-11T06:51:04.377 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.72265496, 2024-09-11T06:51:04.377 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034350790000000002, 2024-09-11T06:51:04.377 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:04.378 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:04.378 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:04.378 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:04.378 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5553", 2024-09-11T06:51:04.378 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3590511604", 2024-09-11T06:51:04.378 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 24 2024-09-11T06:51:04.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.378 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:04.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.378 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:04.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566567+0000", 2024-09-11T06:51:04.378 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.379 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:04.379 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566567+0000", 2024-09-11T06:51:04.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.379 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:04.379 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566569+0000", 2024-09-11T06:51:04.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.973e-06 2024-09-11T06:51:04.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.379 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:04.379 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566578+0000", 2024-09-11T06:51:04.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8170000000000005e-06 2024-09-11T06:51:04.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:04.380 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566602+0000", 2024-09-11T06:51:04.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4596e-05 2024-09-11T06:51:04.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:04.380 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566619+0000", 2024-09-11T06:51:04.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7099999999999999e-05 2024-09-11T06:51:04.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:04.380 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.567321+0000", 2024-09-11T06:51:04.381 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070168200000000002 2024-09-11T06:51:04.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.381 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:04.381 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.567454+0000", 2024-09-11T06:51:04.381 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013286400000000001 2024-09-11T06:51:04.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.381 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:04.381 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.567674+0000", 2024-09-11T06:51:04.381 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000219679 2024-09-11T06:51:04.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.381 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:04.382 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.569014+0000", 2024-09-11T06:51:04.382 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013404129999999999 2024-09-11T06:51:04.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.382 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:04.382 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.569634+0000", 2024-09-11T06:51:04.382 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00062029200000000004 2024-09-11T06:51:04.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.382 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:04.382 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.569805+0000", 2024-09-11T06:51:04.382 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000170366 2024-09-11T06:51:04.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.383 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:04.383 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.570002+0000", 2024-09-11T06:51:04.383 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019729700000000001 2024-09-11T06:51:04.383 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.383 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:04.383 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.383 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:82 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch reconnect cookie 139888024456048 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:04.383 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.690185+0000", 2024-09-11T06:51:04.383 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.599036957999999, 2024-09-11T06:51:04.383 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0045884799999999998, 2024-09-11T06:51:04.383 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:04.383 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:04.384 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:04.384 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:04.384 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:04.384 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:04.384 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 82 2024-09-11T06:51:04.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.384 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:04.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.384 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:04.384 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690185+0000", 2024-09-11T06:51:04.384 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.384 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:04.385 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690185+0000", 2024-09-11T06:51:04.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.385 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:04.385 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690186+0000", 2024-09-11T06:51:04.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8400000000000002e-07 2024-09-11T06:51:04.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.385 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:04.385 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690190+0000", 2024-09-11T06:51:04.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5499999999999996e-06 2024-09-11T06:51:04.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.386 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:04.386 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690203+0000", 2024-09-11T06:51:04.386 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2228999999999999e-05 2024-09-11T06:51:04.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.386 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:04.386 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690228+0000", 2024-09-11T06:51:04.386 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5587000000000001e-05 2024-09-11T06:51:04.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.386 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:04.386 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.691298+0000", 2024-09-11T06:51:04.386 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001070218 2024-09-11T06:51:04.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.691511+0000", 2024-09-11T06:51:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021301299999999999 2024-09-11T06:51:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.694773+0000", 2024-09-11T06:51:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003261899 2024-09-11T06:51:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.388 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:83 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify cookie 139887621709056] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:04.388 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.690242+0000", 2024-09-11T06:51:04.388 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.598979785999999, 2024-09-11T06:51:04.388 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003040067, 2024-09-11T06:51:04.388 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:04.388 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:04.388 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:04.388 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:04.388 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:04.388 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:04.388 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 83 2024-09-11T06:51:04.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.388 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:04.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.389 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:04.389 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690242+0000", 2024-09-11T06:51:04.389 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.389 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:04.389 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690242+0000", 2024-09-11T06:51:04.389 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.389 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:04.389 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690243+0000", 2024-09-11T06:51:04.389 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0640000000000001e-06 2024-09-11T06:51:04.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.390 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:04.390 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690247+0000", 2024-09-11T06:51:04.390 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3760000000000001e-06 2024-09-11T06:51:04.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.390 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:04.390 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690258+0000", 2024-09-11T06:51:04.390 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0874e-05 2024-09-11T06:51:04.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.390 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:04.390 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690266+0000", 2024-09-11T06:51:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5429999999999996e-06 2024-09-11T06:51:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.692564+0000", 2024-09-11T06:51:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0022977010000000001 2024-09-11T06:51:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.692882+0000", 2024-09-11T06:51:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00031862699999999998 2024-09-11T06:51:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:04.392 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693282+0000", 2024-09-11T06:51:04.392 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000399882 2024-09-11T06:51:04.392 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.392 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:04.392 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.392 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:84 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify cookie 139887823033168] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:04.392 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:51:04.392 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.598945182, 2024-09-11T06:51:04.392 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035622620000000001, 2024-09-11T06:51:04.392 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:04.392 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:04.392 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:04.393 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:04.393 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:04.393 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:04.393 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 84 2024-09-11T06:51:04.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.393 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:04.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.393 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:04.393 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:51:04.393 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.393 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:51:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:51:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5300000000000003e-07 2024-09-11T06:51:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690281+0000", 2024-09-11T06:51:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9330000000000003e-06 2024-09-11T06:51:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690291+0000", 2024-09-11T06:51:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7359999999999996e-06 2024-09-11T06:51:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690298+0000", 2024-09-11T06:51:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3699999999999997e-06 2024-09-11T06:51:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693380+0000", 2024-09-11T06:51:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0030811060000000001 2024-09-11T06:51:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.396 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:04.396 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693599+0000", 2024-09-11T06:51:04.396 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021947199999999999 2024-09-11T06:51:04.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.396 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:04.396 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693839+0000", 2024-09-11T06:51:04.396 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023989200000000001 2024-09-11T06:51:04.396 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.396 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:04.396 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.396 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5098.0:25 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 140617967560800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.697727+0000", 2024-09-11T06:51:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.591494605999999, 2024-09-11T06:51:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035341330000000001, 2024-09-11T06:51:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5098", 2024-09-11T06:51:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/844541331", 2024-09-11T06:51:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 25 2024-09-11T06:51:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697727+0000", 2024-09-11T06:51:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697727+0000", 2024-09-11T06:51:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697729+0000", 2024-09-11T06:51:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.33e-06 2024-09-11T06:51:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697734+0000", 2024-09-11T06:51:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6590000000000004e-06 2024-09-11T06:51:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697749+0000", 2024-09-11T06:51:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4867e-05 2024-09-11T06:51:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697759+0000", 2024-09-11T06:51:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0073e-05 2024-09-11T06:51:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.698715+0000", 2024-09-11T06:51:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00095545799999999996 2024-09-11T06:51:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.698850+0000", 2024-09-11T06:51:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000135814 2024-09-11T06:51:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.699085+0000", 2024-09-11T06:51:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023507199999999999 2024-09-11T06:51:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.700258+0000", 2024-09-11T06:51:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011726600000000001 2024-09-11T06:51:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.701061+0000", 2024-09-11T06:51:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00080256100000000003 2024-09-11T06:51:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.701144+0000", 2024-09-11T06:51:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3188000000000004e-05 2024-09-11T06:51:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.701261+0000", 2024-09-11T06:51:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000117451 2024-09-11T06:51:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:04.402 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:04.404 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:51:04.404 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:51:04.404 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:51:04.405 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:51:04.405 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:51:04.405 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:51:04.406 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:51:04.406 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:51:04.406 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:51:04.406 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:04.398+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:04.417 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:51:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:51:04.437 INFO:tasks.workunit.client.0.smithi176.stderr:+ LOG_FILE=/tmp/tmp.IGWYJYR7sz/rbd-nbd.log 2024-09-11T06:51:04.437 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' -n '' ']' 2024-09-11T06:51:04.438 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:51:04.438 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:51:04.438 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo rbd device --device-type nbd map --quiesce rbd/testrbdnbd21607 --log-file=/tmp/tmp.IGWYJYR7sz/rbd-nbd.log 2024-09-11T06:51:04.438 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:51:04.438 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:51:04.440 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:51:04.441 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which rbd 2024-09-11T06:51:04.442 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/rbd 2024-09-11T06:51:04.442 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:51:04.443 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map --quiesce rbd/testrbdnbd21607 --log-file=/tmp/tmp.IGWYJYR7sz/rbd-nbd.log 2024-09-11T06:51:04.504 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:04.498+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:04.538 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:04.539 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:04.553 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:51:04.601 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV=/dev/nbd1 2024-09-11T06:51:04.602 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd 2024-09-11T06:51:04.602 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:51:04.602 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:51:04.602 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:51:04.602 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd1'\'']/id' 2024-09-11T06:51:04.605 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:04.602+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:04.675 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:04.675 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:04.691 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:51:04.695 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=23910 2024-09-11T06:51:04.695 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 23910 2024-09-11T06:51:04.695 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 23910 -C rbd-nbd 2024-09-11T06:51:04.704 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:51:04.704 INFO:tasks.workunit.client.0.smithi176.stdout: 23910 ? 00:00:00 rbd-nbd 2024-09-11T06:51:04.704 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo mkfs /dev/nbd1 2024-09-11T06:51:04.704 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:04.704 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:04.705 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:04.705 INFO:tasks.workunit.client.0.smithi176.stderr:++ which mkfs 2024-09-11T06:51:04.706 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/sbin/mkfs 2024-09-11T06:51:04.707 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:04.707 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/sbin/mkfs /dev/nbd1 2024-09-11T06:51:04.736 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:04.734+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:04.740 INFO:tasks.workunit.client.0.smithi176.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-11T06:51:04.808 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:04.802+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:04.829 INFO:tasks.workunit.client.0.smithi176.stdout:Discarding device blocks: 0/16384 done 2024-09-11T06:51:04.829 INFO:tasks.workunit.client.0.smithi176.stdout:Creating filesystem with 16384 4k blocks and 16384 inodes 2024-09-11T06:51:04.829 INFO:tasks.workunit.client.0.smithi176.stdout: 2024-09-11T06:51:04.829 INFO:tasks.workunit.client.0.smithi176.stdout:Allocating group tables: 0/1 done 2024-09-11T06:51:04.865 INFO:tasks.workunit.client.0.smithi176.stdout:Writing inode tables: 0/1 done 2024-09-11T06:51:04.871 INFO:teuthology.orchestra.run.smithi071.stderr:set 3.3 pg_upmap mapping to [2,0] 2024-09-11T06:51:04.874 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T06:51:04.875 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:04.875 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:04.895 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:51:04.913 INFO:tasks.workunit.client.0.smithi176.stdout:Writing superblocks and filesystem accounting information: 0/1 done 2024-09-11T06:51:04.913 INFO:tasks.workunit.client.0.smithi176.stdout: 2024-09-11T06:51:04.914 INFO:tasks.workunit.client.0.smithi176.stderr:+ mkdir /tmp/tmp.IGWYJYR7sz/mnt 2024-09-11T06:51:04.916 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo mount /dev/nbd1 /tmp/tmp.IGWYJYR7sz/mnt 2024-09-11T06:51:04.917 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:04.917 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:04.918 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:04.918 INFO:tasks.workunit.client.0.smithi176.stderr:++ which mount 2024-09-11T06:51:04.920 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/mount 2024-09-11T06:51:04.920 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:04.920 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/mount /dev/nbd1 /tmp/tmp.IGWYJYR7sz/mnt 2024-09-11T06:51:04.940 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:04.938+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:04.970 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd snap create rbd/testrbdnbd21607@quiesce3 2024-09-11T06:51:05.007 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:05.002+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:05.066 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:05.066 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:05.080 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:51:05.128 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:05.126+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:05.201 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:05.201 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:05.210 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:05.206+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:05.215 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:51:05.310 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:05.306+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:05.378 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:05.378 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:05.392 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:51:05.436 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:05.434+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:05.512 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:05.510+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:05.557 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:05.557 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:05.570 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:51:05.613 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:05.610+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:05.714 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:05.710+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:05.731 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:05.731 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:05.745 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:51:05.815 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:05.810+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:05.885 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is unset 2024-09-11T06:51:05.904 INFO:tasks.workunit.client.0.smithi176.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2024-09-11T06:51:05.913 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo dd if=/tmp/tmp.IGWYJYR7sz/data of=/tmp/tmp.IGWYJYR7sz/mnt/test bs=1M count=1 oflag=direct 2024-09-11T06:51:05.913 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:05.913 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:05.914 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:05.915 INFO:tasks.workunit.client.0.smithi176.stderr:++ which dd 2024-09-11T06:51:05.916 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/dd 2024-09-11T06:51:05.917 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:05.917 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/dd if=/tmp/tmp.IGWYJYR7sz/data of=/tmp/tmp.IGWYJYR7sz/mnt/test bs=1M count=1 oflag=direct 2024-09-11T06:51:05.917 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:05.917 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:05.929 INFO:tasks.workunit.client.0.smithi176.stderr:1+0 records in 2024-09-11T06:51:05.929 INFO:tasks.workunit.client.0.smithi176.stderr:1+0 records out 2024-09-11T06:51:05.930 INFO:tasks.workunit.client.0.smithi176.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00269339 s, 389 MB/s 2024-09-11T06:51:05.930 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo umount /tmp/tmp.IGWYJYR7sz/mnt 2024-09-11T06:51:05.930 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:05.931 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:05.931 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:05.932 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:51:05.933 INFO:tasks.workunit.client.0.smithi176.stderr:++ which umount 2024-09-11T06:51:05.933 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/umount 2024-09-11T06:51:05.933 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:05.933 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/umount /tmp/tmp.IGWYJYR7sz/mnt 2024-09-11T06:51:05.949 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:05.946+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:05.960 INFO:tasks.workunit.client.0.smithi176.stderr:+ unmap_device /dev/nbd1 23910 2024-09-11T06:51:05.960 INFO:tasks.workunit.client.0.smithi176.stderr:+ local args=/dev/nbd1 2024-09-11T06:51:05.961 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pid=23910 2024-09-11T06:51:05.961 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd1 2024-09-11T06:51:05.961 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:05.961 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:05.961 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:05.962 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:51:05.963 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:51:05.963 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:05.963 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd1 2024-09-11T06:51:06.017 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:06.014+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:06.072 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^23910\b' 2024-09-11T06:51:06.073 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:51:06.073 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^23910\b' 2024-09-11T06:51:06.074 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:06.074 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:06.089 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:51:06.118 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:06.114+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:06.139 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:51:06.139 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -C rbd-nbd 2024-09-11T06:51:06.140 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^ *23910\b' 2024-09-11T06:51:06.140 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^ *23910\b' 2024-09-11T06:51:06.147 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:51:06.148 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 0.5 2024-09-11T06:51:06.218 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:06.214+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:06.254 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:06.254 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:06.268 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:51:06.353 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:06.350+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:06.420 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:06.414+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:06.433 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:06.433 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:06.448 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:51:06.454 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:51:06.533 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:06.530+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:06.611 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:06.611 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:06.621 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:06.618+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:06.626 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:51:06.630 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:51:06.631 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:51:06.647 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:51:06.648 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:51:06.649 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV= 2024-09-11T06:51:06.649 INFO:tasks.workunit.client.0.smithi176.stderr:+ cat /tmp/tmp.IGWYJYR7sz/rbd-nbd.log 2024-09-11T06:51:06.650 INFO:tasks.workunit.client.0.smithi176.stdout:2024-09-11T06:51:04.523+0000 7f3c0e3eda80 0 ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process rbd-nbd, pid 23901 2024-09-11T06:51:06.650 INFO:tasks.workunit.client.0.smithi176.stdout:2024-09-11T06:51:04.527+0000 7f3c0e3eda80 0 pidfile_write: ignore empty --pid-file 2024-09-11T06:51:06.650 INFO:tasks.workunit.client.0.smithi176.stdout:2024-09-11T06:51:04.851+0000 7f3c0c6e0640 0 -- 172.21.15.176:0/3128961744 >> [v2:172.21.15.71:6808/620364628,v1:172.21.15.71:6809/620364628] conn(0x7f3be400b400 msgr2=0x7f3be400d8a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-11T06:51:06.650 INFO:tasks.workunit.client.0.smithi176.stdout:2024-09-11T06:51:04.883+0000 7f3c0c6e0640 0 -- 172.21.15.176:0/3128961744 >> [v2:172.21.15.89:6808/558942315,v1:172.21.15.89:6809/558942315] conn(0x7f3bf803b860 msgr2=0x7f3bf803dc50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-11T06:51:06.650 INFO:tasks.workunit.client.0.smithi176.stdout:2024-09-11T06:51:06.047+0000 7f3bf0ff9640 0 rbd-nbd: disconnect request received 2024-09-11T06:51:06.651 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep 'quiesce failed' /tmp/tmp.IGWYJYR7sz/rbd-nbd.log 2024-09-11T06:51:06.651 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep 'quiesce failed' /tmp/tmp.IGWYJYR7sz/rbd-nbd.log 2024-09-11T06:51:06.653 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:51:06.653 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo rbd device --device-type nbd --show-cookie map rbd/testrbdnbd21607 2024-09-11T06:51:06.653 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:51:06.654 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:51:06.655 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:51:06.655 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which rbd 2024-09-11T06:51:06.657 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/rbd 2024-09-11T06:51:06.657 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:51:06.657 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd --show-cookie map rbd/testrbdnbd21607 2024-09-11T06:51:06.664 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:51:06.664 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:51:06.685 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:51:06.685 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:51:06.705 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:51:06.706 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:51:06.722 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:06.718+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:06.727 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:51:06.727 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:51:06.744 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:06.744 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:06.744 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:51:06.744 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:51:06.744 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:06.759 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:51:06.802 INFO:tasks.workunit.client.0.smithi176.stderr:+ OUT='/dev/nbd1 1935c02c-84a6-4710-85dc-6630efac9835' 2024-09-11T06:51:06.802 INFO:tasks.workunit.client.0.smithi176.stderr:+ read DEV COOKIE 2024-09-11T06:51:06.802 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd 2024-09-11T06:51:06.802 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:51:06.803 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:51:06.803 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:51:06.803 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd1'\'']/id' 2024-09-11T06:51:06.845 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:06.842+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:06.889 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=24025 2024-09-11T06:51:06.889 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 24025 2024-09-11T06:51:06.889 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 24025 -C rbd-nbd 2024-09-11T06:51:06.897 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:51:06.897 INFO:tasks.workunit.client.0.smithi176.stdout: 24025 ? 00:00:00 rbd-nbd 2024-09-11T06:51:06.897 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo mount /dev/nbd1 /tmp/tmp.IGWYJYR7sz/mnt 2024-09-11T06:51:06.897 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:06.898 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:06.898 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:06.898 INFO:tasks.workunit.client.0.smithi176.stderr:++ which mount 2024-09-11T06:51:06.900 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/mount 2024-09-11T06:51:06.900 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:06.900 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/mount /dev/nbd1 /tmp/tmp.IGWYJYR7sz/mnt 2024-09-11T06:51:06.923 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:06.918+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:06.928 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:06.929 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:51:06.929 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:51:06.929 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:51:06.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.929 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4558.0:5 3.2 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-11T06:51:06.929 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:51:06.929 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 48.836139799000001, 2024-09-11T06:51:06.929 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0062353959999999998, 2024-09-11T06:51:06.929 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:06.930 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:06.930 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:06.930 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:06.930 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4558", 2024-09-11T06:51:06.930 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1478388898", 2024-09-11T06:51:06.930 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 5 2024-09-11T06:51:06.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.930 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:06.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.930 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:06.930 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:51:06.931 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:06.931 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.931 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.931 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:06.931 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:51:06.931 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:06.931 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.931 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.931 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:06.931 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088422+0000", 2024-09-11T06:51:06.931 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7859999999999998e-06 2024-09-11T06:51:06.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.932 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:06.932 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088440+0000", 2024-09-11T06:51:06.932 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8185999999999998e-05 2024-09-11T06:51:06.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.932 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:06.932 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088479+0000", 2024-09-11T06:51:06.932 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8248999999999999e-05 2024-09-11T06:51:06.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.933 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:06.933 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088497+0000", 2024-09-11T06:51:06.933 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8889000000000002e-05 2024-09-11T06:51:06.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.933 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:06.934 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088658+0000", 2024-09-11T06:51:06.934 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000160271 2024-09-11T06:51:06.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.934 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op must wait for map", 2024-09-11T06:51:06.934 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088668+0000", 2024-09-11T06:51:06.934 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0409e-05 2024-09-11T06:51:06.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.935 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:06.935 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092347+0000", 2024-09-11T06:51:06.935 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00367915 2024-09-11T06:51:06.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.935 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.935 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:06.935 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092541+0000", 2024-09-11T06:51:06.935 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000193833 2024-09-11T06:51:06.936 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.936 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-11T06:51:06.936 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092732+0000", 2024-09-11T06:51:06.936 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000190772 2024-09-11T06:51:06.936 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.936 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:06.937 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094136+0000", 2024-09-11T06:51:06.937 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014040809999999999 2024-09-11T06:51:06.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.937 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.937 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:06.937 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094506+0000", 2024-09-11T06:51:06.937 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000369959 2024-09-11T06:51:06.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.937 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.937 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:06.938 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094590+0000", 2024-09-11T06:51:06.938 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4033e-05 2024-09-11T06:51:06.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.938 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:06.938 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094654+0000", 2024-09-11T06:51:06.938 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3777999999999998e-05 2024-09-11T06:51:06.938 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:06.939 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:06.939 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:06.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.939 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.939 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:261 2.6 2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:51:06.939 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:21.154411+0000", 2024-09-11T06:51:06.939 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.770147000999998, 2024-09-11T06:51:06.939 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002245024, 2024-09-11T06:51:06.939 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:06.939 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:06.939 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:06.939 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:06.940 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:51:06.940 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:51:06.940 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 261 2024-09-11T06:51:06.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.940 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:06.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.940 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:06.940 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154411+0000", 2024-09-11T06:51:06.940 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:06.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.941 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.941 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:06.941 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154411+0000", 2024-09-11T06:51:06.941 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:06.941 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.941 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.941 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:06.941 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154412+0000", 2024-09-11T06:51:06.941 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:51:06.941 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.942 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.942 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:06.942 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154416+0000", 2024-09-11T06:51:06.942 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3930000000000001e-06 2024-09-11T06:51:06.942 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.942 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.942 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:06.942 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154428+0000", 2024-09-11T06:51:06.942 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2099999999999999e-05 2024-09-11T06:51:06.942 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.942 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.942 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:06.942 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.154438+0000", 2024-09-11T06:51:06.943 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2590000000000006e-06 2024-09-11T06:51:06.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.943 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.943 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:06.943 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.155643+0000", 2024-09-11T06:51:06.943 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012048689999999999 2024-09-11T06:51:06.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.943 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.943 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:06.943 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.155838+0000", 2024-09-11T06:51:06.944 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019531299999999999 2024-09-11T06:51:06.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.944 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:06.944 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:21.156656+0000", 2024-09-11T06:51:06.944 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00081824700000000003 2024-09-11T06:51:06.944 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:06.944 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:06.944 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:06.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.944 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:587 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-11T06:51:06.945 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:51:06.945 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 44.695307143999997, 2024-09-11T06:51:06.945 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0023436289999999999, 2024-09-11T06:51:06.945 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:06.945 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:06.945 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:06.945 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:06.945 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:51:06.945 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:51:06.945 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 587 2024-09-11T06:51:06.945 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.945 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:06.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.946 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:06.946 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:51:06.946 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:06.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.946 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:06.946 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:51:06.946 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:06.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.947 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.947 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:06.947 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229252+0000", 2024-09-11T06:51:06.947 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3799999999999999e-06 2024-09-11T06:51:06.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.947 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.947 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:06.947 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229259+0000", 2024-09-11T06:51:06.947 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4760000000000003e-06 2024-09-11T06:51:06.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.947 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.948 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:06.948 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229271+0000", 2024-09-11T06:51:06.948 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2313000000000001e-05 2024-09-11T06:51:06.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.948 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:06.948 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229279+0000", 2024-09-11T06:51:06.948 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1610000000000003e-06 2024-09-11T06:51:06.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.949 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:06.949 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229344+0000", 2024-09-11T06:51:06.949 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4375000000000006e-05 2024-09-11T06:51:06.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.949 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:06.949 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229404+0000", 2024-09-11T06:51:06.949 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0517999999999999e-05 2024-09-11T06:51:06.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.949 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:06.950 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229516+0000", 2024-09-11T06:51:06.950 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011129900000000001 2024-09-11T06:51:06.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.950 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:06.950 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231502+0000", 2024-09-11T06:51:06.950 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001986238 2024-09-11T06:51:06.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.950 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:06.950 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231537+0000", 2024-09-11T06:51:06.951 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4968000000000001e-05 2024-09-11T06:51:06.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.951 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:06.951 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231569+0000", 2024-09-11T06:51:06.951 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2459000000000002e-05 2024-09-11T06:51:06.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.951 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:06.952 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231595+0000", 2024-09-11T06:51:06.952 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5442000000000001e-05 2024-09-11T06:51:06.952 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:06.952 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:06.952 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:06.952 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.952 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.952 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:15 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:06.952 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.709418+0000", 2024-09-11T06:51:06.952 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 36.215139991000001, 2024-09-11T06:51:06.952 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027504280000000001, 2024-09-11T06:51:06.953 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:06.953 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:06.953 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:06.953 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:06.953 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:06.953 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:06.953 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 15 2024-09-11T06:51:06.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.953 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:06.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.953 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:06.953 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709418+0000", 2024-09-11T06:51:06.954 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:06.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.954 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.954 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:06.954 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709418+0000", 2024-09-11T06:51:06.954 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:06.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.954 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.954 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:06.955 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709420+0000", 2024-09-11T06:51:06.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2299999999999998e-06 2024-09-11T06:51:06.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.955 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:06.955 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709432+0000", 2024-09-11T06:51:06.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1897e-05 2024-09-11T06:51:06.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.955 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:06.955 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709464+0000", 2024-09-11T06:51:06.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1819000000000001e-05 2024-09-11T06:51:06.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.956 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:06.956 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709479+0000", 2024-09-11T06:51:06.956 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4853e-05 2024-09-11T06:51:06.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.956 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:06.956 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709624+0000", 2024-09-11T06:51:06.956 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144912 2024-09-11T06:51:06.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.957 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:06.957 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709773+0000", 2024-09-11T06:51:06.957 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000149529 2024-09-11T06:51:06.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.957 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:06.957 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.710055+0000", 2024-09-11T06:51:06.957 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000281692 2024-09-11T06:51:06.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.957 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:06.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.711412+0000", 2024-09-11T06:51:06.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001356931 2024-09-11T06:51:06.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.958 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:06.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.711890+0000", 2024-09-11T06:51:06.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00047768700000000002 2024-09-11T06:51:06.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.958 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:06.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.712050+0000", 2024-09-11T06:51:06.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016041499999999999 2024-09-11T06:51:06.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.959 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.959 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:06.959 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.712169+0000", 2024-09-11T06:51:06.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011846299999999999 2024-09-11T06:51:06.959 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:06.959 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:06.959 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:06.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.959 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.960 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:18 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:06.960 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.713502+0000", 2024-09-11T06:51:06.960 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 36.211056565, 2024-09-11T06:51:06.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032323069999999998, 2024-09-11T06:51:06.960 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:06.960 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:06.960 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:06.960 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:06.960 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:06.960 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:06.960 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18 2024-09-11T06:51:06.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.961 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:06.961 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.961 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:06.961 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713502+0000", 2024-09-11T06:51:06.961 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:06.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.961 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.961 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:06.961 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713502+0000", 2024-09-11T06:51:06.961 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:06.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:06.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713503+0000", 2024-09-11T06:51:06.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8199999999999999e-06 2024-09-11T06:51:06.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:06.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713513+0000", 2024-09-11T06:51:06.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8539999999999992e-06 2024-09-11T06:51:06.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:06.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713540+0000", 2024-09-11T06:51:06.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6594999999999999e-05 2024-09-11T06:51:06.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.963 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:06.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713553+0000", 2024-09-11T06:51:06.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2843999999999999e-05 2024-09-11T06:51:06.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.963 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:06.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713686+0000", 2024-09-11T06:51:06.964 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013294899999999999 2024-09-11T06:51:06.964 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.964 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.964 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:06.964 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713937+0000", 2024-09-11T06:51:06.964 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025110600000000002 2024-09-11T06:51:06.964 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.964 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.964 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:06.964 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.714412+0000", 2024-09-11T06:51:06.964 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00047562 2024-09-11T06:51:06.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.965 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.965 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:06.965 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.715867+0000", 2024-09-11T06:51:06.965 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001455004 2024-09-11T06:51:06.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.965 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.965 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:06.965 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.716569+0000", 2024-09-11T06:51:06.965 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070159400000000005 2024-09-11T06:51:06.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.966 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:06.966 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.716660+0000", 2024-09-11T06:51:06.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1403000000000006e-05 2024-09-11T06:51:06.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.966 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:06.966 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.716734+0000", 2024-09-11T06:51:06.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3517999999999996e-05 2024-09-11T06:51:06.966 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:06.966 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:06.967 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:06.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.967 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:23 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:06.967 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.723392+0000", 2024-09-11T06:51:06.967 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 36.201165969000002, 2024-09-11T06:51:06.967 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027098489999999999, 2024-09-11T06:51:06.967 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:06.967 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:06.968 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:06.968 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:06.968 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:06.968 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:06.968 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23 2024-09-11T06:51:06.968 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.968 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:06.968 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.968 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:06.968 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723392+0000", 2024-09-11T06:51:06.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:06.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.969 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:06.969 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723392+0000", 2024-09-11T06:51:06.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:06.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.969 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:06.969 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723394+0000", 2024-09-11T06:51:06.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0669999999999999e-06 2024-09-11T06:51:06.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.970 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:06.970 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723406+0000", 2024-09-11T06:51:06.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1367e-05 2024-09-11T06:51:06.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.970 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:06.970 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723436+0000", 2024-09-11T06:51:06.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0874999999999998e-05 2024-09-11T06:51:06.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:06.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723451+0000", 2024-09-11T06:51:06.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4192999999999999e-05 2024-09-11T06:51:06.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:06.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723577+0000", 2024-09-11T06:51:06.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000126267 2024-09-11T06:51:06.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:06.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723718+0000", 2024-09-11T06:51:06.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000141206 2024-09-11T06:51:06.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:06.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723982+0000", 2024-09-11T06:51:06.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026338499999999999 2024-09-11T06:51:06.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:06.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.725377+0000", 2024-09-11T06:51:06.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013956999999999999 2024-09-11T06:51:06.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:06.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.725908+0000", 2024-09-11T06:51:06.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00053036200000000004 2024-09-11T06:51:06.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:06.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.725999+0000", 2024-09-11T06:51:06.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1016999999999999e-05 2024-09-11T06:51:06.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:06.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726102+0000", 2024-09-11T06:51:06.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010341 2024-09-11T06:51:06.974 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:06.974 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:06.974 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:06.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.974 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:15 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:06.974 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.868921+0000", 2024-09-11T06:51:06.974 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 36.055636927999998, 2024-09-11T06:51:06.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0026879, 2024-09-11T06:51:06.974 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:06.974 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:06.974 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:06.975 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:06.975 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:06.975 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:06.975 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 15 2024-09-11T06:51:06.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.975 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:06.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:06.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868921+0000", 2024-09-11T06:51:06.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:06.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.976 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.976 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:06.976 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868921+0000", 2024-09-11T06:51:06.976 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:06.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.976 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.976 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:06.976 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868923+0000", 2024-09-11T06:51:06.976 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8959999999999999e-06 2024-09-11T06:51:06.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:06.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868934+0000", 2024-09-11T06:51:06.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1083e-05 2024-09-11T06:51:06.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:06.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868965+0000", 2024-09-11T06:51:06.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0833000000000001e-05 2024-09-11T06:51:06.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.978 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:06.978 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868990+0000", 2024-09-11T06:51:06.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4545999999999998e-05 2024-09-11T06:51:06.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.978 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:06.978 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.869124+0000", 2024-09-11T06:51:06.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013489599999999999 2024-09-11T06:51:06.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:06.979 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.869270+0000", 2024-09-11T06:51:06.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000145976 2024-09-11T06:51:06.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:06.979 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.869538+0000", 2024-09-11T06:51:06.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026780599999999999 2024-09-11T06:51:06.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.980 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.980 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:06.980 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.870691+0000", 2024-09-11T06:51:06.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00115277 2024-09-11T06:51:06.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.980 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.981 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:06.981 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.871456+0000", 2024-09-11T06:51:06.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00076473800000000005 2024-09-11T06:51:06.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.981 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:06.981 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.871543+0000", 2024-09-11T06:51:06.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7040999999999996e-05 2024-09-11T06:51:06.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.982 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:06.982 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.871609+0000", 2024-09-11T06:51:06.982 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6315000000000005e-05 2024-09-11T06:51:06.982 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:06.982 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:06.982 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:06.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.982 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:20 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:06.983 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.877429+0000", 2024-09-11T06:51:06.983 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 36.047128639, 2024-09-11T06:51:06.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032382629999999999, 2024-09-11T06:51:06.983 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:06.983 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:06.983 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:06.983 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:06.983 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:06.984 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:06.984 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 20 2024-09-11T06:51:06.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.984 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:06.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.984 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:06.984 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877429+0000", 2024-09-11T06:51:06.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:06.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:06.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877429+0000", 2024-09-11T06:51:06.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:06.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:06.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877431+0000", 2024-09-11T06:51:06.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9429999999999999e-06 2024-09-11T06:51:06.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:06.986 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877444+0000", 2024-09-11T06:51:06.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2396e-05 2024-09-11T06:51:06.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:06.986 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877474+0000", 2024-09-11T06:51:06.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0239999999999998e-05 2024-09-11T06:51:06.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.987 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:06.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877496+0000", 2024-09-11T06:51:06.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1914999999999999e-05 2024-09-11T06:51:06.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.987 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:06.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877616+0000", 2024-09-11T06:51:06.988 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000119814 2024-09-11T06:51:06.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.988 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.988 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:06.988 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877836+0000", 2024-09-11T06:51:06.988 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022029600000000001 2024-09-11T06:51:06.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.988 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.988 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:06.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.878482+0000", 2024-09-11T06:51:06.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00064605200000000004 2024-09-11T06:51:06.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:06.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.879804+0000", 2024-09-11T06:51:06.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013216460000000001 2024-09-11T06:51:06.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:06.990 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.880519+0000", 2024-09-11T06:51:06.990 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00071547600000000005 2024-09-11T06:51:06.990 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.990 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:06.990 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.880606+0000", 2024-09-11T06:51:06.990 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6907e-05 2024-09-11T06:51:06.990 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.990 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:06.991 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.880668+0000", 2024-09-11T06:51:06.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1577999999999999e-05 2024-09-11T06:51:06.991 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:06.991 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:06.991 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:06.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.991 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:23 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:06.991 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.885067+0000", 2024-09-11T06:51:06.991 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 36.039490985, 2024-09-11T06:51:06.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028127320000000001, 2024-09-11T06:51:06.991 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:06.992 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:06.992 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:06.992 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:06.992 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:06.992 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:06.992 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23 2024-09-11T06:51:06.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.992 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:06.992 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.992 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:06.992 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885067+0000", 2024-09-11T06:51:06.993 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:06.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.993 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:06.993 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885067+0000", 2024-09-11T06:51:06.993 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:06.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.993 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:06.993 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885069+0000", 2024-09-11T06:51:06.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2000000000000001e-06 2024-09-11T06:51:06.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.994 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:06.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885081+0000", 2024-09-11T06:51:06.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1209999999999999e-05 2024-09-11T06:51:06.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.994 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:06.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885111+0000", 2024-09-11T06:51:06.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0318999999999999e-05 2024-09-11T06:51:06.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.995 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:06.995 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885132+0000", 2024-09-11T06:51:06.995 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0649000000000002e-05 2024-09-11T06:51:06.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.995 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:06.995 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885243+0000", 2024-09-11T06:51:06.995 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000111284 2024-09-11T06:51:06.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:06.996 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885387+0000", 2024-09-11T06:51:06.996 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144222 2024-09-11T06:51:06.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:06.996 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885699+0000", 2024-09-11T06:51:06.996 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00031239800000000003 2024-09-11T06:51:06.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:06.996 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.886817+0000", 2024-09-11T06:51:06.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001117191 2024-09-11T06:51:06.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.997 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.997 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:06.997 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.887684+0000", 2024-09-11T06:51:06.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00086699499999999996 2024-09-11T06:51:06.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.997 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.997 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:06.997 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.887772+0000", 2024-09-11T06:51:06.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8224000000000001e-05 2024-09-11T06:51:06.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.997 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:06.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.887880+0000", 2024-09-11T06:51:06.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010804 2024-09-11T06:51:06.998 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:06.998 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:06.998 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:06.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.998 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "MOSDPGPull(2.0 e70/69 cost 1048576)", 2024-09-11T06:51:06.998 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:35.528435+0000", 2024-09-11T06:51:06.998 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.396123248999999, 2024-09-11T06:51:06.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027390520000000001, 2024-09-11T06:51:06.998 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:06.999 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:06.999 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-11T06:51:06.999 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:06.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.999 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:06.999 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528435+0000", 2024-09-11T06:51:06.999 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:06.999 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:06.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:06.999 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:06.999 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528435+0000", 2024-09-11T06:51:06.999 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:06.999 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.000 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.000 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528437+0000", 2024-09-11T06:51:07.000 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5959999999999998e-06 2024-09-11T06:51:07.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.000 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.000 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528465+0000", 2024-09-11T06:51:07.000 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7436e-05 2024-09-11T06:51:07.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.000 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.000 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528499+0000", 2024-09-11T06:51:07.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3599000000000001e-05 2024-09-11T06:51:07.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.001 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528515+0000", 2024-09-11T06:51:07.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.677e-05 2024-09-11T06:51:07.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.001 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528652+0000", 2024-09-11T06:51:07.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000136546 2024-09-11T06:51:07.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.002 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.531174+0000", 2024-09-11T06:51:07.002 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0025221050000000002 2024-09-11T06:51:07.002 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.002 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.002 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.002 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "MOSDPGPull(2.0 e70/69 cost 1048576)", 2024-09-11T06:51:07.002 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:35.607413+0000", 2024-09-11T06:51:07.002 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.317145095000001, 2024-09-11T06:51:07.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0021226639999999998, 2024-09-11T06:51:07.003 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.003 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.003 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-11T06:51:07.003 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.003 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.607413+0000", 2024-09-11T06:51:07.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.004 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.004 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.607413+0000", 2024-09-11T06:51:07.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.004 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.004 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.607415+0000", 2024-09-11T06:51:07.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7069999999999999e-06 2024-09-11T06:51:07.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.004 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.004 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.607436+0000", 2024-09-11T06:51:07.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1151000000000001e-05 2024-09-11T06:51:07.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.607456+0000", 2024-09-11T06:51:07.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0559000000000001e-05 2024-09-11T06:51:07.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.607466+0000", 2024-09-11T06:51:07.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3419999999999997e-06 2024-09-11T06:51:07.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.006 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.006 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.607646+0000", 2024-09-11T06:51:07.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017984000000000001 2024-09-11T06:51:07.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.006 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.006 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.609536+0000", 2024-09-11T06:51:07.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001890065 2024-09-11T06:51:07.006 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.007 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.007 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.007 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4828.0:15 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-11T06:51:07.007 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:36.253730+0000", 2024-09-11T06:51:07.007 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.670827906, 2024-09-11T06:51:07.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0025257589999999998, 2024-09-11T06:51:07.007 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.007 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.007 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:07.008 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.008 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4828", 2024-09-11T06:51:07.008 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2410288650", 2024-09-11T06:51:07.008 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 15 2024-09-11T06:51:07.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.008 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.008 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.008 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253730+0000", 2024-09-11T06:51:07.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.009 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.009 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253730+0000", 2024-09-11T06:51:07.009 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.009 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.009 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253732+0000", 2024-09-11T06:51:07.009 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2400000000000002e-06 2024-09-11T06:51:07.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.009 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253744+0000", 2024-09-11T06:51:07.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1316e-05 2024-09-11T06:51:07.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253771+0000", 2024-09-11T06:51:07.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.724e-05 2024-09-11T06:51:07.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253785+0000", 2024-09-11T06:51:07.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3786e-05 2024-09-11T06:51:07.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.011 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.011 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253886+0000", 2024-09-11T06:51:07.011 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000100981 2024-09-11T06:51:07.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.011 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.011 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.254029+0000", 2024-09-11T06:51:07.011 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014311900000000001 2024-09-11T06:51:07.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.012 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:07.012 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.254317+0000", 2024-09-11T06:51:07.012 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00028809300000000001 2024-09-11T06:51:07.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.012 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.012 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:07.012 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.255403+0000", 2024-09-11T06:51:07.012 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001085639 2024-09-11T06:51:07.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.012 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.012 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:07.013 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.256107+0000", 2024-09-11T06:51:07.013 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070468799999999997 2024-09-11T06:51:07.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.013 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:07.013 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.256191+0000", 2024-09-11T06:51:07.013 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3714999999999994e-05 2024-09-11T06:51:07.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.013 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.013 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.256256+0000", 2024-09-11T06:51:07.014 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4942000000000003e-05 2024-09-11T06:51:07.014 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.014 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.014 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.014 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5313.0:26 2.6 2:7b712441:ns::rbd_object_map.11adc335555f:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:51:07.014 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.260230+0000", 2024-09-11T06:51:07.014 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.664328498, 2024-09-11T06:51:07.014 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002457201, 2024-09-11T06:51:07.014 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.014 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.014 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:07.015 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.015 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5313", 2024-09-11T06:51:07.015 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/624314229", 2024-09-11T06:51:07.015 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:51:07.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.015 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.015 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.015 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260230+0000", 2024-09-11T06:51:07.015 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.016 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.016 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260230+0000", 2024-09-11T06:51:07.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.016 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.016 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.016 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260232+0000", 2024-09-11T06:51:07.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.956e-06 2024-09-11T06:51:07.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.016 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.016 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.017 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260243+0000", 2024-09-11T06:51:07.017 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1687e-05 2024-09-11T06:51:07.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.017 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.017 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260269+0000", 2024-09-11T06:51:07.017 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6211999999999999e-05 2024-09-11T06:51:07.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.017 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.017 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260284+0000", 2024-09-11T06:51:07.018 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4053000000000001e-05 2024-09-11T06:51:07.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.018 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.018 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.018 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260391+0000", 2024-09-11T06:51:07.018 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000107278 2024-09-11T06:51:07.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.018 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.018 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.018 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260541+0000", 2024-09-11T06:51:07.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015033000000000001 2024-09-11T06:51:07.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 3", 2024-09-11T06:51:07.019 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260951+0000", 2024-09-11T06:51:07.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040981299999999997 2024-09-11T06:51:07.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:07.020 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262399+0000", 2024-09-11T06:51:07.020 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014475930000000001 2024-09-11T06:51:07.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.020 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:07.020 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262531+0000", 2024-09-11T06:51:07.020 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013239 2024-09-11T06:51:07.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.021 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.021 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:07.021 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262623+0000", 2024-09-11T06:51:07.021 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2361000000000006e-05 2024-09-11T06:51:07.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.021 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.021 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.021 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262687+0000", 2024-09-11T06:51:07.021 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3528000000000006e-05 2024-09-11T06:51:07.022 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.022 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.022 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.022 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5313.0:31 2.6 2:7b712441:ns::rbd_object_map.11adc335555f:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:51:07.022 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.266495+0000", 2024-09-11T06:51:07.022 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.658062724000001, 2024-09-11T06:51:07.022 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0026763870000000001, 2024-09-11T06:51:07.022 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.022 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.023 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:07.023 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.023 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5313", 2024-09-11T06:51:07.023 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/624314229", 2024-09-11T06:51:07.023 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 31 2024-09-11T06:51:07.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.023 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.023 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.023 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.023 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266495+0000", 2024-09-11T06:51:07.023 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.024 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.024 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.024 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266495+0000", 2024-09-11T06:51:07.024 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.024 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.024 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.024 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266498+0000", 2024-09-11T06:51:07.024 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2369999999999999e-06 2024-09-11T06:51:07.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.024 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.024 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.024 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266509+0000", 2024-09-11T06:51:07.025 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1459e-05 2024-09-11T06:51:07.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.025 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.025 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.025 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266540+0000", 2024-09-11T06:51:07.025 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1189999999999998e-05 2024-09-11T06:51:07.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.025 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.025 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.025 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266563+0000", 2024-09-11T06:51:07.025 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2529000000000001e-05 2024-09-11T06:51:07.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.026 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.026 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266708+0000", 2024-09-11T06:51:07.026 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014511 2024-09-11T06:51:07.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.026 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.026 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266962+0000", 2024-09-11T06:51:07.026 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000253897 2024-09-11T06:51:07.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.026 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 3", 2024-09-11T06:51:07.027 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.267430+0000", 2024-09-11T06:51:07.027 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00046799899999999998 2024-09-11T06:51:07.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.027 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.027 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:07.027 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.268765+0000", 2024-09-11T06:51:07.027 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013350359999999999 2024-09-11T06:51:07.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.027 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.027 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:07.027 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.269019+0000", 2024-09-11T06:51:07.027 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025397000000000001 2024-09-11T06:51:07.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.028 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:07.028 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.269105+0000", 2024-09-11T06:51:07.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5952000000000006e-05 2024-09-11T06:51:07.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.028 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.028 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.269172+0000", 2024-09-11T06:51:07.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7008000000000006e-05 2024-09-11T06:51:07.028 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.028 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.028 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.029 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:5 3.4 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:07.029 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.588582+0000", 2024-09-11T06:51:07.029 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.335976571, 2024-09-11T06:51:07.029 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002889857, 2024-09-11T06:51:07.029 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.029 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.029 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:07.029 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.030 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:51:07.030 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:51:07.030 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 5 2024-09-11T06:51:07.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.030 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588582+0000", 2024-09-11T06:51:07.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.031 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.031 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588582+0000", 2024-09-11T06:51:07.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.031 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.031 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588583+0000", 2024-09-11T06:51:07.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.223e-06 2024-09-11T06:51:07.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588588+0000", 2024-09-11T06:51:07.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4299999999999997e-06 2024-09-11T06:51:07.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588606+0000", 2024-09-11T06:51:07.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7652999999999999e-05 2024-09-11T06:51:07.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.033 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588615+0000", 2024-09-11T06:51:07.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4399999999999994e-06 2024-09-11T06:51:07.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.033 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588725+0000", 2024-09-11T06:51:07.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010945799999999999 2024-09-11T06:51:07.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.034 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.034 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.034 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.589031+0000", 2024-09-11T06:51:07.034 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030590000000000001 2024-09-11T06:51:07.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.034 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.034 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:07.034 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.589521+0000", 2024-09-11T06:51:07.034 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00049061299999999999 2024-09-11T06:51:07.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.035 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:07.035 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.590792+0000", 2024-09-11T06:51:07.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012708039999999999 2024-09-11T06:51:07.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.035 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:07.035 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591339+0000", 2024-09-11T06:51:07.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00054710499999999999 2024-09-11T06:51:07.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:07.036 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591412+0000", 2024-09-11T06:51:07.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2811999999999993e-05 2024-09-11T06:51:07.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.036 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591471+0000", 2024-09-11T06:51:07.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9419000000000001e-05 2024-09-11T06:51:07.036 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.036 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.037 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.037 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:8 3.4 3:3f0de396:::rbd_header.1415bf940188:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:07.037 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.596120+0000", 2024-09-11T06:51:07.037 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.328437687000001, 2024-09-11T06:51:07.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027911020000000002, 2024-09-11T06:51:07.037 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.037 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.038 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:07.038 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.038 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:51:07.038 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:51:07.038 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 8 2024-09-11T06:51:07.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.038 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.039 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596120+0000", 2024-09-11T06:51:07.039 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.039 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.039 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.039 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596120+0000", 2024-09-11T06:51:07.039 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.040 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.040 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.040 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596122+0000", 2024-09-11T06:51:07.040 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.513e-06 2024-09-11T06:51:07.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.040 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.040 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.040 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596130+0000", 2024-09-11T06:51:07.040 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4999999999999999e-06 2024-09-11T06:51:07.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596152+0000", 2024-09-11T06:51:07.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1846000000000001e-05 2024-09-11T06:51:07.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596163+0000", 2024-09-11T06:51:07.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1029999999999999e-05 2024-09-11T06:51:07.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596242+0000", 2024-09-11T06:51:07.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8275000000000005e-05 2024-09-11T06:51:07.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596439+0000", 2024-09-11T06:51:07.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000196999 2024-09-11T06:51:07.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.043 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:07.043 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596755+0000", 2024-09-11T06:51:07.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00031626100000000003 2024-09-11T06:51:07.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.043 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:07.043 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.598043+0000", 2024-09-11T06:51:07.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001288389 2024-09-11T06:51:07.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.043 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:07.043 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.598774+0000", 2024-09-11T06:51:07.044 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00073080499999999995 2024-09-11T06:51:07.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.044 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.044 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:07.044 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.598855+0000", 2024-09-11T06:51:07.044 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0615e-05 2024-09-11T06:51:07.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.044 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.044 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.044 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.598912+0000", 2024-09-11T06:51:07.044 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6869e-05 2024-09-11T06:51:07.045 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.045 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.045 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.045 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.045 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:9 3.0 3:00834abc:::rbd_object_map.1415bf940188:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:07.045 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.599161+0000", 2024-09-11T06:51:07.045 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.32539667, 2024-09-11T06:51:07.045 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002826051, 2024-09-11T06:51:07.045 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.045 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.046 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:07.046 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.046 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:51:07.046 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:51:07.046 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 9 2024-09-11T06:51:07.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.046 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.046 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599161+0000", 2024-09-11T06:51:07.046 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.047 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.047 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599161+0000", 2024-09-11T06:51:07.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.047 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.047 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599163+0000", 2024-09-11T06:51:07.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8700000000000001e-06 2024-09-11T06:51:07.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.048 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.048 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.048 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599174+0000", 2024-09-11T06:51:07.048 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0416e-05 2024-09-11T06:51:07.048 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.048 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.048 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.048 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599202+0000", 2024-09-11T06:51:07.048 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7946e-05 2024-09-11T06:51:07.048 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.048 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.049 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.049 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599216+0000", 2024-09-11T06:51:07.049 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3934000000000001e-05 2024-09-11T06:51:07.049 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.049 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.049 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.049 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599345+0000", 2024-09-11T06:51:07.049 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012954000000000001 2024-09-11T06:51:07.049 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.049 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.049 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.049 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599540+0000", 2024-09-11T06:51:07.050 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019516299999999999 2024-09-11T06:51:07.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.050 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.050 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:07.050 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599796+0000", 2024-09-11T06:51:07.050 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025556500000000002 2024-09-11T06:51:07.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.050 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.050 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:07.050 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.601071+0000", 2024-09-11T06:51:07.050 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001275396 2024-09-11T06:51:07.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:07.051 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.601852+0000", 2024-09-11T06:51:07.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000780524 2024-09-11T06:51:07.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:07.051 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.601925+0000", 2024-09-11T06:51:07.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3574999999999999e-05 2024-09-11T06:51:07.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.052 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.601988+0000", 2024-09-11T06:51:07.052 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2121999999999999e-05 2024-09-11T06:51:07.052 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.052 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.052 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.052 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.052 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5038.0:12 3.4 3:3f0de396:::rbd_header.1415bf940188:head [watch watch cookie 140693532068064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:07.052 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.718024+0000", 2024-09-11T06:51:07.052 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.206533732, 2024-09-11T06:51:07.052 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002724583, 2024-09-11T06:51:07.052 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.053 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.053 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:07.053 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.053 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5038", 2024-09-11T06:51:07.053 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/945190275", 2024-09-11T06:51:07.053 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:51:07.053 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.053 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.053 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.053 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.053 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718024+0000", 2024-09-11T06:51:07.053 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.054 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.054 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.054 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.054 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718024+0000", 2024-09-11T06:51:07.054 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.054 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.054 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.054 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.054 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718027+0000", 2024-09-11T06:51:07.055 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1430000000000001e-06 2024-09-11T06:51:07.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.055 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.055 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718037+0000", 2024-09-11T06:51:07.055 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0253e-05 2024-09-11T06:51:07.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.055 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.055 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718068+0000", 2024-09-11T06:51:07.055 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1050000000000003e-05 2024-09-11T06:51:07.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.056 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.056 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718086+0000", 2024-09-11T06:51:07.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8629000000000001e-05 2024-09-11T06:51:07.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.056 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.056 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718206+0000", 2024-09-11T06:51:07.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000119358 2024-09-11T06:51:07.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.057 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.057 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.057 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718425+0000", 2024-09-11T06:51:07.057 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000218975 2024-09-11T06:51:07.057 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.057 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.057 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:07.057 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718830+0000", 2024-09-11T06:51:07.057 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040538199999999998 2024-09-11T06:51:07.057 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.057 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.057 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:07.058 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.719849+0000", 2024-09-11T06:51:07.058 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010187589999999999 2024-09-11T06:51:07.058 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.058 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.058 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:07.058 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.720548+0000", 2024-09-11T06:51:07.058 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000699112 2024-09-11T06:51:07.058 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.058 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.058 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:07.058 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.720632+0000", 2024-09-11T06:51:07.059 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4314999999999995e-05 2024-09-11T06:51:07.059 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.059 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.059 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.059 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.720749+0000", 2024-09-11T06:51:07.059 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116607 2024-09-11T06:51:07.059 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.059 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.059 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.059 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.059 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.060 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:47 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:07.060 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.553228+0000", 2024-09-11T06:51:07.060 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.371329641999999, 2024-09-11T06:51:07.060 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040683100000000003, 2024-09-11T06:51:07.060 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.060 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.060 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:07.060 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.060 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:07.060 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:07.060 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 47 2024-09-11T06:51:07.061 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.061 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.061 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.061 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.061 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553228+0000", 2024-09-11T06:51:07.061 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.061 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.061 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.061 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.061 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553228+0000", 2024-09-11T06:51:07.061 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.062 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.062 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.062 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.062 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553231+0000", 2024-09-11T06:51:07.062 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3130000000000001e-06 2024-09-11T06:51:07.062 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.062 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.062 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.062 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553242+0000", 2024-09-11T06:51:07.062 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1345999999999999e-05 2024-09-11T06:51:07.062 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.062 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.063 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.063 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553272+0000", 2024-09-11T06:51:07.063 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0179999999999999e-05 2024-09-11T06:51:07.063 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.063 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.063 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.063 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553287+0000", 2024-09-11T06:51:07.063 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4283e-05 2024-09-11T06:51:07.063 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.063 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.063 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.064 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553415+0000", 2024-09-11T06:51:07.064 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000128491 2024-09-11T06:51:07.064 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.064 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.064 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.064 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553651+0000", 2024-09-11T06:51:07.064 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000236328 2024-09-11T06:51:07.064 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.064 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.064 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:07.064 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.554356+0000", 2024-09-11T06:51:07.064 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070436299999999995 2024-09-11T06:51:07.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.065 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.065 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:07.065 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.556064+0000", 2024-09-11T06:51:07.065 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0017084439999999999 2024-09-11T06:51:07.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.065 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.065 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:07.065 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.556993+0000", 2024-09-11T06:51:07.065 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00092912400000000003 2024-09-11T06:51:07.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.066 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.066 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:07.066 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.557173+0000", 2024-09-11T06:51:07.066 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017940400000000001 2024-09-11T06:51:07.066 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.066 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.066 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.066 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.557297+0000", 2024-09-11T06:51:07.066 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000124034 2024-09-11T06:51:07.066 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.066 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.066 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.067 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.067 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.067 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:66 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:07.067 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.589465+0000", 2024-09-11T06:51:07.067 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.335092837000001, 2024-09-11T06:51:07.067 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039855489999999997, 2024-09-11T06:51:07.067 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.067 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.067 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:07.067 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.067 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:07.068 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:07.068 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 66 2024-09-11T06:51:07.068 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.068 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.068 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.068 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.068 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589465+0000", 2024-09-11T06:51:07.068 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.068 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.068 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.068 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.069 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589465+0000", 2024-09-11T06:51:07.069 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.069 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.069 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.069 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.069 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589468+0000", 2024-09-11T06:51:07.069 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3539999999999998e-06 2024-09-11T06:51:07.069 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.069 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.069 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.069 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589480+0000", 2024-09-11T06:51:07.069 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1946e-05 2024-09-11T06:51:07.070 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.070 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.070 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.070 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589512+0000", 2024-09-11T06:51:07.070 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2712999999999997e-05 2024-09-11T06:51:07.070 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.070 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.070 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.070 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589527+0000", 2024-09-11T06:51:07.070 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4686e-05 2024-09-11T06:51:07.070 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.070 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.071 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.071 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589673+0000", 2024-09-11T06:51:07.071 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014630700000000001 2024-09-11T06:51:07.071 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.071 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.071 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.071 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589936+0000", 2024-09-11T06:51:07.071 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026234499999999998 2024-09-11T06:51:07.071 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.071 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.071 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:07.072 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.591001+0000", 2024-09-11T06:51:07.072 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001065023 2024-09-11T06:51:07.072 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.072 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.072 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:07.072 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.592757+0000", 2024-09-11T06:51:07.072 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0017560849999999999 2024-09-11T06:51:07.072 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.072 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.072 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:07.072 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.593297+0000", 2024-09-11T06:51:07.073 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00054007600000000001 2024-09-11T06:51:07.073 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.073 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.073 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:07.073 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.593385+0000", 2024-09-11T06:51:07.073 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8327999999999999e-05 2024-09-11T06:51:07.073 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.073 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.073 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.073 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.593451+0000", 2024-09-11T06:51:07.073 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5685999999999994e-05 2024-09-11T06:51:07.073 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.074 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.074 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.074 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.074 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.074 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:07.075 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:51:07.076 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:07.070+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:07.168 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device detach rbd/testrbdnbd21607 --device-type nbd 2024-09-11T06:51:07.168 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:07.168 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:07.170 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:07.171 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:51:07.173 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:51:07.173 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:07.173 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device detach rbd/testrbdnbd21607 --device-type nbd 2024-09-11T06:51:07.176 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:07.170+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:07.248 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:07.249 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:07.249 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:51:07.249 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:07.263 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:51:07.277 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:07.274+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:07.291 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false get_pid rbd 2024-09-11T06:51:07.291 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd 2024-09-11T06:51:07.291 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:51:07.291 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:51:07.292 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:51:07.292 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd1'\'']/id' 2024-09-11T06:51:07.363 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID= 2024-09-11T06:51:07.363 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n '' 2024-09-11T06:51:07.363 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 1 2024-09-11T06:51:07.363 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:51:07.363 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false _sudo rbd device attach --device /dev/nbd1 rbd/testrbdnbd21607 --device-type nbd 2024-09-11T06:51:07.363 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device attach --device /dev/nbd1 rbd/testrbdnbd21607 --device-type nbd 2024-09-11T06:51:07.363 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:07.363 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:07.364 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:07.364 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:51:07.365 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:51:07.365 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:07.365 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device attach --device /dev/nbd1 rbd/testrbdnbd21607 --device-type nbd 2024-09-11T06:51:07.377 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:07.374+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:07.400 INFO:tasks.workunit.client.0.smithi176.stderr:rbd: could not validate attach request 2024-09-11T06:51:07.400 INFO:tasks.workunit.client.0.smithi176.stderr:rbd: mismatching the image and the device may lead to data corruption 2024-09-11T06:51:07.400 INFO:tasks.workunit.client.0.smithi176.stderr:rbd: must specify --cookie or --force to proceed 2024-09-11T06:51:07.403 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:51:07.403 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' -n 1935c02c-84a6-4710-85dc-6630efac9835 ']' 2024-09-11T06:51:07.403 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device attach --device /dev/nbd1 --cookie 1935c02c-84a6-4710-85dc-6630efac9835 rbd/testrbdnbd21607 --device-type nbd 2024-09-11T06:51:07.403 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:07.403 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:07.403 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:07.404 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:51:07.405 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:51:07.405 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:07.405 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device attach --device /dev/nbd1 --cookie 1935c02c-84a6-4710-85dc-6630efac9835 rbd/testrbdnbd21607 --device-type nbd 2024-09-11T06:51:07.423 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:07.423 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:07.424 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:51:07.424 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:51:07.424 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:07.438 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:51:07.478 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:07.474+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:07.532 INFO:tasks.workunit.client.0.smithi176.stdout:/dev/nbd1 2024-09-11T06:51:07.539 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd 2024-09-11T06:51:07.539 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:51:07.539 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:51:07.539 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:51:07.540 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd1'\'']/id' 2024-09-11T06:51:07.566 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:07.566 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:51:07.566 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:51:07.566 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:51:07.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.566 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:20 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:07.567 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.717149+0000", 2024-09-11T06:51:07.567 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 36.845339936000002, 2024-09-11T06:51:07.567 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002913297, 2024-09-11T06:51:07.567 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.567 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.567 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:07.567 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.567 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:07.567 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:07.567 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 20 2024-09-11T06:51:07.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.567 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717149+0000", 2024-09-11T06:51:07.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717149+0000", 2024-09-11T06:51:07.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717152+0000", 2024-09-11T06:51:07.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.21e-06 2024-09-11T06:51:07.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.569 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.569 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717164+0000", 2024-09-11T06:51:07.569 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1912999999999999e-05 2024-09-11T06:51:07.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.569 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.569 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717191+0000", 2024-09-11T06:51:07.569 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7526e-05 2024-09-11T06:51:07.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.570 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.570 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717206+0000", 2024-09-11T06:51:07.570 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5113000000000001e-05 2024-09-11T06:51:07.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.570 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.570 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.570 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717355+0000", 2024-09-11T06:51:07.570 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014896500000000001 2024-09-11T06:51:07.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.570 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.570 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.570 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.717607+0000", 2024-09-11T06:51:07.570 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000252017 2024-09-11T06:51:07.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.571 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.571 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:07.571 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.718128+0000", 2024-09-11T06:51:07.571 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00052114499999999998 2024-09-11T06:51:07.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.571 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.571 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:07.571 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.719432+0000", 2024-09-11T06:51:07.571 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013041629999999999 2024-09-11T06:51:07.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.571 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.571 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:07.571 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.719864+0000", 2024-09-11T06:51:07.571 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00043115299999999998 2024-09-11T06:51:07.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.572 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:07.572 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.719949+0000", 2024-09-11T06:51:07.572 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4862000000000002e-05 2024-09-11T06:51:07.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.572 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.572 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.720063+0000", 2024-09-11T06:51:07.572 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011423 2024-09-11T06:51:07.572 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.572 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.572 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.573 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.573 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:26 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 140250278007648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:07.573 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:51:07.573 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 36.832856800999998, 2024-09-11T06:51:07.573 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034645140000000001, 2024-09-11T06:51:07.573 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.573 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.573 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:07.573 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.573 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:07.573 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:07.573 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:51:07.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.573 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.574 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.574 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.574 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:51:07.574 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.574 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.574 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.574 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:51:07.574 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.574 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.574 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.574 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729634+0000", 2024-09-11T06:51:07.574 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0470000000000001e-06 2024-09-11T06:51:07.575 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.575 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.575 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.575 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729640+0000", 2024-09-11T06:51:07.575 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8860000000000001e-06 2024-09-11T06:51:07.575 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.575 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.575 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.575 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729654+0000", 2024-09-11T06:51:07.575 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:51:07.575 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.575 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.575 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.575 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729664+0000", 2024-09-11T06:51:07.575 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.594e-06 2024-09-11T06:51:07.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.576 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.576 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.576 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730422+0000", 2024-09-11T06:51:07.576 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000758648 2024-09-11T06:51:07.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.576 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.576 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.576 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730571+0000", 2024-09-11T06:51:07.576 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000148932 2024-09-11T06:51:07.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.576 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.576 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:07.577 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730936+0000", 2024-09-11T06:51:07.577 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036510900000000002 2024-09-11T06:51:07.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.577 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.577 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:07.577 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732159+0000", 2024-09-11T06:51:07.577 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001222302 2024-09-11T06:51:07.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.577 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.577 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:07.577 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732763+0000", 2024-09-11T06:51:07.577 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00060421599999999998 2024-09-11T06:51:07.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.577 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.577 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:07.578 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732916+0000", 2024-09-11T06:51:07.578 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015264 2024-09-11T06:51:07.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.578 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.578 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.578 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733097+0000", 2024-09-11T06:51:07.578 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000181504 2024-09-11T06:51:07.578 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.578 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.578 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.578 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.578 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:27 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:07.579 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:51:07.579 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 36.831882899, 2024-09-11T06:51:07.579 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003072852, 2024-09-11T06:51:07.579 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.579 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.579 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:07.579 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.579 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:07.579 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:07.579 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27 2024-09-11T06:51:07.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.579 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.579 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.579 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.580 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:51:07.580 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.580 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.580 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.580 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.580 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:51:07.580 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.580 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.580 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.580 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.580 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730608+0000", 2024-09-11T06:51:07.580 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8369999999999999e-06 2024-09-11T06:51:07.580 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.581 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.581 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.581 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730617+0000", 2024-09-11T06:51:07.581 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1859999999999992e-06 2024-09-11T06:51:07.581 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.581 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.581 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.581 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730636+0000", 2024-09-11T06:51:07.581 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9296e-05 2024-09-11T06:51:07.581 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.581 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.581 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.581 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730674+0000", 2024-09-11T06:51:07.581 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8609000000000003e-05 2024-09-11T06:51:07.582 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.582 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.582 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.582 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731496+0000", 2024-09-11T06:51:07.582 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00082112299999999999 2024-09-11T06:51:07.582 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.582 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.582 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:07.582 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731722+0000", 2024-09-11T06:51:07.582 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022651400000000001 2024-09-11T06:51:07.582 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.582 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.582 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.583 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733215+0000", 2024-09-11T06:51:07.583 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001493257 2024-09-11T06:51:07.583 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.583 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.583 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.583 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733452+0000", 2024-09-11T06:51:07.583 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023631600000000001 2024-09-11T06:51:07.583 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.583 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.583 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.583 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733679+0000", 2024-09-11T06:51:07.583 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022771400000000001 2024-09-11T06:51:07.583 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.584 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.584 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.584 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.584 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.584 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:73 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:07.584 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:51:07.584 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 36.831798614999997, 2024-09-11T06:51:07.584 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033454439999999999, 2024-09-11T06:51:07.584 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.584 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.584 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:07.584 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.584 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:51:07.585 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:51:07.585 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73 2024-09-11T06:51:07.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.585 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.585 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.585 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.585 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:51:07.585 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.585 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.585 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.585 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:51:07.585 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.586 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.586 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.586 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730694+0000", 2024-09-11T06:51:07.586 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5099999999999999e-06 2024-09-11T06:51:07.586 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.586 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.586 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.586 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730708+0000", 2024-09-11T06:51:07.586 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3237e-05 2024-09-11T06:51:07.586 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.586 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.586 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.586 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730738+0000", 2024-09-11T06:51:07.587 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0688999999999997e-05 2024-09-11T06:51:07.587 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.587 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.587 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.587 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730759+0000", 2024-09-11T06:51:07.587 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0818999999999998e-05 2024-09-11T06:51:07.587 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.587 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.587 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.587 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731841+0000", 2024-09-11T06:51:07.587 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010821819999999999 2024-09-11T06:51:07.587 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.587 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.588 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:07.588 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732092+0000", 2024-09-11T06:51:07.588 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025086299999999999 2024-09-11T06:51:07.588 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.588 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.588 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.588 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733730+0000", 2024-09-11T06:51:07.588 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0016374600000000001 2024-09-11T06:51:07.588 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.588 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.588 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.588 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733866+0000", 2024-09-11T06:51:07.588 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000136666 2024-09-11T06:51:07.589 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.589 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.589 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.589 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.734036+0000", 2024-09-11T06:51:07.589 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000170018 2024-09-11T06:51:07.589 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.589 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.589 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.589 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.589 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.589 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:18 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:07.589 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:51:07.589 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 36.688381425999999, 2024-09-11T06:51:07.590 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0031646579999999999, 2024-09-11T06:51:07.590 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.590 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.590 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:07.590 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.590 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:07.590 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:07.590 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18 2024-09-11T06:51:07.590 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.590 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.590 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.590 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.590 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:51:07.590 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.591 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.591 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.591 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.591 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:51:07.591 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.591 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.591 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.591 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.591 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874109+0000", 2024-09-11T06:51:07.591 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.426e-06 2024-09-11T06:51:07.591 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.591 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.591 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.591 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874118+0000", 2024-09-11T06:51:07.592 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6929999999999996e-06 2024-09-11T06:51:07.592 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.592 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.592 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.592 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874134+0000", 2024-09-11T06:51:07.592 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5503e-05 2024-09-11T06:51:07.592 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.592 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.592 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.592 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874144+0000", 2024-09-11T06:51:07.592 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0207000000000001e-05 2024-09-11T06:51:07.592 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.592 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.592 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.593 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874545+0000", 2024-09-11T06:51:07.593 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040120800000000001 2024-09-11T06:51:07.593 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.593 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.593 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.593 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874686+0000", 2024-09-11T06:51:07.593 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014128599999999999 2024-09-11T06:51:07.593 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.593 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.593 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:07.593 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875023+0000", 2024-09-11T06:51:07.593 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033706600000000001 2024-09-11T06:51:07.593 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.593 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.594 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:07.594 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.876217+0000", 2024-09-11T06:51:07.594 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011938999999999999 2024-09-11T06:51:07.594 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.594 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.594 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:07.594 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877042+0000", 2024-09-11T06:51:07.594 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00082499899999999998 2024-09-11T06:51:07.594 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.594 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.594 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:07.594 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877128+0000", 2024-09-11T06:51:07.594 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5903999999999996e-05 2024-09-11T06:51:07.594 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.595 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.595 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.595 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877273+0000", 2024-09-11T06:51:07.595 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144466 2024-09-11T06:51:07.595 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.595 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.595 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.595 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.595 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.595 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:24 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:07.595 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:51:07.595 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 36.673875719999998, 2024-09-11T06:51:07.595 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002980271, 2024-09-11T06:51:07.596 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.596 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.596 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:07.596 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.596 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:07.596 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:07.596 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 24 2024-09-11T06:51:07.596 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.596 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.596 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.596 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.596 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:51:07.596 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.597 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.597 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.597 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.597 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:51:07.597 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.597 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.597 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.597 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.597 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888616+0000", 2024-09-11T06:51:07.597 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0030000000000001e-06 2024-09-11T06:51:07.597 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.597 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.597 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.597 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888626+0000", 2024-09-11T06:51:07.598 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0502999999999999e-05 2024-09-11T06:51:07.598 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.598 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.598 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.598 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888655+0000", 2024-09-11T06:51:07.598 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8419999999999999e-05 2024-09-11T06:51:07.598 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.598 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.598 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.598 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888688+0000", 2024-09-11T06:51:07.598 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3212e-05 2024-09-11T06:51:07.598 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.598 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.599 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.599 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888829+0000", 2024-09-11T06:51:07.599 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014114299999999999 2024-09-11T06:51:07.599 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.599 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.599 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.599 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.889082+0000", 2024-09-11T06:51:07.599 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025279499999999999 2024-09-11T06:51:07.599 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.599 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.599 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:07.599 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.889577+0000", 2024-09-11T06:51:07.599 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00049536299999999997 2024-09-11T06:51:07.599 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.600 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.600 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:07.600 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.890811+0000", 2024-09-11T06:51:07.600 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001233858 2024-09-11T06:51:07.600 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.600 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.600 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:07.600 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891368+0000", 2024-09-11T06:51:07.600 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00055677400000000001 2024-09-11T06:51:07.600 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.600 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.600 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:07.600 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891456+0000", 2024-09-11T06:51:07.600 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8246999999999997e-05 2024-09-11T06:51:07.601 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.601 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.601 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.601 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891594+0000", 2024-09-11T06:51:07.601 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013795300000000001 2024-09-11T06:51:07.601 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.601 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.601 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.601 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.601 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.601 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:26 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 139740787512064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:07.601 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:51:07.601 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 36.670173075999998, 2024-09-11T06:51:07.602 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003613733, 2024-09-11T06:51:07.602 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.602 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.602 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:07.602 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.602 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:07.602 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:07.602 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:51:07.602 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.602 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.602 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.602 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.602 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:51:07.602 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.603 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.603 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.603 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.603 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:51:07.603 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.603 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.603 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.603 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.603 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892318+0000", 2024-09-11T06:51:07.603 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.623e-06 2024-09-11T06:51:07.603 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.603 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.603 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.603 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892325+0000", 2024-09-11T06:51:07.604 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:51:07.604 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.604 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.604 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.604 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892346+0000", 2024-09-11T06:51:07.604 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1246e-05 2024-09-11T06:51:07.604 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.604 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.604 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.604 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892362+0000", 2024-09-11T06:51:07.604 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5409000000000001e-05 2024-09-11T06:51:07.604 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.604 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.604 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.605 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892883+0000", 2024-09-11T06:51:07.605 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00052087599999999998 2024-09-11T06:51:07.605 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.605 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.605 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.605 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893123+0000", 2024-09-11T06:51:07.605 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000240769 2024-09-11T06:51:07.605 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.605 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.605 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:07.605 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893526+0000", 2024-09-11T06:51:07.605 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040300200000000002 2024-09-11T06:51:07.605 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.605 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.606 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:07.606 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894820+0000", 2024-09-11T06:51:07.606 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012934159999999999 2024-09-11T06:51:07.606 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.606 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.606 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:07.606 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895608+0000", 2024-09-11T06:51:07.606 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00078788100000000004 2024-09-11T06:51:07.606 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.606 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.606 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:07.606 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895724+0000", 2024-09-11T06:51:07.606 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011595999999999999 2024-09-11T06:51:07.606 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.606 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.607 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.607 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895930+0000", 2024-09-11T06:51:07.607 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000206391 2024-09-11T06:51:07.607 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.607 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.607 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.607 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.607 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.607 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:133 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:07.607 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:51:07.607 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 36.669375059000004, 2024-09-11T06:51:07.607 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035017360000000001, 2024-09-11T06:51:07.607 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.607 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.608 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:07.608 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.608 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:51:07.608 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:51:07.608 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 133 2024-09-11T06:51:07.608 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.608 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.608 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.608 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.608 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:51:07.608 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.608 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.608 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.608 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.609 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:51:07.609 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.609 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.609 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.609 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.609 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893116+0000", 2024-09-11T06:51:07.609 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1629999999999999e-06 2024-09-11T06:51:07.609 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.609 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.609 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.609 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893128+0000", 2024-09-11T06:51:07.609 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.11e-05 2024-09-11T06:51:07.609 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.610 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.610 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.610 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893156+0000", 2024-09-11T06:51:07.610 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8266e-05 2024-09-11T06:51:07.610 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.610 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.610 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.610 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893171+0000", 2024-09-11T06:51:07.610 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5426000000000001e-05 2024-09-11T06:51:07.610 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.610 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.610 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.610 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894561+0000", 2024-09-11T06:51:07.610 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013899959999999999 2024-09-11T06:51:07.611 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.611 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.611 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:07.611 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894781+0000", 2024-09-11T06:51:07.611 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021987100000000001 2024-09-11T06:51:07.611 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.611 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.611 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.611 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896349+0000", 2024-09-11T06:51:07.611 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015681779999999999 2024-09-11T06:51:07.611 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.611 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.611 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.612 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896458+0000", 2024-09-11T06:51:07.612 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000108187 2024-09-11T06:51:07.612 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.612 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.612 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.612 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896616+0000", 2024-09-11T06:51:07.612 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015854899999999999 2024-09-11T06:51:07.612 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.612 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.612 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.612 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.612 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.612 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:27 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:07.612 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:51:07.613 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 36.669363679, 2024-09-11T06:51:07.613 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003151989, 2024-09-11T06:51:07.613 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.613 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.613 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:07.613 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.613 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:07.613 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:07.613 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27 2024-09-11T06:51:07.613 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.613 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.613 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.613 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.614 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:51:07.614 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.614 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.614 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.614 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.614 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:51:07.614 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.614 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.614 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.614 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.614 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893128+0000", 2024-09-11T06:51:07.614 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.013e-06 2024-09-11T06:51:07.614 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.614 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.615 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.615 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893136+0000", 2024-09-11T06:51:07.615 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4190000000000002e-06 2024-09-11T06:51:07.615 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.615 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.615 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.615 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893158+0000", 2024-09-11T06:51:07.615 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1506999999999999e-05 2024-09-11T06:51:07.615 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.615 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.615 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.615 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893170+0000", 2024-09-11T06:51:07.615 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2741999999999999e-05 2024-09-11T06:51:07.615 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.616 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.616 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.616 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894255+0000", 2024-09-11T06:51:07.616 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001084453 2024-09-11T06:51:07.616 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.616 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.616 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:07.616 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894461+0000", 2024-09-11T06:51:07.616 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00020586700000000001 2024-09-11T06:51:07.616 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.616 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.616 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.616 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896033+0000", 2024-09-11T06:51:07.616 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001572092 2024-09-11T06:51:07.616 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.617 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.617 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.617 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896149+0000", 2024-09-11T06:51:07.617 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116636 2024-09-11T06:51:07.617 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.617 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.617 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.617 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896278+0000", 2024-09-11T06:51:07.617 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012826 2024-09-11T06:51:07.617 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.617 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.617 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.617 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.617 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.618 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:176 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch reconnect cookie 139735284585312 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:07.618 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:51:07.618 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 36.637996860000001, 2024-09-11T06:51:07.618 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033620960000000002, 2024-09-11T06:51:07.618 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.618 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.618 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:07.618 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.618 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:51:07.618 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:51:07.618 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 176 2024-09-11T06:51:07.618 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.618 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.618 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.619 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.619 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:51:07.619 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.619 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.619 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.619 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.619 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:51:07.619 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.619 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.619 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.619 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.619 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924494+0000", 2024-09-11T06:51:07.619 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.967e-06 2024-09-11T06:51:07.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.620 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.620 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.620 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924501+0000", 2024-09-11T06:51:07.620 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2659999999999998e-06 2024-09-11T06:51:07.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.620 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.620 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.620 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924520+0000", 2024-09-11T06:51:07.620 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9038999999999998e-05 2024-09-11T06:51:07.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.620 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.620 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.621 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924535+0000", 2024-09-11T06:51:07.621 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5359999999999999e-05 2024-09-11T06:51:07.621 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.621 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.621 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.621 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.925544+0000", 2024-09-11T06:51:07.621 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010085109999999999 2024-09-11T06:51:07.621 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.621 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.621 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.621 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.925671+0000", 2024-09-11T06:51:07.621 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012756599999999999 2024-09-11T06:51:07.621 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.622 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.622 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.622 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.927855+0000", 2024-09-11T06:51:07.622 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0021833870000000002 2024-09-11T06:51:07.622 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.622 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.622 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.622 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.622 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4828.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch watch cookie 140587969811264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-11T06:51:07.622 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:51:07.622 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.316293902000002, 2024-09-11T06:51:07.622 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032778859999999998, 2024-09-11T06:51:07.622 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.623 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.623 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:07.623 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.623 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4828", 2024-09-11T06:51:07.623 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2410288650", 2024-09-11T06:51:07.623 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:51:07.623 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.623 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.623 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.623 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.623 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:51:07.623 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.624 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.624 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.624 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.624 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:51:07.624 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.624 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.624 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.624 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.624 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246198+0000", 2024-09-11T06:51:07.624 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.667e-06 2024-09-11T06:51:07.624 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.624 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.624 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.624 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246209+0000", 2024-09-11T06:51:07.625 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0409e-05 2024-09-11T06:51:07.625 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.625 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.625 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.625 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246241+0000", 2024-09-11T06:51:07.625 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2592999999999998e-05 2024-09-11T06:51:07.625 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.625 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.625 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.625 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246260+0000", 2024-09-11T06:51:07.625 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9343000000000001e-05 2024-09-11T06:51:07.625 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.625 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.625 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.626 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246466+0000", 2024-09-11T06:51:07.626 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000205701 2024-09-11T06:51:07.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.626 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.626 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.626 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246713+0000", 2024-09-11T06:51:07.626 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024646999999999999 2024-09-11T06:51:07.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.626 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.626 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:07.626 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.247133+0000", 2024-09-11T06:51:07.626 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000420536 2024-09-11T06:51:07.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.626 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.627 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:07.627 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.248541+0000", 2024-09-11T06:51:07.627 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001407801 2024-09-11T06:51:07.627 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.627 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.627 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:07.627 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249241+0000", 2024-09-11T06:51:07.627 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070043500000000001 2024-09-11T06:51:07.627 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.627 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.627 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:07.627 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249326+0000", 2024-09-11T06:51:07.627 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5037999999999995e-05 2024-09-11T06:51:07.627 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.628 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.628 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.628 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249473+0000", 2024-09-11T06:51:07.628 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014689299999999999 2024-09-11T06:51:07.628 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.628 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.628 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.628 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.628 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.628 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5313.0:25 2.1 2:967e1786:ns::rbd_object_map.11adc335555f.0000000000000004:head [writefull 0~38] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:51:07.628 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.256788+0000", 2024-09-11T06:51:07.628 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 28.305700903000002, 2024-09-11T06:51:07.628 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0030446449999999999, 2024-09-11T06:51:07.629 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.629 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.629 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:07.629 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.629 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5313", 2024-09-11T06:51:07.629 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/624314229", 2024-09-11T06:51:07.629 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 25 2024-09-11T06:51:07.629 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.629 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.629 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.629 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.629 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256788+0000", 2024-09-11T06:51:07.629 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.629 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.630 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.630 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.630 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256788+0000", 2024-09-11T06:51:07.630 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.630 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.630 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.630 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.630 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256791+0000", 2024-09-11T06:51:07.630 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5770000000000001e-06 2024-09-11T06:51:07.630 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.630 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.630 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.630 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256802+0000", 2024-09-11T06:51:07.630 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0742e-05 2024-09-11T06:51:07.631 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.631 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.631 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.631 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256823+0000", 2024-09-11T06:51:07.631 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0999999999999999e-05 2024-09-11T06:51:07.631 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.631 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.631 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.631 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256841+0000", 2024-09-11T06:51:07.631 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8099000000000001e-05 2024-09-11T06:51:07.631 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.631 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.631 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.632 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256988+0000", 2024-09-11T06:51:07.632 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014756400000000001 2024-09-11T06:51:07.632 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.632 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.632 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.632 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.257234+0000", 2024-09-11T06:51:07.632 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024524399999999999 2024-09-11T06:51:07.632 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.632 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.632 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:07.632 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.257446+0000", 2024-09-11T06:51:07.632 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000212241 2024-09-11T06:51:07.632 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.632 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.632 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:07.633 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259292+0000", 2024-09-11T06:51:07.633 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001845937 2024-09-11T06:51:07.633 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.633 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.633 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:07.633 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259681+0000", 2024-09-11T06:51:07.633 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00038940699999999998 2024-09-11T06:51:07.633 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.633 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.633 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:07.633 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259770+0000", 2024-09-11T06:51:07.633 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8385000000000002e-05 2024-09-11T06:51:07.633 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.633 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.633 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.634 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259833+0000", 2024-09-11T06:51:07.634 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3448999999999995e-05 2024-09-11T06:51:07.634 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.634 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.634 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.634 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.634 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.634 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5328.0:14 2.1 2:967e1786:ns::rbd_object_map.11adc335555f.0000000000000004:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:51:07.634 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.396257+0000", 2024-09-11T06:51:07.634 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 28.166232453999999, 2024-09-11T06:51:07.634 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032125370000000001, 2024-09-11T06:51:07.634 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.634 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.634 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:07.634 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.635 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5328", 2024-09-11T06:51:07.635 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2386493698", 2024-09-11T06:51:07.635 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 14 2024-09-11T06:51:07.635 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.635 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.635 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.635 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.635 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396257+0000", 2024-09-11T06:51:07.635 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.635 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.635 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.635 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.635 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396257+0000", 2024-09-11T06:51:07.635 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.636 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.636 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.636 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.636 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396259+0000", 2024-09-11T06:51:07.636 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4830000000000002e-06 2024-09-11T06:51:07.636 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.636 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.636 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.636 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396271+0000", 2024-09-11T06:51:07.636 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.115e-05 2024-09-11T06:51:07.636 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.636 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.636 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.636 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396292+0000", 2024-09-11T06:51:07.636 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1761999999999999e-05 2024-09-11T06:51:07.637 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.637 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.637 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.637 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396314+0000", 2024-09-11T06:51:07.637 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1982999999999999e-05 2024-09-11T06:51:07.637 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.637 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.637 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.637 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396409+0000", 2024-09-11T06:51:07.637 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4250000000000001e-05 2024-09-11T06:51:07.637 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.637 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.637 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.637 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396464+0000", 2024-09-11T06:51:07.637 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5884000000000002e-05 2024-09-11T06:51:07.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.638 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.638 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.638 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.399469+0000", 2024-09-11T06:51:07.638 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0030050250000000001 2024-09-11T06:51:07.638 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.638 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.638 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.638 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.638 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:6 3.5 3:b2cc3336:::rbd_id.testrbdnbd21607:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:07.638 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.591899+0000", 2024-09-11T06:51:07.638 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.970589953000001, 2024-09-11T06:51:07.638 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032382999999999999, 2024-09-11T06:51:07.638 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.639 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.639 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:07.639 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.639 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:51:07.639 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:51:07.639 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 6 2024-09-11T06:51:07.639 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.639 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.639 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.639 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.639 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591899+0000", 2024-09-11T06:51:07.639 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.639 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.640 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.640 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.640 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591899+0000", 2024-09-11T06:51:07.640 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.640 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.640 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.640 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591902+0000", 2024-09-11T06:51:07.640 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0770000000000002e-06 2024-09-11T06:51:07.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.640 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.640 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.640 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591913+0000", 2024-09-11T06:51:07.641 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1182999999999999e-05 2024-09-11T06:51:07.641 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.641 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.641 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.641 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591940+0000", 2024-09-11T06:51:07.641 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7233000000000002e-05 2024-09-11T06:51:07.641 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.641 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.641 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.641 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591954+0000", 2024-09-11T06:51:07.642 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4503e-05 2024-09-11T06:51:07.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.642 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.642 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.642 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.592071+0000", 2024-09-11T06:51:07.642 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116731 2024-09-11T06:51:07.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.642 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.642 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.642 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.592274+0000", 2024-09-11T06:51:07.642 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00020320900000000001 2024-09-11T06:51:07.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.643 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.643 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 3", 2024-09-11T06:51:07.643 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.592576+0000", 2024-09-11T06:51:07.643 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030179000000000002 2024-09-11T06:51:07.643 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.643 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.643 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:07.643 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.594072+0000", 2024-09-11T06:51:07.643 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001495496 2024-09-11T06:51:07.643 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.643 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.643 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:07.643 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.594945+0000", 2024-09-11T06:51:07.643 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00087379799999999996 2024-09-11T06:51:07.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.644 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.644 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:07.644 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.595023+0000", 2024-09-11T06:51:07.644 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7751999999999997e-05 2024-09-11T06:51:07.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.644 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.644 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.644 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.595138+0000", 2024-09-11T06:51:07.644 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000114528 2024-09-11T06:51:07.644 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.644 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.644 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.645 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.645 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.645 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:44 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:07.645 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.540846+0000", 2024-09-11T06:51:07.645 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.021643821000001, 2024-09-11T06:51:07.645 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032138639999999999, 2024-09-11T06:51:07.645 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.645 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.645 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:07.645 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.645 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:07.645 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:07.645 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 44 2024-09-11T06:51:07.646 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.646 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.646 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.646 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.646 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540846+0000", 2024-09-11T06:51:07.646 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.646 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.646 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.646 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.646 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540846+0000", 2024-09-11T06:51:07.646 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.646 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.646 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.647 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.647 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540848+0000", 2024-09-11T06:51:07.647 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5469999999999999e-06 2024-09-11T06:51:07.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.647 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.647 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.647 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540863+0000", 2024-09-11T06:51:07.647 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4613000000000001e-05 2024-09-11T06:51:07.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.647 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.647 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.647 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540895+0000", 2024-09-11T06:51:07.647 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2153000000000002e-05 2024-09-11T06:51:07.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.648 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.648 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.648 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540912+0000", 2024-09-11T06:51:07.648 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7405999999999999e-05 2024-09-11T06:51:07.648 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.648 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.648 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.648 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.541111+0000", 2024-09-11T06:51:07.648 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019849599999999999 2024-09-11T06:51:07.648 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.648 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.648 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.648 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.541269+0000", 2024-09-11T06:51:07.649 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015792399999999999 2024-09-11T06:51:07.649 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.649 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.649 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:07.649 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.541853+0000", 2024-09-11T06:51:07.649 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00058443799999999995 2024-09-11T06:51:07.649 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.649 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.649 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:07.649 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.543133+0000", 2024-09-11T06:51:07.649 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012797850000000001 2024-09-11T06:51:07.649 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.649 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.649 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:07.650 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.543905+0000", 2024-09-11T06:51:07.650 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000771655 2024-09-11T06:51:07.650 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.650 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.650 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:07.650 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.543993+0000", 2024-09-11T06:51:07.650 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8168e-05 2024-09-11T06:51:07.650 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.650 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.650 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.650 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.544059+0000", 2024-09-11T06:51:07.650 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6679000000000003e-05 2024-09-11T06:51:07.650 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.651 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.651 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.651 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.651 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.651 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5553.0:24 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 139650593197904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:07.651 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.566567+0000", 2024-09-11T06:51:07.651 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.995922551, 2024-09-11T06:51:07.651 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034350790000000002, 2024-09-11T06:51:07.651 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.651 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.651 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:07.651 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.651 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5553", 2024-09-11T06:51:07.651 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3590511604", 2024-09-11T06:51:07.652 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 24 2024-09-11T06:51:07.652 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.652 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.652 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.652 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.652 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566567+0000", 2024-09-11T06:51:07.652 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.652 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.652 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.652 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.652 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566567+0000", 2024-09-11T06:51:07.652 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.652 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.653 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.653 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.653 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566569+0000", 2024-09-11T06:51:07.653 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.973e-06 2024-09-11T06:51:07.653 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.653 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.653 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.653 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566578+0000", 2024-09-11T06:51:07.653 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8170000000000005e-06 2024-09-11T06:51:07.653 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.653 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.653 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.653 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566602+0000", 2024-09-11T06:51:07.654 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4596e-05 2024-09-11T06:51:07.654 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.654 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.654 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.654 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566619+0000", 2024-09-11T06:51:07.654 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7099999999999999e-05 2024-09-11T06:51:07.654 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.654 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.654 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.654 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.567321+0000", 2024-09-11T06:51:07.654 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070168200000000002 2024-09-11T06:51:07.654 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.654 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.654 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.654 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.567454+0000", 2024-09-11T06:51:07.655 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013286400000000001 2024-09-11T06:51:07.655 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.655 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.655 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:07.655 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.567674+0000", 2024-09-11T06:51:07.655 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000219679 2024-09-11T06:51:07.655 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.655 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.655 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:07.655 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.569014+0000", 2024-09-11T06:51:07.655 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013404129999999999 2024-09-11T06:51:07.655 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.655 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.655 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:07.656 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.569634+0000", 2024-09-11T06:51:07.656 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00062029200000000004 2024-09-11T06:51:07.656 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.656 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.656 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:07.656 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.569805+0000", 2024-09-11T06:51:07.656 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000170366 2024-09-11T06:51:07.656 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.656 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.656 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.656 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.570002+0000", 2024-09-11T06:51:07.656 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019729700000000001 2024-09-11T06:51:07.656 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.657 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.657 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.657 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.657 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.657 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:82 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch reconnect cookie 139888024456048 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:07.657 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.690185+0000", 2024-09-11T06:51:07.657 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.872304548999999, 2024-09-11T06:51:07.657 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0045884799999999998, 2024-09-11T06:51:07.657 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.657 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.657 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:07.657 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.657 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:07.657 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:07.658 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 82 2024-09-11T06:51:07.658 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.658 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.658 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.658 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.658 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690185+0000", 2024-09-11T06:51:07.658 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.658 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.658 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.658 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.658 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690185+0000", 2024-09-11T06:51:07.658 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.658 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.658 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.658 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.658 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690186+0000", 2024-09-11T06:51:07.659 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8400000000000002e-07 2024-09-11T06:51:07.659 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.659 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.659 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.659 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690190+0000", 2024-09-11T06:51:07.659 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5499999999999996e-06 2024-09-11T06:51:07.659 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.659 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.659 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.659 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690203+0000", 2024-09-11T06:51:07.659 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2228999999999999e-05 2024-09-11T06:51:07.659 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.659 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.660 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.660 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690228+0000", 2024-09-11T06:51:07.660 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5587000000000001e-05 2024-09-11T06:51:07.660 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.660 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.660 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.660 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.691298+0000", 2024-09-11T06:51:07.660 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001070218 2024-09-11T06:51:07.660 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.660 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.660 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.660 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.691511+0000", 2024-09-11T06:51:07.660 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021301299999999999 2024-09-11T06:51:07.660 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.661 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.661 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.661 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.694773+0000", 2024-09-11T06:51:07.661 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003261899 2024-09-11T06:51:07.661 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.661 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.661 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.661 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.661 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.661 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:83 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify cookie 139887621709056] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:07.661 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.690242+0000", 2024-09-11T06:51:07.661 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.872247377000001, 2024-09-11T06:51:07.661 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003040067, 2024-09-11T06:51:07.661 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.662 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.662 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:07.662 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.662 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:07.662 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:07.662 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 83 2024-09-11T06:51:07.662 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.662 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.662 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.662 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.662 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690242+0000", 2024-09-11T06:51:07.662 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.662 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.662 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.663 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.663 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690242+0000", 2024-09-11T06:51:07.663 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.663 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.663 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.663 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.663 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690243+0000", 2024-09-11T06:51:07.663 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0640000000000001e-06 2024-09-11T06:51:07.663 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.663 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.663 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.663 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690247+0000", 2024-09-11T06:51:07.663 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3760000000000001e-06 2024-09-11T06:51:07.663 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.663 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.663 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.664 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690258+0000", 2024-09-11T06:51:07.664 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0874e-05 2024-09-11T06:51:07.664 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.664 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.664 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.664 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690266+0000", 2024-09-11T06:51:07.664 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5429999999999996e-06 2024-09-11T06:51:07.664 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.664 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.664 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.664 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.692564+0000", 2024-09-11T06:51:07.664 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0022977010000000001 2024-09-11T06:51:07.664 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.664 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.665 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.665 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.692882+0000", 2024-09-11T06:51:07.665 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00031862699999999998 2024-09-11T06:51:07.665 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.665 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.665 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.665 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693282+0000", 2024-09-11T06:51:07.665 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000399882 2024-09-11T06:51:07.665 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.665 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.665 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.665 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.665 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.665 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:84 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify cookie 139887823033168] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:07.666 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:51:07.666 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.872212773000001, 2024-09-11T06:51:07.666 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035622620000000001, 2024-09-11T06:51:07.666 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.666 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.666 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:07.666 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.666 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:07.666 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:07.666 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 84 2024-09-11T06:51:07.666 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.666 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.666 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.666 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.667 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:51:07.667 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.667 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.667 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.667 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.667 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:51:07.667 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.667 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.667 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.667 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.667 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:51:07.667 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5300000000000003e-07 2024-09-11T06:51:07.667 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.668 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.668 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.668 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690281+0000", 2024-09-11T06:51:07.668 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9330000000000003e-06 2024-09-11T06:51:07.668 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.668 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.668 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.668 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690291+0000", 2024-09-11T06:51:07.668 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7359999999999996e-06 2024-09-11T06:51:07.668 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.668 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.668 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.668 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690298+0000", 2024-09-11T06:51:07.669 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3699999999999997e-06 2024-09-11T06:51:07.669 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.669 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.669 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.669 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693380+0000", 2024-09-11T06:51:07.669 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0030811060000000001 2024-09-11T06:51:07.669 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.669 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.669 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.669 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693599+0000", 2024-09-11T06:51:07.669 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021947199999999999 2024-09-11T06:51:07.669 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.669 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.669 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.670 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693839+0000", 2024-09-11T06:51:07.670 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023989200000000001 2024-09-11T06:51:07.670 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.670 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.670 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.670 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.670 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.670 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5098.0:25 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 140617967560800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:07.670 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.697727+0000", 2024-09-11T06:51:07.670 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.864762197000001, 2024-09-11T06:51:07.670 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035341330000000001, 2024-09-11T06:51:07.670 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:07.670 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:07.670 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:07.671 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:07.671 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5098", 2024-09-11T06:51:07.671 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/844541331", 2024-09-11T06:51:07.671 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 25 2024-09-11T06:51:07.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.671 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:07.671 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.671 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:07.671 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697727+0000", 2024-09-11T06:51:07.671 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.671 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.671 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:07.671 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697727+0000", 2024-09-11T06:51:07.672 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:07.672 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.672 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.672 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:07.672 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697729+0000", 2024-09-11T06:51:07.672 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.33e-06 2024-09-11T06:51:07.672 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.672 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.672 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:07.672 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697734+0000", 2024-09-11T06:51:07.672 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6590000000000004e-06 2024-09-11T06:51:07.672 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.672 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.673 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:07.673 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697749+0000", 2024-09-11T06:51:07.673 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4867e-05 2024-09-11T06:51:07.673 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.673 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.673 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:07.673 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697759+0000", 2024-09-11T06:51:07.673 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0073e-05 2024-09-11T06:51:07.673 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.673 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.673 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:07.673 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.698715+0000", 2024-09-11T06:51:07.673 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00095545799999999996 2024-09-11T06:51:07.673 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.674 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.674 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:07.674 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.698850+0000", 2024-09-11T06:51:07.674 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000135814 2024-09-11T06:51:07.674 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.674 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.674 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:07.674 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.699085+0000", 2024-09-11T06:51:07.674 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023507199999999999 2024-09-11T06:51:07.674 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.674 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.674 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:07.674 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.700258+0000", 2024-09-11T06:51:07.675 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011726600000000001 2024-09-11T06:51:07.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.675 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.675 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:07.675 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.701061+0000", 2024-09-11T06:51:07.675 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00080256100000000003 2024-09-11T06:51:07.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.675 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.675 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:07.675 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.701144+0000", 2024-09-11T06:51:07.675 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3188000000000004e-05 2024-09-11T06:51:07.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:07.675 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:07.675 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:07.675 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.701261+0000", 2024-09-11T06:51:07.675 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000117451 2024-09-11T06:51:07.676 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.676 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.676 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.676 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:07.676 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:07.676 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:07.677 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=24095 2024-09-11T06:51:07.677 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 24095 2024-09-11T06:51:07.677 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 24095 -C rbd-nbd 2024-09-11T06:51:07.678 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:51:07.678 INFO:tasks.workunit.client.0.smithi176.stdout: 24095 ? 00:00:00 rbd-nbd 2024-09-11T06:51:07.679 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:51:07.679 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device detach /dev/nbd1 --device-type nbd 2024-09-11T06:51:07.680 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:07.680 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:07.680 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:07.680 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:51:07.680 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:51:07.680 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:07.680 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device detach /dev/nbd1 --device-type nbd 2024-09-11T06:51:07.680 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:07.674+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:07.729 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false get_pid rbd 2024-09-11T06:51:07.729 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd 2024-09-11T06:51:07.729 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:51:07.729 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:51:07.730 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:51:07.730 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd1'\'']/id' 2024-09-11T06:51:07.778 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:07.774+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:07.801 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID= 2024-09-11T06:51:07.801 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n '' 2024-09-11T06:51:07.801 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 1 2024-09-11T06:51:07.801 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:51:07.801 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' -n 1935c02c-84a6-4710-85dc-6630efac9835 ']' 2024-09-11T06:51:07.801 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device attach --device /dev/nbd1 --cookie 1935c02c-84a6-4710-85dc-6630efac9835 rbd/testrbdnbd21607 --device-type nbd 2024-09-11T06:51:07.801 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:07.802 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:07.802 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:07.802 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:51:07.803 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:51:07.804 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:07.804 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device attach --device /dev/nbd1 --cookie 1935c02c-84a6-4710-85dc-6630efac9835 rbd/testrbdnbd21607 --device-type nbd 2024-09-11T06:51:07.810 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:07.811 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:07.811 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:51:07.811 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:07.824 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:51:07.879 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:07.874+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:07.909 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T06:51:07.937 INFO:tasks.workunit.client.0.smithi176.stdout:/dev/nbd1 2024-09-11T06:51:07.944 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd 2024-09-11T06:51:07.944 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:51:07.944 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:51:07.944 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:51:07.945 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd1'\'']/id' 2024-09-11T06:51:07.949 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:07.949 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:07.949 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:51:07.950 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:51:07.950 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:07.964 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:51:08.004 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:08.002+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:08.026 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=24153 2024-09-11T06:51:08.026 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 24153 2024-09-11T06:51:08.026 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 24153 -C rbd-nbd 2024-09-11T06:51:08.033 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:51:08.034 INFO:tasks.workunit.client.0.smithi176.stdout: 24153 ? 00:00:00 rbd-nbd 2024-09-11T06:51:08.034 INFO:tasks.workunit.client.0.smithi176.stderr:+ ls /tmp/tmp.IGWYJYR7sz/mnt/ 2024-09-11T06:51:08.036 INFO:tasks.workunit.client.0.smithi176.stdout:lost+found 2024-09-11T06:51:08.037 INFO:tasks.workunit.client.0.smithi176.stdout:test 2024-09-11T06:51:08.037 INFO:tasks.workunit.client.0.smithi176.stderr:+ dd if=/tmp/tmp.IGWYJYR7sz/mnt/test of=/dev/null bs=1M count=1 2024-09-11T06:51:08.047 INFO:tasks.workunit.client.0.smithi176.stderr:1+0 records in 2024-09-11T06:51:08.047 INFO:tasks.workunit.client.0.smithi176.stderr:1+0 records out 2024-09-11T06:51:08.047 INFO:tasks.workunit.client.0.smithi176.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00934819 s, 112 MB/s 2024-09-11T06:51:08.047 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo dd if=/tmp/tmp.IGWYJYR7sz/data of=/tmp/tmp.IGWYJYR7sz/mnt/test1 bs=1M count=1 oflag=direct 2024-09-11T06:51:08.047 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:08.047 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:08.048 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:08.048 INFO:tasks.workunit.client.0.smithi176.stderr:++ which dd 2024-09-11T06:51:08.049 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/dd 2024-09-11T06:51:08.050 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:08.050 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/dd if=/tmp/tmp.IGWYJYR7sz/data of=/tmp/tmp.IGWYJYR7sz/mnt/test1 bs=1M count=1 oflag=direct 2024-09-11T06:51:08.075 INFO:tasks.workunit.client.0.smithi176.stderr:1+0 records in 2024-09-11T06:51:08.075 INFO:tasks.workunit.client.0.smithi176.stderr:1+0 records out 2024-09-11T06:51:08.075 INFO:tasks.workunit.client.0.smithi176.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01776 s, 59.0 MB/s 2024-09-11T06:51:08.076 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo umount /tmp/tmp.IGWYJYR7sz/mnt 2024-09-11T06:51:08.076 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:08.077 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:08.077 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:08.078 INFO:tasks.workunit.client.0.smithi176.stderr:++ which umount 2024-09-11T06:51:08.079 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/umount 2024-09-11T06:51:08.079 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:08.079 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/umount /tmp/tmp.IGWYJYR7sz/mnt 2024-09-11T06:51:08.081 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:08.074+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:08.096 INFO:tasks.workunit.client.0.smithi176.stderr:+ unmap_device /dev/nbd1 24153 2024-09-11T06:51:08.096 INFO:tasks.workunit.client.0.smithi176.stderr:+ local args=/dev/nbd1 2024-09-11T06:51:08.096 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pid=24153 2024-09-11T06:51:08.096 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd1 2024-09-11T06:51:08.096 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:08.096 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:08.097 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:08.097 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:51:08.098 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:51:08.098 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:08.098 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd1 2024-09-11T06:51:08.134 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:08.134 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:51:08.134 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:51:08.134 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:51:08.134 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.134 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:08.134 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:51:08.134 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 121.858828053, 2024-09-11T06:51:08.135 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283336, 2024-09-11T06:51:08.135 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:08.135 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:08.135 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:08.135 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:08.135 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:08.135 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:08.135 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 87 2024-09-11T06:51:08.135 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.135 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:08.135 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.135 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:08.135 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:51:08.136 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.136 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.136 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.136 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:08.136 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:51:08.136 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.136 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.136 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.136 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:08.136 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271768+0000", 2024-09-11T06:51:08.136 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7599999999999996e-07 2024-09-11T06:51:08.136 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.136 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.136 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:08.137 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271860+0000", 2024-09-11T06:51:08.137 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2639999999999994e-05 2024-09-11T06:51:08.137 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.137 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.137 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:08.137 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271873+0000", 2024-09-11T06:51:08.137 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3139000000000001e-05 2024-09-11T06:51:08.137 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.137 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.137 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:08.138 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271882+0000", 2024-09-11T06:51:08.138 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2369999999999992e-06 2024-09-11T06:51:08.138 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.138 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.138 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:08.138 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277150+0000", 2024-09-11T06:51:08.138 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052684419999999999 2024-09-11T06:51:08.138 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.138 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.138 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:08.138 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277273+0000", 2024-09-11T06:51:08.138 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012278899999999999 2024-09-11T06:51:08.138 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.138 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.139 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:08.139 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277466+0000", 2024-09-11T06:51:08.139 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193578 2024-09-11T06:51:08.139 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.139 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.139 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:08.139 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.280808+0000", 2024-09-11T06:51:08.139 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003341971 2024-09-11T06:51:08.139 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.139 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.139 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:08.139 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284419+0000", 2024-09-11T06:51:08.139 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003610982 2024-09-11T06:51:08.139 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.140 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.140 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:08.140 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284531+0000", 2024-09-11T06:51:08.140 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011121299999999999 2024-09-11T06:51:08.140 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.140 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.140 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:08.140 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284600+0000", 2024-09-11T06:51:08.140 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9592999999999996e-05 2024-09-11T06:51:08.140 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.140 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:08.140 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.140 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.141 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.141 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:08.141 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:51:08.141 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 121.85869780100001, 2024-09-11T06:51:08.141 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283314, 2024-09-11T06:51:08.141 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:08.141 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:08.141 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:08.141 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:08.141 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:08.141 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:08.142 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 88 2024-09-11T06:51:08.142 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.142 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:08.142 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.142 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:08.142 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:51:08.142 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.142 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.142 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.142 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:08.142 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:51:08.142 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.142 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.142 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.143 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:08.143 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271898+0000", 2024-09-11T06:51:08.143 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2999999999999999e-07 2024-09-11T06:51:08.143 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.143 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.143 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:08.143 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271904+0000", 2024-09-11T06:51:08.143 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.767e-06 2024-09-11T06:51:08.143 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.143 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.144 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:08.144 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271913+0000", 2024-09-11T06:51:08.144 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7200000000000001e-06 2024-09-11T06:51:08.144 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.144 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.144 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:08.144 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271920+0000", 2024-09-11T06:51:08.144 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7100000000000001e-06 2024-09-11T06:51:08.144 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.144 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.144 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:08.144 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278684+0000", 2024-09-11T06:51:08.144 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0067642650000000002 2024-09-11T06:51:08.145 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.145 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.145 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:08.145 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278838+0000", 2024-09-11T06:51:08.145 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015371499999999999 2024-09-11T06:51:08.145 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.145 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.145 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:08.145 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.279040+0000", 2024-09-11T06:51:08.145 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020207499999999999 2024-09-11T06:51:08.145 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.145 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.145 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:08.145 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.281448+0000", 2024-09-11T06:51:08.145 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002407686 2024-09-11T06:51:08.146 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.146 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.146 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:08.146 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284615+0000", 2024-09-11T06:51:08.146 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031675499999999999 2024-09-11T06:51:08.146 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.146 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.146 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:08.146 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284682+0000", 2024-09-11T06:51:08.146 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7032000000000005e-05 2024-09-11T06:51:08.146 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.146 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.146 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:08.147 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284730+0000", 2024-09-11T06:51:08.147 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7790000000000002e-05 2024-09-11T06:51:08.147 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.147 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:08.147 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.147 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.147 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.147 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:08.147 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:51:08.147 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 121.68984860099999, 2024-09-11T06:51:08.147 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014944941, 2024-09-11T06:51:08.147 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:08.147 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:08.148 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:08.148 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:08.148 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:08.148 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:08.148 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 111 2024-09-11T06:51:08.148 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.148 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:08.148 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.148 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:08.148 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:51:08.148 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.148 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.148 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.148 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:08.149 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:51:08.149 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.149 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.149 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.149 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:08.149 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440748+0000", 2024-09-11T06:51:08.149 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:51:08.149 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.149 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.149 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:08.149 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440760+0000", 2024-09-11T06:51:08.149 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1897e-05 2024-09-11T06:51:08.149 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.150 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.150 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:08.150 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440782+0000", 2024-09-11T06:51:08.150 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2552999999999999e-05 2024-09-11T06:51:08.150 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.150 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.150 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:08.150 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440793+0000", 2024-09-11T06:51:08.150 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0643e-05 2024-09-11T06:51:08.150 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.150 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.150 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:08.150 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449755+0000", 2024-09-11T06:51:08.150 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089622620000000004 2024-09-11T06:51:08.151 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.151 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.151 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:08.151 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449918+0000", 2024-09-11T06:51:08.151 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163086 2024-09-11T06:51:08.151 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.151 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.151 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:08.151 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450150+0000", 2024-09-11T06:51:08.151 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023204500000000001 2024-09-11T06:51:08.151 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.151 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.151 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:08.152 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454533+0000", 2024-09-11T06:51:08.152 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0043831349999999998 2024-09-11T06:51:08.152 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.152 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.152 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:08.152 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455511+0000", 2024-09-11T06:51:08.152 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0009776050000000001 2024-09-11T06:51:08.152 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.152 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.152 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:08.152 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455597+0000", 2024-09-11T06:51:08.152 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6446000000000005e-05 2024-09-11T06:51:08.152 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.153 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.153 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:08.153 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455691+0000", 2024-09-11T06:51:08.153 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3832000000000006e-05 2024-09-11T06:51:08.153 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.153 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:08.153 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.153 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.153 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.153 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:08.153 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:51:08.153 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 121.689778838, 2024-09-11T06:51:08.153 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018885411000000001, 2024-09-11T06:51:08.154 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:08.154 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:08.154 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:08.154 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:08.154 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:08.154 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:08.154 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 112 2024-09-11T06:51:08.154 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.154 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:08.154 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.154 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:08.155 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:51:08.155 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.155 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.155 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.155 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:08.155 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:51:08.155 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.155 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.155 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.155 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:08.155 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440817+0000", 2024-09-11T06:51:08.156 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:51:08.156 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.156 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.156 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:08.156 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441003+0000", 2024-09-11T06:51:08.156 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018661200000000001 2024-09-11T06:51:08.156 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.156 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.156 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:08.156 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441027+0000", 2024-09-11T06:51:08.156 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3920000000000001e-05 2024-09-11T06:51:08.156 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.156 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.156 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:08.156 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441062+0000", 2024-09-11T06:51:08.157 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4663000000000003e-05 2024-09-11T06:51:08.157 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.157 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.157 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:08.157 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451981+0000", 2024-09-11T06:51:08.157 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010918926000000001 2024-09-11T06:51:08.157 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.157 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.157 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:08.157 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452111+0000", 2024-09-11T06:51:08.157 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129529 2024-09-11T06:51:08.157 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.157 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.158 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:08.158 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452311+0000", 2024-09-11T06:51:08.158 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020013799999999999 2024-09-11T06:51:08.158 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.158 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.158 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:08.158 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455809+0000", 2024-09-11T06:51:08.158 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003498443 2024-09-11T06:51:08.158 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.158 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.158 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:08.158 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459566+0000", 2024-09-11T06:51:08.158 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003757271 2024-09-11T06:51:08.158 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.159 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.159 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:08.159 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459640+0000", 2024-09-11T06:51:08.159 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3949000000000006e-05 2024-09-11T06:51:08.159 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.159 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.159 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:08.159 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459701+0000", 2024-09-11T06:51:08.159 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1117000000000005e-05 2024-09-11T06:51:08.159 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.159 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:08.159 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.159 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.159 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.160 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:08.160 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:51:08.160 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 121.689499987, 2024-09-11T06:51:08.160 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018760915, 2024-09-11T06:51:08.160 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:08.160 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:08.160 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:08.160 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:08.160 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:08.160 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:08.160 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 113 2024-09-11T06:51:08.160 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.160 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:08.160 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.161 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:08.161 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:51:08.161 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.161 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.161 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:08.161 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:51:08.161 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.161 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.161 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:08.161 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441096+0000", 2024-09-11T06:51:08.161 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4130000000000001e-06 2024-09-11T06:51:08.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.162 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.162 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:08.162 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441110+0000", 2024-09-11T06:51:08.162 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3900000000000001e-05 2024-09-11T06:51:08.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.162 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.162 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:08.162 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441131+0000", 2024-09-11T06:51:08.162 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0636999999999999e-05 2024-09-11T06:51:08.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.162 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.162 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:08.162 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441147+0000", 2024-09-11T06:51:08.162 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6512999999999999e-05 2024-09-11T06:51:08.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.163 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.163 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:08.163 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453274+0000", 2024-09-11T06:51:08.163 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012126764999999999 2024-09-11T06:51:08.163 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.163 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.163 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:08.163 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453498+0000", 2024-09-11T06:51:08.163 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000223568 2024-09-11T06:51:08.163 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.163 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.163 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:08.163 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453872+0000", 2024-09-11T06:51:08.164 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037467099999999998 2024-09-11T06:51:08.164 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.164 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.164 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:08.164 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458709+0000", 2024-09-11T06:51:08.164 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0048365680000000003 2024-09-11T06:51:08.164 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.164 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.164 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:08.164 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459716+0000", 2024-09-11T06:51:08.164 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010075240000000001 2024-09-11T06:51:08.164 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.165 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.165 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:08.165 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459779+0000", 2024-09-11T06:51:08.165 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2336999999999996e-05 2024-09-11T06:51:08.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.165 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.165 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:08.165 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459856+0000", 2024-09-11T06:51:08.165 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7019000000000002e-05 2024-09-11T06:51:08.165 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.165 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:08.165 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.166 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.166 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:08.166 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:51:08.166 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 121.689423248, 2024-09-11T06:51:08.166 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018963114, 2024-09-11T06:51:08.166 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:08.166 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:08.166 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:08.166 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:08.166 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:08.166 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:08.166 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 114 2024-09-11T06:51:08.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.166 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:08.167 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.167 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:08.167 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:51:08.167 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.167 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.167 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.167 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:08.167 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:51:08.167 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.167 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.167 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.167 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:08.167 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:51:08.167 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9299999999999996e-07 2024-09-11T06:51:08.168 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.168 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.168 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:08.168 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441180+0000", 2024-09-11T06:51:08.168 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7200000000000006e-06 2024-09-11T06:51:08.168 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.168 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.168 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:08.168 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441265+0000", 2024-09-11T06:51:08.168 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4753000000000007e-05 2024-09-11T06:51:08.168 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.168 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.168 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:08.168 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441288+0000", 2024-09-11T06:51:08.169 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2860000000000001e-05 2024-09-11T06:51:08.169 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.169 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.169 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:08.169 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454585+0000", 2024-09-11T06:51:08.169 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013297286 2024-09-11T06:51:08.169 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.169 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.169 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:08.169 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454714+0000", 2024-09-11T06:51:08.169 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000128465 2024-09-11T06:51:08.169 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.169 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.169 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:08.170 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454909+0000", 2024-09-11T06:51:08.170 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019522499999999999 2024-09-11T06:51:08.170 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.170 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.170 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:08.170 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459868+0000", 2024-09-11T06:51:08.170 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0049591239999999997 2024-09-11T06:51:08.170 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.170 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.170 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:08.170 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459988+0000", 2024-09-11T06:51:08.170 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000120532 2024-09-11T06:51:08.171 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.171 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.171 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:08.171 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460071+0000", 2024-09-11T06:51:08.171 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2256000000000007e-05 2024-09-11T06:51:08.171 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.171 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.171 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:08.171 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460135+0000", 2024-09-11T06:51:08.171 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3999999999999997e-05 2024-09-11T06:51:08.171 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.171 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:08.171 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.172 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.172 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.172 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:08.172 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:51:08.172 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 121.689265619, 2024-09-11T06:51:08.172 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.019036849000000002, 2024-09-11T06:51:08.172 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:08.172 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:08.172 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:08.172 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:08.172 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:08.172 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:08.172 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 115 2024-09-11T06:51:08.173 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.173 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:08.173 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.173 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:08.173 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:51:08.173 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.173 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.173 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.173 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:08.173 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:51:08.173 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.173 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.174 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.174 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:08.174 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441330+0000", 2024-09-11T06:51:08.174 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1930000000000001e-06 2024-09-11T06:51:08.174 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.174 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.174 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:08.174 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441544+0000", 2024-09-11T06:51:08.174 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021405799999999999 2024-09-11T06:51:08.174 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.174 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.174 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:08.174 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441572+0000", 2024-09-11T06:51:08.175 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7317e-05 2024-09-11T06:51:08.175 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.175 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.175 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:08.175 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441593+0000", 2024-09-11T06:51:08.175 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1702999999999999e-05 2024-09-11T06:51:08.175 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.175 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.175 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:08.175 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455891+0000", 2024-09-11T06:51:08.175 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01429708 2024-09-11T06:51:08.175 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.175 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.175 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:08.176 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456046+0000", 2024-09-11T06:51:08.176 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000155682 2024-09-11T06:51:08.176 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.176 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.176 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:08.176 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456242+0000", 2024-09-11T06:51:08.176 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000195692 2024-09-11T06:51:08.176 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.176 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.176 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:08.176 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459876+0000", 2024-09-11T06:51:08.176 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036336480000000002 2024-09-11T06:51:08.176 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.177 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.177 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:08.177 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460245+0000", 2024-09-11T06:51:08.177 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036901700000000002 2024-09-11T06:51:08.177 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.177 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.177 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:08.177 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460316+0000", 2024-09-11T06:51:08.177 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1846000000000003e-05 2024-09-11T06:51:08.177 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.177 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.177 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:08.177 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460366+0000", 2024-09-11T06:51:08.177 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9613000000000001e-05 2024-09-11T06:51:08.178 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.178 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:08.178 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.178 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.178 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.178 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:08.178 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:51:08.178 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 121.68894353899999, 2024-09-11T06:51:08.178 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01904432, 2024-09-11T06:51:08.178 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:08.178 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:08.178 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:08.178 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:08.178 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:08.179 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:08.179 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 116 2024-09-11T06:51:08.179 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.179 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:08.179 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.179 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:08.179 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:51:08.179 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.179 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.179 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.180 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:08.180 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:51:08.180 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.180 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.180 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.180 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:08.180 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441653+0000", 2024-09-11T06:51:08.180 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:51:08.180 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.180 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.180 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:08.181 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441665+0000", 2024-09-11T06:51:08.181 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1653e-05 2024-09-11T06:51:08.181 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.181 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.181 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:08.181 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441684+0000", 2024-09-11T06:51:08.181 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9490000000000001e-05 2024-09-11T06:51:08.181 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.181 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.181 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:08.181 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441699+0000", 2024-09-11T06:51:08.181 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5044e-05 2024-09-11T06:51:08.181 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.181 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.182 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:08.182 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457297+0000", 2024-09-11T06:51:08.182 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015598241000000001 2024-09-11T06:51:08.182 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.182 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.182 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:08.182 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457577+0000", 2024-09-11T06:51:08.182 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027930799999999998 2024-09-11T06:51:08.182 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.182 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.182 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:08.182 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457957+0000", 2024-09-11T06:51:08.182 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00038088 2024-09-11T06:51:08.182 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.183 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.183 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:08.183 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460432+0000", 2024-09-11T06:51:08.183 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024743349999999998 2024-09-11T06:51:08.183 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.183 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.183 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:08.183 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460584+0000", 2024-09-11T06:51:08.183 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015232900000000001 2024-09-11T06:51:08.183 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.183 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.183 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:08.183 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460646+0000", 2024-09-11T06:51:08.183 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1883000000000004e-05 2024-09-11T06:51:08.184 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.184 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.184 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:08.184 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460696+0000", 2024-09-11T06:51:08.184 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9589999999999998e-05 2024-09-11T06:51:08.184 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.184 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:08.184 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.184 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.184 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.184 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:08.185 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:51:08.185 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 121.68887147300001, 2024-09-11T06:51:08.185 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.021834473, 2024-09-11T06:51:08.185 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:08.185 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:08.185 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:08.185 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:08.185 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:08.185 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:08.185 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 117 2024-09-11T06:51:08.185 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.185 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:08.185 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.185 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:08.186 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:51:08.186 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.186 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.186 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:08.186 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:51:08.186 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.186 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.186 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:08.186 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441725+0000", 2024-09-11T06:51:08.186 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5069999999999999e-06 2024-09-11T06:51:08.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.186 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.187 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:08.187 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441738+0000", 2024-09-11T06:51:08.187 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2813000000000001e-05 2024-09-11T06:51:08.187 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.187 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.187 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:08.187 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441759+0000", 2024-09-11T06:51:08.187 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1246999999999998e-05 2024-09-11T06:51:08.187 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.187 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.187 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:08.187 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441779+0000", 2024-09-11T06:51:08.187 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0392999999999999e-05 2024-09-11T06:51:08.187 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.188 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.188 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:08.188 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458804+0000", 2024-09-11T06:51:08.188 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.017024182999999998 2024-09-11T06:51:08.188 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.188 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.188 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:08.188 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458977+0000", 2024-09-11T06:51:08.188 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000173926 2024-09-11T06:51:08.188 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.188 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.188 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:08.188 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459170+0000", 2024-09-11T06:51:08.189 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000192184 2024-09-11T06:51:08.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.189 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.189 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:08.189 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.462300+0000", 2024-09-11T06:51:08.189 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031304969999999999 2024-09-11T06:51:08.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.189 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.189 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:08.189 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463408+0000", 2024-09-11T06:51:08.189 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011077839999999999 2024-09-11T06:51:08.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.189 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.189 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:08.190 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463497+0000", 2024-09-11T06:51:08.190 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8828999999999999e-05 2024-09-11T06:51:08.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.190 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.190 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:08.190 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463558+0000", 2024-09-11T06:51:08.190 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1110000000000003e-05 2024-09-11T06:51:08.190 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.190 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:08.190 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.190 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.190 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:08.190 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:51:08.191 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 121.31222997099999, 2024-09-11T06:51:08.191 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013136297, 2024-09-11T06:51:08.191 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:08.191 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:08.191 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:08.191 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:08.191 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:08.191 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:08.191 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 138 2024-09-11T06:51:08.191 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.191 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:08.191 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.191 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:08.191 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:51:08.192 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.192 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.192 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.192 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:08.192 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:51:08.192 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.192 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.192 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.192 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:08.192 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:51:08.192 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.89e-07 2024-09-11T06:51:08.192 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.192 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.192 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:08.193 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818369+0000", 2024-09-11T06:51:08.193 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4170000000000001e-06 2024-09-11T06:51:08.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.193 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.193 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:08.193 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818376+0000", 2024-09-11T06:51:08.193 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4270000000000002e-06 2024-09-11T06:51:08.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.193 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.193 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:08.193 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818383+0000", 2024-09-11T06:51:08.193 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3060000000000003e-06 2024-09-11T06:51:08.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.194 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.194 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:08.194 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824783+0000", 2024-09-11T06:51:08.194 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064003020000000001 2024-09-11T06:51:08.194 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.194 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.194 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:08.194 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825013+0000", 2024-09-11T06:51:08.194 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023039499999999999 2024-09-11T06:51:08.194 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.194 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.194 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:08.194 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825484+0000", 2024-09-11T06:51:08.194 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000470562 2024-09-11T06:51:08.195 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.195 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.195 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:08.195 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828633+0000", 2024-09-11T06:51:08.195 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031496689999999999 2024-09-11T06:51:08.195 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.195 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.195 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:08.195 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831413+0000", 2024-09-11T06:51:08.195 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027793219999999999 2024-09-11T06:51:08.195 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.195 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.195 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:08.195 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831450+0000", 2024-09-11T06:51:08.196 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7370999999999998e-05 2024-09-11T06:51:08.196 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.196 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.196 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:08.196 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831501+0000", 2024-09-11T06:51:08.196 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1036999999999999e-05 2024-09-11T06:51:08.196 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.196 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:08.196 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.196 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.196 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.196 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:08.196 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:51:08.196 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 121.312205491, 2024-09-11T06:51:08.196 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013223011, 2024-09-11T06:51:08.197 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:08.197 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:08.197 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:08.197 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:08.197 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:08.197 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:08.197 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 139 2024-09-11T06:51:08.197 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.197 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:08.197 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.198 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:08.198 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:51:08.198 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.198 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.198 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.198 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:08.198 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:51:08.198 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.198 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.198 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.198 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:08.198 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818390+0000", 2024-09-11T06:51:08.198 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5600000000000001e-07 2024-09-11T06:51:08.198 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.199 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.199 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:08.199 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818394+0000", 2024-09-11T06:51:08.199 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5929999999999999e-06 2024-09-11T06:51:08.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.199 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.199 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:08.199 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818399+0000", 2024-09-11T06:51:08.199 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7740000000000004e-06 2024-09-11T06:51:08.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.199 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.199 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:08.199 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818412+0000", 2024-09-11T06:51:08.199 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2625e-05 2024-09-11T06:51:08.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.200 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.200 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:08.200 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826205+0000", 2024-09-11T06:51:08.200 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0077930730000000002 2024-09-11T06:51:08.200 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.200 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.200 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:08.200 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826281+0000", 2024-09-11T06:51:08.200 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5876000000000003e-05 2024-09-11T06:51:08.200 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.200 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.200 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:08.200 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826384+0000", 2024-09-11T06:51:08.201 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103838 2024-09-11T06:51:08.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.201 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.201 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:08.201 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830642+0000", 2024-09-11T06:51:08.201 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004257083 2024-09-11T06:51:08.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.201 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.201 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:08.201 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831509+0000", 2024-09-11T06:51:08.201 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086730900000000003 2024-09-11T06:51:08.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.201 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.201 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:08.202 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831547+0000", 2024-09-11T06:51:08.202 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7793000000000002e-05 2024-09-11T06:51:08.202 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.202 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.202 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:08.202 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831612+0000", 2024-09-11T06:51:08.202 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5691000000000005e-05 2024-09-11T06:51:08.202 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.202 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:08.202 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.202 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.202 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.202 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:08.202 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:51:08.203 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 121.31217679700001, 2024-09-11T06:51:08.203 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016098024999999998, 2024-09-11T06:51:08.203 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:08.203 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:08.203 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:08.203 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:08.203 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:08.203 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:08.203 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 140 2024-09-11T06:51:08.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.203 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:08.203 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.203 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:08.203 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:51:08.204 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.204 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.204 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.204 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:08.204 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:51:08.204 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.204 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.204 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.204 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:08.204 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:51:08.204 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5199999999999998e-07 2024-09-11T06:51:08.204 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.204 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.204 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:08.205 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818479+0000", 2024-09-11T06:51:08.205 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0368000000000002e-05 2024-09-11T06:51:08.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.205 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.205 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:08.205 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818487+0000", 2024-09-11T06:51:08.205 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2339999999999994e-06 2024-09-11T06:51:08.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.205 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.205 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:08.205 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818491+0000", 2024-09-11T06:51:08.205 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4329999999999997e-06 2024-09-11T06:51:08.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.206 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.206 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:08.206 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828080+0000", 2024-09-11T06:51:08.206 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0095890449999999992 2024-09-11T06:51:08.206 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.206 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.206 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:08.206 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828130+0000", 2024-09-11T06:51:08.206 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9239000000000001e-05 2024-09-11T06:51:08.206 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.206 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.207 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:08.207 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828208+0000", 2024-09-11T06:51:08.207 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.852e-05 2024-09-11T06:51:08.207 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.207 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.207 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:08.207 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831625+0000", 2024-09-11T06:51:08.207 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034169410000000002 2024-09-11T06:51:08.207 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.207 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.207 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:08.207 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834411+0000", 2024-09-11T06:51:08.208 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002786111 2024-09-11T06:51:08.208 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.208 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.208 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:08.208 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834457+0000", 2024-09-11T06:51:08.208 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6041000000000002e-05 2024-09-11T06:51:08.208 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.208 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.208 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:08.208 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834516+0000", 2024-09-11T06:51:08.209 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8740999999999999e-05 2024-09-11T06:51:08.209 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.209 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:08.209 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.209 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.209 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.209 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:08.209 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:51:08.209 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 121.31209573, 2024-09-11T06:51:08.209 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016588772000000002, 2024-09-11T06:51:08.209 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:08.209 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:08.209 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:08.209 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:08.210 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:08.210 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:08.210 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 141 2024-09-11T06:51:08.210 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.210 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:08.210 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.210 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:08.210 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:51:08.210 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.210 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.210 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.210 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:08.210 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:51:08.210 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.211 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.211 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.211 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:08.211 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818500+0000", 2024-09-11T06:51:08.211 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0800000000000005e-07 2024-09-11T06:51:08.211 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.211 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.211 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:08.211 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818505+0000", 2024-09-11T06:51:08.211 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2429999999999996e-06 2024-09-11T06:51:08.211 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.211 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.211 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:08.212 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818510+0000", 2024-09-11T06:51:08.212 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9339999999999997e-06 2024-09-11T06:51:08.212 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.212 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.212 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:08.212 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818514+0000", 2024-09-11T06:51:08.212 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7270000000000001e-06 2024-09-11T06:51:08.212 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.212 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.212 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:08.212 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828711+0000", 2024-09-11T06:51:08.212 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010197031 2024-09-11T06:51:08.212 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.213 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.213 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:08.213 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828853+0000", 2024-09-11T06:51:08.213 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014201200000000001 2024-09-11T06:51:08.213 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.213 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.213 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:08.213 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.829115+0000", 2024-09-11T06:51:08.213 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026257500000000003 2024-09-11T06:51:08.213 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.213 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.213 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:08.213 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831629+0000", 2024-09-11T06:51:08.213 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025138169999999998 2024-09-11T06:51:08.213 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.214 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.214 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:08.214 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834991+0000", 2024-09-11T06:51:08.214 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033624459999999998 2024-09-11T06:51:08.214 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.214 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.214 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:08.214 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835032+0000", 2024-09-11T06:51:08.214 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1001000000000003e-05 2024-09-11T06:51:08.214 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.214 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.214 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:08.214 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835088+0000", 2024-09-11T06:51:08.214 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5478e-05 2024-09-11T06:51:08.215 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.215 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:08.215 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.215 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.215 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.215 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5235.0:1 2.5 2:b2cc3336:::rbd_id.testrbdnbd21607:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-11T06:51:08.215 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:51:08.215 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 33.571293707999999, 2024-09-11T06:51:08.215 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.68353462499999995, 2024-09-11T06:51:08.215 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:08.215 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:08.215 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:08.215 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:08.215 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5235", 2024-09-11T06:51:08.215 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2008977164", 2024-09-11T06:51:08.216 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:51:08.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.216 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:08.216 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.216 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:08.216 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:51:08.216 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.216 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.216 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:08.216 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:51:08.216 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.217 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.217 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:08.217 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559303+0000", 2024-09-11T06:51:08.217 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1830000000000001e-06 2024-09-11T06:51:08.217 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.217 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.217 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:08.217 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559314+0000", 2024-09-11T06:51:08.217 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0396e-05 2024-09-11T06:51:08.217 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.217 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.217 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:08.217 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559335+0000", 2024-09-11T06:51:08.217 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1183e-05 2024-09-11T06:51:08.217 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.218 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.218 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:08.218 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559354+0000", 2024-09-11T06:51:08.218 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9225999999999999e-05 2024-09-11T06:51:08.218 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.218 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.218 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:08.218 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559522+0000", 2024-09-11T06:51:08.218 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000168021 2024-09-11T06:51:08.218 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.218 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.218 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:51:08.218 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559543+0000", 2024-09-11T06:51:08.218 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0616999999999999e-05 2024-09-11T06:51:08.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.219 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.219 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:08.219 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.238584+0000", 2024-09-11T06:51:08.219 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.67904107499999999 2024-09-11T06:51:08.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.219 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.219 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for missing object", 2024-09-11T06:51:08.219 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.238933+0000", 2024-09-11T06:51:08.219 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034879600000000001 2024-09-11T06:51:08.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.219 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.219 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:08.219 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.241968+0000", 2024-09-11T06:51:08.219 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030350830000000001 2024-09-11T06:51:08.220 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.220 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.220 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:08.220 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.242281+0000", 2024-09-11T06:51:08.220 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00031299499999999999 2024-09-11T06:51:08.220 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.220 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.220 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:08.220 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.242836+0000", 2024-09-11T06:51:08.220 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00055504999999999997 2024-09-11T06:51:08.220 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.220 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:08.220 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.221 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.221 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5235.0:18 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 0~1024 [fadvise_sequential]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T06:51:08.221 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:51:08.221 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 32.868801269000002, 2024-09-11T06:51:08.221 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.048993760999999997, 2024-09-11T06:51:08.221 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:08.221 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:08.221 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:08.221 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:08.221 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5235", 2024-09-11T06:51:08.222 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2008977164", 2024-09-11T06:51:08.222 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 18 2024-09-11T06:51:08.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.222 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:08.222 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.222 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:08.222 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:51:08.222 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.222 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.222 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:08.222 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:51:08.222 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.223 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:08.223 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261797+0000", 2024-09-11T06:51:08.223 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2059999999999999e-06 2024-09-11T06:51:08.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.223 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:08.223 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261812+0000", 2024-09-11T06:51:08.223 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5159e-05 2024-09-11T06:51:08.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.223 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:08.223 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261852+0000", 2024-09-11T06:51:08.223 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.9906e-05 2024-09-11T06:51:08.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.224 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.224 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:08.224 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261865+0000", 2024-09-11T06:51:08.224 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.312e-05 2024-09-11T06:51:08.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.224 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.224 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:08.224 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.263166+0000", 2024-09-11T06:51:08.224 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001301134 2024-09-11T06:51:08.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.224 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.224 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for missing object", 2024-09-11T06:51:08.224 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.263529+0000", 2024-09-11T06:51:08.225 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036279900000000002 2024-09-11T06:51:08.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.225 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:08.225 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.300567+0000", 2024-09-11T06:51:08.225 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.037038326000000003 2024-09-11T06:51:08.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.225 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:08.225 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.300800+0000", 2024-09-11T06:51:08.225 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023323300000000001 2024-09-11T06:51:08.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.226 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:08.226 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.306709+0000", 2024-09-11T06:51:08.226 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0059087690000000003 2024-09-11T06:51:08.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.226 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.579 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:08.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.306850+0000", 2024-09-11T06:51:08.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014106200000000001 2024-09-11T06:51:08.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.579 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:08.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.307121+0000", 2024-09-11T06:51:08.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027083599999999999 2024-09-11T06:51:08.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.579 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:08.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.308809+0000", 2024-09-11T06:51:08.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0016880759999999999 2024-09-11T06:51:08.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.580 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:08.580 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310382+0000", 2024-09-11T06:51:08.580 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0015732840000000001 2024-09-11T06:51:08.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.580 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:08.580 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310501+0000", 2024-09-11T06:51:08.580 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000118276 2024-09-11T06:51:08.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.580 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:08.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310787+0000", 2024-09-11T06:51:08.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00028657500000000001 2024-09-11T06:51:08.581 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.581 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:08.581 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.581 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:70 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 393216~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:08.581 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596118+0000", 2024-09-11T06:51:08.581 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 19.534476866999999, 2024-09-11T06:51:08.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012279145, 2024-09-11T06:51:08.582 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:08.582 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:08.582 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:08.582 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:08.582 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:08.582 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:08.582 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 70 2024-09-11T06:51:08.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.582 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:08.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:08.582 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596118+0000", 2024-09-11T06:51:08.582 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:08.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596118+0000", 2024-09-11T06:51:08.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:08.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596119+0000", 2024-09-11T06:51:08.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0299999999999997e-07 2024-09-11T06:51:08.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:08.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596238+0000", 2024-09-11T06:51:08.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000118798 2024-09-11T06:51:08.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:08.584 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596250+0000", 2024-09-11T06:51:08.584 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2775999999999999e-05 2024-09-11T06:51:08.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.584 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:08.584 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596259+0000", 2024-09-11T06:51:08.584 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3100000000000001e-06 2024-09-11T06:51:08.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.584 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:08.584 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.603389+0000", 2024-09-11T06:51:08.584 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0071300759999999999 2024-09-11T06:51:08.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.585 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:08.585 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.603540+0000", 2024-09-11T06:51:08.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015111700000000001 2024-09-11T06:51:08.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.585 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:08.585 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.603766+0000", 2024-09-11T06:51:08.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022570899999999999 2024-09-11T06:51:08.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.585 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:08.585 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.606676+0000", 2024-09-11T06:51:08.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0029106380000000001 2024-09-11T06:51:08.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:08.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608173+0000", 2024-09-11T06:51:08.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014967629999999999 2024-09-11T06:51:08.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:08.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608277+0000", 2024-09-11T06:51:08.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000104398 2024-09-11T06:51:08.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:08.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608397+0000", 2024-09-11T06:51:08.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000119657 2024-09-11T06:51:08.587 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.587 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:08.587 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.587 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:71 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 524288~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:08.587 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596272+0000", 2024-09-11T06:51:08.587 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 19.534322614000001, 2024-09-11T06:51:08.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013390681999999999, 2024-09-11T06:51:08.587 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:08.587 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:08.588 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:08.588 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:08.588 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:08.588 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:08.588 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 71 2024-09-11T06:51:08.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.588 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:08.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.588 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:08.588 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596272+0000", 2024-09-11T06:51:08.588 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.588 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:08.589 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596272+0000", 2024-09-11T06:51:08.589 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.589 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:08.589 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596273+0000", 2024-09-11T06:51:08.589 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.9999999999999996e-07 2024-09-11T06:51:08.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.589 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:08.589 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596395+0000", 2024-09-11T06:51:08.589 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012204799999999999 2024-09-11T06:51:08.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:08.590 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596411+0000", 2024-09-11T06:51:08.590 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5746000000000001e-05 2024-09-11T06:51:08.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:08.590 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596419+0000", 2024-09-11T06:51:08.590 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2600000000000005e-06 2024-09-11T06:51:08.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:08.591 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.605311+0000", 2024-09-11T06:51:08.591 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0088915179999999993 2024-09-11T06:51:08.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.591 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:08.591 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.605436+0000", 2024-09-11T06:51:08.591 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000125203 2024-09-11T06:51:08.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.591 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:08.591 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.605625+0000", 2024-09-11T06:51:08.591 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018961299999999999 2024-09-11T06:51:08.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.592 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:08.592 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609477+0000", 2024-09-11T06:51:08.592 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0038513549999999999 2024-09-11T06:51:08.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.592 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:08.592 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609499+0000", 2024-09-11T06:51:08.592 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2215e-05 2024-09-11T06:51:08.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.592 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:08.592 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609589+0000", 2024-09-11T06:51:08.593 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0029000000000001e-05 2024-09-11T06:51:08.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.593 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:08.593 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609663+0000", 2024-09-11T06:51:08.593 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3894999999999996e-05 2024-09-11T06:51:08.593 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.593 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:08.593 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.593 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:72 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 655360~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:08.594 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596432+0000", 2024-09-11T06:51:08.594 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 19.534163126999999, 2024-09-11T06:51:08.594 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01414727, 2024-09-11T06:51:08.594 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:08.594 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:08.594 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:08.594 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:08.594 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:08.594 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:08.594 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 72 2024-09-11T06:51:08.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.594 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:08.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.595 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:08.595 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596432+0000", 2024-09-11T06:51:08.595 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.595 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:08.595 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596432+0000", 2024-09-11T06:51:08.595 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.595 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:08.595 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596433+0000", 2024-09-11T06:51:08.595 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1999999999999998e-07 2024-09-11T06:51:08.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.596 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:08.596 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596556+0000", 2024-09-11T06:51:08.596 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012379000000000001 2024-09-11T06:51:08.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.596 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:08.596 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596571+0000", 2024-09-11T06:51:08.596 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4749999999999999e-05 2024-09-11T06:51:08.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.596 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:08.596 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596579+0000", 2024-09-11T06:51:08.596 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2500000000000006e-06 2024-09-11T06:51:08.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.597 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:08.597 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.606784+0000", 2024-09-11T06:51:08.597 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010204757 2024-09-11T06:51:08.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.597 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:08.597 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.606950+0000", 2024-09-11T06:51:08.597 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000166229 2024-09-11T06:51:08.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.597 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:08.597 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.607176+0000", 2024-09-11T06:51:08.598 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022549899999999999 2024-09-11T06:51:08.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.598 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:08.598 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610363+0000", 2024-09-11T06:51:08.598 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031874690000000001 2024-09-11T06:51:08.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.598 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:08.598 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610490+0000", 2024-09-11T06:51:08.598 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012639899999999999 2024-09-11T06:51:08.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.599 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:08.599 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610529+0000", 2024-09-11T06:51:08.599 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.8936000000000001e-05 2024-09-11T06:51:08.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.599 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:08.599 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610579+0000", 2024-09-11T06:51:08.599 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0371000000000003e-05 2024-09-11T06:51:08.599 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.599 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:08.599 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.600 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:73 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 786432~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:08.600 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596592+0000", 2024-09-11T06:51:08.600 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 19.534002873999999, 2024-09-11T06:51:08.600 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014914779, 2024-09-11T06:51:08.600 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:08.600 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:08.600 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:08.600 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:08.600 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:08.600 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:08.600 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 73 2024-09-11T06:51:08.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.600 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:08.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.601 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:08.601 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596592+0000", 2024-09-11T06:51:08.601 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.601 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:08.601 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596592+0000", 2024-09-11T06:51:08.601 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.601 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:08.601 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596593+0000", 2024-09-11T06:51:08.602 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.0699999999999996e-07 2024-09-11T06:51:08.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.602 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:08.602 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596786+0000", 2024-09-11T06:51:08.602 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019278200000000001 2024-09-11T06:51:08.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.602 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:08.602 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596801+0000", 2024-09-11T06:51:08.602 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5690000000000001e-05 2024-09-11T06:51:08.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.602 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:08.603 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596815+0000", 2024-09-11T06:51:08.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3753e-05 2024-09-11T06:51:08.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.603 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:08.603 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608442+0000", 2024-09-11T06:51:08.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011627385 2024-09-11T06:51:08.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.603 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:08.603 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608570+0000", 2024-09-11T06:51:08.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012740399999999999 2024-09-11T06:51:08.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.604 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:08.604 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608754+0000", 2024-09-11T06:51:08.604 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018392899999999999 2024-09-11T06:51:08.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.604 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:08.604 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610370+0000", 2024-09-11T06:51:08.604 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001615998 2024-09-11T06:51:08.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.604 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:08.604 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611420+0000", 2024-09-11T06:51:08.605 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010502790000000001 2024-09-11T06:51:08.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.605 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:08.605 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611458+0000", 2024-09-11T06:51:08.605 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7749000000000001e-05 2024-09-11T06:51:08.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.605 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:08.605 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611507+0000", 2024-09-11T06:51:08.605 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9002999999999998e-05 2024-09-11T06:51:08.605 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.606 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:08.606 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.606 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.606 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:74 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 917504~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:08.606 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596849+0000", 2024-09-11T06:51:08.606 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 19.533746235999999, 2024-09-11T06:51:08.606 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015202963999999999, 2024-09-11T06:51:08.606 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:08.606 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:08.606 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:08.606 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:08.606 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:08.606 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:08.607 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 74 2024-09-11T06:51:08.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.607 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:08.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.607 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:08.607 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596849+0000", 2024-09-11T06:51:08.607 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.607 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:08.607 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596849+0000", 2024-09-11T06:51:08.607 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:08.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.607 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:08.608 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596850+0000", 2024-09-11T06:51:08.608 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3170000000000001e-06 2024-09-11T06:51:08.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.608 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:08.608 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.597118+0000", 2024-09-11T06:51:08.608 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026772999999999999 2024-09-11T06:51:08.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.608 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:08.608 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.597136+0000", 2024-09-11T06:51:08.608 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8467000000000001e-05 2024-09-11T06:51:08.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.608 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:08.609 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.597148+0000", 2024-09-11T06:51:08.609 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1486e-05 2024-09-11T06:51:08.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.609 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:08.609 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609714+0000", 2024-09-11T06:51:08.609 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012566124 2024-09-11T06:51:08.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.609 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:08.610 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609792+0000", 2024-09-11T06:51:08.610 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.8579999999999996e-05 2024-09-11T06:51:08.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.610 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:08.610 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609899+0000", 2024-09-11T06:51:08.610 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00010716499999999999 2024-09-11T06:51:08.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.610 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:08.610 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610865+0000", 2024-09-11T06:51:08.610 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00096540299999999999 2024-09-11T06:51:08.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.611 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.611 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:08.611 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611970+0000", 2024-09-11T06:51:08.611 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001105118 2024-09-11T06:51:08.611 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.611 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.611 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:08.611 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.612005+0000", 2024-09-11T06:51:08.611 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4789000000000002e-05 2024-09-11T06:51:08.611 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:08.611 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:08.611 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:08.611 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.612052+0000", 2024-09-11T06:51:08.611 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6785e-05 2024-09-11T06:51:08.612 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.612 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:08.612 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.612 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:08.612 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:08.612 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:08.614 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:51:08.615 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:51:08.615 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^24153\b' 2024-09-11T06:51:08.615 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^24153\b' 2024-09-11T06:51:08.615 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:51:08.615 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -C rbd-nbd 2024-09-11T06:51:08.615 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^ *24153\b' 2024-09-11T06:51:08.615 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^ *24153\b' 2024-09-11T06:51:08.615 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:51:08.615 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 0.5 2024-09-11T06:51:08.616 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:08.610+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:08.714 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:08.710+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:08.746 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:51:08.749 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:08.749 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:08.749 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:51:08.749 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:08.762 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' -n 1935c02c-84a6-4710-85dc-6630efac9835 ']' 2024-09-11T06:51:08.763 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:51:08.764 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo rbd device --device-type nbd --show-cookie --cookie 'abc de' map rbd/testrbdnbd21607 2024-09-11T06:51:08.764 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:51:08.764 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:51:08.765 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:51:08.766 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which rbd 2024-09-11T06:51:08.767 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/rbd 2024-09-11T06:51:08.768 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:51:08.768 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd --show-cookie --cookie 'abc de' map rbd/testrbdnbd21607 2024-09-11T06:51:08.815 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:08.810+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:08.890 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:08.890 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:08.890 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:51:08.890 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:51:08.890 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:08.895 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is set 2024-09-11T06:51:08.903 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:51:08.903 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:51:08.905 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:51:08.916 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:08.910+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:08.920 INFO:tasks.workunit.client.0.smithi176.stderr:+ OUT='/dev/nbd1 abc de' 2024-09-11T06:51:08.921 INFO:tasks.workunit.client.0.smithi176.stderr:+ read DEV ANOTHER_COOKIE 2024-09-11T06:51:08.921 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd 2024-09-11T06:51:08.921 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:51:08.921 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:51:08.921 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:51:08.922 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:51:08.922 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:51:08.922 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd1'\'']/id' 2024-09-11T06:51:08.937 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:51:08.937 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:51:08.955 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:51:08.956 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:51:08.973 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:51:08.973 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:51:08.992 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:51:08.992 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:51:09.015 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=24227 2024-09-11T06:51:09.015 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 24227 2024-09-11T06:51:09.015 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 24227 -C rbd-nbd 2024-09-11T06:51:09.020 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:51:09.021 INFO:tasks.workunit.client.0.smithi176.stdout: 24227 ? 00:00:00 rbd-nbd 2024-09-11T06:51:09.021 INFO:tasks.workunit.client.0.smithi176.stderr:+ test 'abc de' = 'abc de' 2024-09-11T06:51:09.021 INFO:tasks.workunit.client.0.smithi176.stderr:+ unmap_device /dev/nbd1 24227 2024-09-11T06:51:09.021 INFO:tasks.workunit.client.0.smithi176.stderr:+ local args=/dev/nbd1 2024-09-11T06:51:09.021 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pid=24227 2024-09-11T06:51:09.021 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd1 2024-09-11T06:51:09.022 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:09.022 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:09.022 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:09.023 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:51:09.024 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:51:09.024 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:09.024 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd1 2024-09-11T06:51:09.028 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:09.028 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:09.042 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:51:09.053 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:09.050+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:09.117 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:09.114+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:09.131 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:51:09.131 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^24227\b' 2024-09-11T06:51:09.131 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^24227\b' 2024-09-11T06:51:09.165 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:09.166 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:09.179 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:51:09.191 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:51:09.192 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -C rbd-nbd 2024-09-11T06:51:09.192 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^ *24227\b' 2024-09-11T06:51:09.192 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^ *24227\b' 2024-09-11T06:51:09.198 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:51:09.198 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 0.5 2024-09-11T06:51:09.217 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:09.214+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:09.318 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:09.314+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:09.344 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:09.344 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:09.358 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:51:09.419 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:09.414+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:09.520 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:09.514+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:09.523 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:09.523 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:09.537 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:51:09.621 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:09.618+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:09.668 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:09.668 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:09.682 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:51:09.699 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV= 2024-09-11T06:51:09.700 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd snap ls rbd/testrbdnbd21607 2024-09-11T06:51:09.700 INFO:tasks.workunit.client.0.smithi176.stderr:++ awk '$2 == "snap" {print $1}' 2024-09-11T06:51:09.725 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:09.722+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:09.761 INFO:tasks.workunit.client.0.smithi176.stderr:+ SNAPID=3 2024-09-11T06:51:09.761 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo rbd device --device-type nbd --show-cookie map --snap-id 3 rbd/testrbdnbd21607 2024-09-11T06:51:09.761 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:51:09.761 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:51:09.762 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:51:09.762 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which rbd 2024-09-11T06:51:09.763 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/rbd 2024-09-11T06:51:09.763 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:51:09.763 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd --show-cookie map --snap-id 3 rbd/testrbdnbd21607 2024-09-11T06:51:09.823 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:09.822+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:09.852 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:09.852 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:09.866 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:51:09.902 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 1, 3, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T06:51:09.902 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-11T06:51:09.924 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:09.918+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:09.924 INFO:tasks.workunit.client.0.smithi176.stderr:+ OUT='/dev/nbd1 a0124953-6322-43f9-87aa-df785650153f' 2024-09-11T06:51:09.924 INFO:tasks.workunit.client.0.smithi176.stderr:+ read DEV COOKIE 2024-09-11T06:51:09.924 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd 2024-09-11T06:51:09.925 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:51:09.925 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:51:09.925 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:51:09.925 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd1'\'']/id' 2024-09-11T06:51:09.986 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:09.986 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:10.001 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:51:10.017 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=24311 2024-09-11T06:51:10.017 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 24311 2024-09-11T06:51:10.017 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 24311 -C rbd-nbd 2024-09-11T06:51:10.023 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:51:10.023 INFO:tasks.workunit.client.0.smithi176.stdout: 24311 ? 00:00:00 rbd-nbd 2024-09-11T06:51:10.024 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device detach rbd/testrbdnbd21607 --snap-id 3 --device-type nbd 2024-09-11T06:51:10.024 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:10.024 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:10.025 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:10.025 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:51:10.027 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:51:10.027 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:10.027 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device detach rbd/testrbdnbd21607 --snap-id 3 --device-type nbd 2024-09-11T06:51:10.045 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:10.042+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:10.125 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:10.118+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:10.133 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:10.133 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:10.141 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false get_pid rbd 2024-09-11T06:51:10.141 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd 2024-09-11T06:51:10.141 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:51:10.141 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:51:10.141 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:51:10.141 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd1'\'']/id' 2024-09-11T06:51:10.148 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:51:10.209 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID= 2024-09-11T06:51:10.209 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n '' 2024-09-11T06:51:10.209 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 1 2024-09-11T06:51:10.209 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:51:10.209 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false _sudo rbd device attach --device /dev/nbd1 --snap-id 3 rbd/testrbdnbd21607 --device-type nbd 2024-09-11T06:51:10.209 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device attach --device /dev/nbd1 --snap-id 3 rbd/testrbdnbd21607 --device-type nbd 2024-09-11T06:51:10.209 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:10.209 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:10.210 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:10.210 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:51:10.211 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:51:10.211 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:10.212 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device attach --device /dev/nbd1 --snap-id 3 rbd/testrbdnbd21607 --device-type nbd 2024-09-11T06:51:10.233 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:10.230+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:10.250 INFO:tasks.workunit.client.0.smithi176.stderr:rbd: could not validate attach request 2024-09-11T06:51:10.250 INFO:tasks.workunit.client.0.smithi176.stderr:rbd: mismatching the image and the device may lead to data corruption 2024-09-11T06:51:10.250 INFO:tasks.workunit.client.0.smithi176.stderr:rbd: must specify --cookie or --force to proceed 2024-09-11T06:51:10.252 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:51:10.252 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' -n a0124953-6322-43f9-87aa-df785650153f ']' 2024-09-11T06:51:10.252 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device attach --device /dev/nbd1 --cookie a0124953-6322-43f9-87aa-df785650153f --snap-id 3 rbd/testrbdnbd21607 --device-type nbd 2024-09-11T06:51:10.252 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:10.252 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:10.253 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:10.253 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:51:10.255 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:51:10.255 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:10.255 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device attach --device /dev/nbd1 --cookie a0124953-6322-43f9-87aa-df785650153f --snap-id 3 rbd/testrbdnbd21607 --device-type nbd 2024-09-11T06:51:10.303 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:10.304 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:10.318 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:51:10.326 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:10.322+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:10.390 INFO:tasks.workunit.client.0.smithi176.stdout:/dev/nbd1 2024-09-11T06:51:10.397 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd 2024-09-11T06:51:10.397 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:51:10.397 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:51:10.397 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:51:10.397 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd1'\'']/id' 2024-09-11T06:51:10.427 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:10.422+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:10.479 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=24375 2024-09-11T06:51:10.479 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 24375 2024-09-11T06:51:10.479 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 24375 -C rbd-nbd 2024-09-11T06:51:10.484 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:10.484 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:10.485 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:51:10.485 INFO:tasks.workunit.client.0.smithi176.stdout: 24375 ? 00:00:00 rbd-nbd 2024-09-11T06:51:10.485 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device detach /dev/nbd1 --device-type nbd 2024-09-11T06:51:10.485 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:10.485 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:10.486 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:10.486 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:51:10.488 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:51:10.488 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:10.488 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device detach /dev/nbd1 --device-type nbd 2024-09-11T06:51:10.500 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:51:10.541 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:10.538+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:10.587 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false get_pid rbd 2024-09-11T06:51:10.587 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd 2024-09-11T06:51:10.587 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:51:10.587 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:51:10.588 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:51:10.588 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd1'\'']/id' 2024-09-11T06:51:10.629 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:10.626+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:10.658 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID= 2024-09-11T06:51:10.659 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n '' 2024-09-11T06:51:10.659 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 1 2024-09-11T06:51:10.659 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:51:10.659 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV= 2024-09-11T06:51:10.659 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd config image set rbd/testrbdnbd21607 rbd_discard_granularity_bytes 4096 2024-09-11T06:51:10.664 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:10.664 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:10.678 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:51:10.730 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:10.726+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:10.736 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd feature enable rbd/testrbdnbd21607 journaling 2024-09-11T06:51:10.810 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:10.810 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:10.826 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:51:10.830 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:10.826+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:10.836 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo rbd device --device-type nbd map rbd/testrbdnbd21607 2024-09-11T06:51:10.837 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:51:10.837 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:51:10.838 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:51:10.838 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which rbd 2024-09-11T06:51:10.840 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/rbd 2024-09-11T06:51:10.840 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:51:10.840 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map rbd/testrbdnbd21607 2024-09-11T06:51:10.907 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.0 to 1 (10000) 2024-09-11T06:51:10.917 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T06:51:10.931 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-11T06:51:10.931 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:10.926+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:11.000 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:11.000 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:11.004 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV=/dev/nbd2 2024-09-11T06:51:11.005 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd 2024-09-11T06:51:11.005 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:51:11.005 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:51:11.005 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:51:11.005 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd2'\'']/id' 2024-09-11T06:51:11.010 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:51:11.016 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:51:11.061 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:11.058+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:11.103 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=24474 2024-09-11T06:51:11.103 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 24474 2024-09-11T06:51:11.103 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 24474 -C rbd-nbd 2024-09-11T06:51:11.109 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:51:11.109 INFO:tasks.workunit.client.0.smithi176.stdout: 24474 ? 00:00:00 rbd-nbd 2024-09-11T06:51:11.110 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo blkdiscard --offset 0 --length 3584 /dev/nbd2 2024-09-11T06:51:11.110 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:11.110 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:11.110 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:11.111 INFO:tasks.workunit.client.0.smithi176.stderr:++ which blkdiscard 2024-09-11T06:51:11.112 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/sbin/blkdiscard 2024-09-11T06:51:11.112 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:11.112 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/sbin/blkdiscard --offset 0 --length 3584 /dev/nbd2 2024-09-11T06:51:11.133 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:11.126+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:11.143 INFO:tasks.workunit.client.0.smithi176.stderr:blkdiscard: /dev/nbd2 contains existing file system (ext2). 2024-09-11T06:51:11.145 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo blkdiscard --offset 0 --length 4096 /dev/nbd2 2024-09-11T06:51:11.145 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:11.145 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:11.146 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:11.146 INFO:tasks.workunit.client.0.smithi176.stderr:++ which blkdiscard 2024-09-11T06:51:11.147 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/sbin/blkdiscard 2024-09-11T06:51:11.148 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:11.148 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/sbin/blkdiscard --offset 0 --length 4096 /dev/nbd2 2024-09-11T06:51:11.150 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:11.150 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:11.165 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:51:11.169 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo blkdiscard --offset 0 --length 4608 /dev/nbd2 2024-09-11T06:51:11.169 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:11.169 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:11.170 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:11.170 INFO:tasks.workunit.client.0.smithi176.stderr:++ which blkdiscard 2024-09-11T06:51:11.172 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/sbin/blkdiscard 2024-09-11T06:51:11.172 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:11.172 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/sbin/blkdiscard --offset 0 --length 4608 /dev/nbd2 2024-09-11T06:51:11.177 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:51:11.177 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:51:11.186 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo blkdiscard --offset 512 --length 7168 /dev/nbd2 2024-09-11T06:51:11.186 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:11.186 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:11.187 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:11.188 INFO:tasks.workunit.client.0.smithi176.stderr:++ which blkdiscard 2024-09-11T06:51:11.189 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/sbin/blkdiscard 2024-09-11T06:51:11.189 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:11.189 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/sbin/blkdiscard --offset 512 --length 7168 /dev/nbd2 2024-09-11T06:51:11.196 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:51:11.197 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:51:11.199 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo blkdiscard --offset 512 --length 7680 /dev/nbd2 2024-09-11T06:51:11.199 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:11.199 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:11.200 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:11.200 INFO:tasks.workunit.client.0.smithi176.stderr:++ which blkdiscard 2024-09-11T06:51:11.201 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/sbin/blkdiscard 2024-09-11T06:51:11.201 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:11.201 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/sbin/blkdiscard --offset 512 --length 7680 /dev/nbd2 2024-09-11T06:51:11.214 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo blkdiscard --offset 512 --length 8192 /dev/nbd2 2024-09-11T06:51:11.214 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:11.214 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:11.215 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:11.215 INFO:tasks.workunit.client.0.smithi176.stderr:++ which blkdiscard 2024-09-11T06:51:11.217 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/sbin/blkdiscard 2024-09-11T06:51:11.217 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:11.217 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/sbin/blkdiscard --offset 512 --length 8192 /dev/nbd2 2024-09-11T06:51:11.226 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:51:11.227 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:51:11.230 INFO:tasks.workunit.client.0.smithi176.stderr:+ tries=0 2024-09-11T06:51:11.231 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd journal inspect --pool rbd --image testrbdnbd21607 2024-09-11T06:51:11.231 INFO:tasks.workunit.client.0.smithi176.stderr:++ awk '/entries inspected/ {print $1}' 2024-09-11T06:51:11.233 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:11.230+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:11.245 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:51:11.245 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:51:11.279 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:51:11.280 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:51:11.294 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:11.294 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:11.294 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:51:11.294 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:51:11.294 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:11.302 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:51:11.302 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:51:11.303 INFO:tasks.workunit.client.0.smithi176.stderr:2024-09-11T06:51:11.299+0000 7f0094bdd640 -1 JournalMetadata: 0x55bd5fbe15e0 failed to locate client: INSPECT 2024-09-11T06:51:11.310 INFO:tasks.workunit.client.0.smithi176.stderr:+ queue_length=4 2024-09-11T06:51:11.310 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 0 -lt 10 ']' 2024-09-11T06:51:11.310 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 4 -gt 0 ']' 2024-09-11T06:51:11.310 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd journal inspect --pool rbd --image testrbdnbd21607 --verbose 2024-09-11T06:51:11.311 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:51:11.352 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:11.350+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:11.371 INFO:tasks.workunit.client.0.smithi176.stdout:Entry: tag_id=1, commit_tid=1 2024-09-11T06:51:11.371 INFO:tasks.workunit.client.0.smithi176.stdout:{ 2024-09-11T06:51:11.371 INFO:tasks.workunit.client.0.smithi176.stdout: "event_type": "AioDiscard", 2024-09-11T06:51:11.371 INFO:tasks.workunit.client.0.smithi176.stdout: "offset": 0, 2024-09-11T06:51:11.371 INFO:tasks.workunit.client.0.smithi176.stdout: "length": 4096, 2024-09-11T06:51:11.371 INFO:tasks.workunit.client.0.smithi176.stdout: "discard_granularity_bytes": 4096, 2024-09-11T06:51:11.371 INFO:tasks.workunit.client.0.smithi176.stdout: "timestamp": "2024-09-11T06:51:11.164111+0000" 2024-09-11T06:51:11.371 INFO:tasks.workunit.client.0.smithi176.stdout:} 2024-09-11T06:51:11.372 INFO:tasks.workunit.client.0.smithi176.stdout:Entry: tag_id=1, commit_tid=2 2024-09-11T06:51:11.372 INFO:tasks.workunit.client.0.smithi176.stdout:{ 2024-09-11T06:51:11.372 INFO:tasks.workunit.client.0.smithi176.stdout: "event_type": "AioDiscard", 2024-09-11T06:51:11.372 INFO:tasks.workunit.client.0.smithi176.stdout: "offset": 0, 2024-09-11T06:51:11.372 INFO:tasks.workunit.client.0.smithi176.stdout: "length": 4096, 2024-09-11T06:51:11.372 INFO:tasks.workunit.client.0.smithi176.stdout: "discard_granularity_bytes": 4096, 2024-09-11T06:51:11.372 INFO:tasks.workunit.client.0.smithi176.stdout: "timestamp": "2024-09-11T06:51:11.182249+0000" 2024-09-11T06:51:11.372 INFO:tasks.workunit.client.0.smithi176.stdout:} 2024-09-11T06:51:11.372 INFO:tasks.workunit.client.0.smithi176.stdout:Entry: tag_id=1, commit_tid=3 2024-09-11T06:51:11.372 INFO:tasks.workunit.client.0.smithi176.stdout:{ 2024-09-11T06:51:11.372 INFO:tasks.workunit.client.0.smithi176.stdout: "event_type": "AioDiscard", 2024-09-11T06:51:11.372 INFO:tasks.workunit.client.0.smithi176.stdout: "offset": 4096, 2024-09-11T06:51:11.372 INFO:tasks.workunit.client.0.smithi176.stdout: "length": 4096, 2024-09-11T06:51:11.372 INFO:tasks.workunit.client.0.smithi176.stdout: "discard_granularity_bytes": 4096, 2024-09-11T06:51:11.373 INFO:tasks.workunit.client.0.smithi176.stdout: "timestamp": "2024-09-11T06:51:11.209791+0000" 2024-09-11T06:51:11.373 INFO:tasks.workunit.client.0.smithi176.stdout:} 2024-09-11T06:51:11.373 INFO:tasks.workunit.client.0.smithi176.stdout:Entry: tag_id=1, commit_tid=4 2024-09-11T06:51:11.373 INFO:tasks.workunit.client.0.smithi176.stdout:{ 2024-09-11T06:51:11.373 INFO:tasks.workunit.client.0.smithi176.stdout: "event_type": "AioDiscard", 2024-09-11T06:51:11.373 INFO:tasks.workunit.client.0.smithi176.stdout: "offset": 4096, 2024-09-11T06:51:11.373 INFO:tasks.workunit.client.0.smithi176.stdout: "length": 4096, 2024-09-11T06:51:11.373 INFO:tasks.workunit.client.0.smithi176.stdout: "discard_granularity_bytes": 4096, 2024-09-11T06:51:11.373 INFO:tasks.workunit.client.0.smithi176.stdout: "timestamp": "2024-09-11T06:51:11.226109+0000" 2024-09-11T06:51:11.373 INFO:tasks.workunit.client.0.smithi176.stdout:} 2024-09-11T06:51:11.373 INFO:tasks.workunit.client.0.smithi176.stdout:Summary: 2024-09-11T06:51:11.373 INFO:tasks.workunit.client.0.smithi176.stdout: 4 entries inspected, 0 errors 2024-09-11T06:51:11.374 INFO:tasks.workunit.client.0.smithi176.stderr:2024-09-11T06:51:11.371+0000 7f4a91b59640 -1 JournalMetadata: 0x55e49d25e020 failed to locate client: INSPECT 2024-09-11T06:51:11.380 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 1 2024-09-11T06:51:11.435 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:11.430+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:11.476 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:11.476 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:51:11.476 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:51:11.476 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:51:11.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.476 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:11.477 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:51:11.477 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 125.2005595, 2024-09-11T06:51:11.477 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283336, 2024-09-11T06:51:11.477 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:11.477 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:11.477 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:11.477 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:11.477 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:11.477 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:11.477 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 87 2024-09-11T06:51:11.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.477 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:11.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.477 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:11.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:51:11.478 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.478 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:11.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:51:11.478 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.478 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:11.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271768+0000", 2024-09-11T06:51:11.478 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7599999999999996e-07 2024-09-11T06:51:11.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.478 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:11.479 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271860+0000", 2024-09-11T06:51:11.479 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2639999999999994e-05 2024-09-11T06:51:11.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.479 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.479 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:11.479 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271873+0000", 2024-09-11T06:51:11.479 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3139000000000001e-05 2024-09-11T06:51:11.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.479 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.479 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:11.479 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271882+0000", 2024-09-11T06:51:11.479 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2369999999999992e-06 2024-09-11T06:51:11.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.479 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.479 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:11.480 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277150+0000", 2024-09-11T06:51:11.480 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052684419999999999 2024-09-11T06:51:11.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.480 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:11.480 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277273+0000", 2024-09-11T06:51:11.480 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012278899999999999 2024-09-11T06:51:11.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.480 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:11.480 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277466+0000", 2024-09-11T06:51:11.480 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193578 2024-09-11T06:51:11.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.481 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:11.481 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.280808+0000", 2024-09-11T06:51:11.481 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003341971 2024-09-11T06:51:11.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.481 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:11.481 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284419+0000", 2024-09-11T06:51:11.481 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003610982 2024-09-11T06:51:11.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.481 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:11.482 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284531+0000", 2024-09-11T06:51:11.482 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011121299999999999 2024-09-11T06:51:11.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.482 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:11.482 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284600+0000", 2024-09-11T06:51:11.482 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9592999999999996e-05 2024-09-11T06:51:11.482 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.482 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:11.482 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.482 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:11.482 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:51:11.483 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 125.20042924800001, 2024-09-11T06:51:11.483 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283314, 2024-09-11T06:51:11.483 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:11.483 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:11.483 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:11.483 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:11.483 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:11.483 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:11.483 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 88 2024-09-11T06:51:11.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.483 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:11.483 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.483 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:11.483 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:51:11.484 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.484 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.484 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:11.484 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:51:11.484 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.484 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.484 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:11.484 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271898+0000", 2024-09-11T06:51:11.484 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2999999999999999e-07 2024-09-11T06:51:11.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.484 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.485 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:11.485 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271904+0000", 2024-09-11T06:51:11.485 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.767e-06 2024-09-11T06:51:11.485 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.485 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.485 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:11.485 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271913+0000", 2024-09-11T06:51:11.485 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7200000000000001e-06 2024-09-11T06:51:11.485 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.485 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.485 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:11.485 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271920+0000", 2024-09-11T06:51:11.485 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7100000000000001e-06 2024-09-11T06:51:11.486 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.486 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.486 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:11.486 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278684+0000", 2024-09-11T06:51:11.486 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0067642650000000002 2024-09-11T06:51:11.486 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.486 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.486 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:11.486 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278838+0000", 2024-09-11T06:51:11.486 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015371499999999999 2024-09-11T06:51:11.486 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.486 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.486 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:11.486 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.279040+0000", 2024-09-11T06:51:11.486 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020207499999999999 2024-09-11T06:51:11.487 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.487 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.487 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:11.487 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.281448+0000", 2024-09-11T06:51:11.487 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002407686 2024-09-11T06:51:11.487 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.487 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.487 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:11.487 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284615+0000", 2024-09-11T06:51:11.487 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031675499999999999 2024-09-11T06:51:11.487 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.487 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.487 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:11.488 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284682+0000", 2024-09-11T06:51:11.488 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7032000000000005e-05 2024-09-11T06:51:11.488 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.488 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.488 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:11.488 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284730+0000", 2024-09-11T06:51:11.488 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7790000000000002e-05 2024-09-11T06:51:11.488 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.488 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:11.488 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.488 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.488 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.488 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:11.488 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:51:11.489 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 125.031580048, 2024-09-11T06:51:11.489 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014944941, 2024-09-11T06:51:11.489 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:11.489 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:11.489 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:11.489 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:11.489 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:11.489 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:11.489 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 111 2024-09-11T06:51:11.489 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.489 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:11.489 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.489 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:11.489 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:51:11.490 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.490 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.490 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.490 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:11.490 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:51:11.490 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.490 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.490 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.490 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:11.490 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440748+0000", 2024-09-11T06:51:11.490 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:51:11.490 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.490 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.490 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:11.491 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440760+0000", 2024-09-11T06:51:11.491 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1897e-05 2024-09-11T06:51:11.491 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.491 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.491 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:11.491 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440782+0000", 2024-09-11T06:51:11.491 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2552999999999999e-05 2024-09-11T06:51:11.491 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.491 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.491 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:11.491 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440793+0000", 2024-09-11T06:51:11.491 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0643e-05 2024-09-11T06:51:11.491 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.492 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.492 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:11.492 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449755+0000", 2024-09-11T06:51:11.492 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089622620000000004 2024-09-11T06:51:11.492 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.492 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.492 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:11.492 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449918+0000", 2024-09-11T06:51:11.492 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163086 2024-09-11T06:51:11.492 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.492 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.492 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:11.492 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450150+0000", 2024-09-11T06:51:11.492 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023204500000000001 2024-09-11T06:51:11.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.493 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.493 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:11.493 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454533+0000", 2024-09-11T06:51:11.493 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0043831349999999998 2024-09-11T06:51:11.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.493 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.493 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:11.493 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455511+0000", 2024-09-11T06:51:11.493 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0009776050000000001 2024-09-11T06:51:11.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.493 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.493 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:11.494 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455597+0000", 2024-09-11T06:51:11.494 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6446000000000005e-05 2024-09-11T06:51:11.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.494 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.494 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:11.494 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455691+0000", 2024-09-11T06:51:11.494 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3832000000000006e-05 2024-09-11T06:51:11.494 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.494 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:11.494 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.494 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.494 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:11.494 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:51:11.494 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 125.031510285, 2024-09-11T06:51:11.495 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018885411000000001, 2024-09-11T06:51:11.495 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:11.495 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:11.495 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:11.495 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:11.495 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:11.495 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:11.495 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 112 2024-09-11T06:51:11.495 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.495 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:11.495 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.495 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:11.495 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:51:11.495 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.495 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.496 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.496 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:11.496 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:51:11.496 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.496 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.496 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:11.496 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440817+0000", 2024-09-11T06:51:11.496 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:51:11.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.496 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.496 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:11.496 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441003+0000", 2024-09-11T06:51:11.496 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018661200000000001 2024-09-11T06:51:11.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.497 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.497 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:11.497 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441027+0000", 2024-09-11T06:51:11.497 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3920000000000001e-05 2024-09-11T06:51:11.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.497 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.497 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:11.497 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441062+0000", 2024-09-11T06:51:11.497 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4663000000000003e-05 2024-09-11T06:51:11.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.497 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.497 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:11.498 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451981+0000", 2024-09-11T06:51:11.498 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010918926000000001 2024-09-11T06:51:11.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.498 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.498 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:11.498 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452111+0000", 2024-09-11T06:51:11.498 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129529 2024-09-11T06:51:11.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.498 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.498 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:11.498 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452311+0000", 2024-09-11T06:51:11.498 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020013799999999999 2024-09-11T06:51:11.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.498 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.498 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:11.498 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455809+0000", 2024-09-11T06:51:11.499 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003498443 2024-09-11T06:51:11.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.499 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.499 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:11.499 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459566+0000", 2024-09-11T06:51:11.499 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003757271 2024-09-11T06:51:11.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.499 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.499 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:11.499 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459640+0000", 2024-09-11T06:51:11.499 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3949000000000006e-05 2024-09-11T06:51:11.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.499 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.499 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:11.499 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459701+0000", 2024-09-11T06:51:11.500 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1117000000000005e-05 2024-09-11T06:51:11.500 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.500 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:11.500 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.500 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.500 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:11.500 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:51:11.500 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 125.03123143400001, 2024-09-11T06:51:11.500 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018760915, 2024-09-11T06:51:11.500 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:11.500 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:11.500 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:11.500 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:11.501 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:11.501 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:11.501 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 113 2024-09-11T06:51:11.501 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.501 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:11.501 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.501 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:11.501 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:51:11.501 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.501 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.501 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.501 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:11.502 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:51:11.502 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.502 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.502 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:11.502 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441096+0000", 2024-09-11T06:51:11.502 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4130000000000001e-06 2024-09-11T06:51:11.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.502 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.502 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:11.502 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441110+0000", 2024-09-11T06:51:11.502 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3900000000000001e-05 2024-09-11T06:51:11.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.502 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.502 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:11.502 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441131+0000", 2024-09-11T06:51:11.503 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0636999999999999e-05 2024-09-11T06:51:11.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.503 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:11.503 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441147+0000", 2024-09-11T06:51:11.503 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6512999999999999e-05 2024-09-11T06:51:11.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.503 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:11.503 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453274+0000", 2024-09-11T06:51:11.503 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012126764999999999 2024-09-11T06:51:11.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.503 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:11.503 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453498+0000", 2024-09-11T06:51:11.504 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000223568 2024-09-11T06:51:11.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.504 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.504 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:11.504 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453872+0000", 2024-09-11T06:51:11.504 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037467099999999998 2024-09-11T06:51:11.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.504 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.504 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:11.504 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458709+0000", 2024-09-11T06:51:11.504 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0048365680000000003 2024-09-11T06:51:11.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.504 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.504 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:11.505 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459716+0000", 2024-09-11T06:51:11.505 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010075240000000001 2024-09-11T06:51:11.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.505 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:11.505 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459779+0000", 2024-09-11T06:51:11.505 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2336999999999996e-05 2024-09-11T06:51:11.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.505 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:11.505 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459856+0000", 2024-09-11T06:51:11.505 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7019000000000002e-05 2024-09-11T06:51:11.505 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.505 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:11.505 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.506 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.506 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:11.506 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:51:11.506 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 125.031154695, 2024-09-11T06:51:11.506 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018963114, 2024-09-11T06:51:11.506 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:11.506 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:11.506 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:11.506 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:11.506 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:11.506 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:11.506 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 114 2024-09-11T06:51:11.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.507 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:11.507 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.507 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:11.507 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:51:11.507 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.507 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.507 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.507 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:11.507 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:51:11.507 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.507 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.507 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.507 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:11.507 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:51:11.507 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9299999999999996e-07 2024-09-11T06:51:11.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.508 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.508 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:11.508 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441180+0000", 2024-09-11T06:51:11.508 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7200000000000006e-06 2024-09-11T06:51:11.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.508 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.508 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:11.508 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441265+0000", 2024-09-11T06:51:11.508 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4753000000000007e-05 2024-09-11T06:51:11.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.508 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.508 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:11.508 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441288+0000", 2024-09-11T06:51:11.509 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2860000000000001e-05 2024-09-11T06:51:11.509 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.509 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.509 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:11.509 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454585+0000", 2024-09-11T06:51:11.509 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013297286 2024-09-11T06:51:11.509 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.509 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.509 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:11.509 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454714+0000", 2024-09-11T06:51:11.509 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000128465 2024-09-11T06:51:11.509 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.509 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.509 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:11.509 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454909+0000", 2024-09-11T06:51:11.510 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019522499999999999 2024-09-11T06:51:11.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.510 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:11.510 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459868+0000", 2024-09-11T06:51:11.510 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0049591239999999997 2024-09-11T06:51:11.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.510 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:11.510 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459988+0000", 2024-09-11T06:51:11.510 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000120532 2024-09-11T06:51:11.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.510 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:11.510 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460071+0000", 2024-09-11T06:51:11.511 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2256000000000007e-05 2024-09-11T06:51:11.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.511 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.511 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:11.511 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460135+0000", 2024-09-11T06:51:11.511 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3999999999999997e-05 2024-09-11T06:51:11.511 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.511 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:11.511 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.511 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.511 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:11.511 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:51:11.511 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 125.030997066, 2024-09-11T06:51:11.512 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.019036849000000002, 2024-09-11T06:51:11.512 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:11.512 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:11.512 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:11.512 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:11.512 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:11.512 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:11.512 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 115 2024-09-11T06:51:11.512 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.512 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:11.512 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.512 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:11.512 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:51:11.512 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.513 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.513 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.513 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:11.513 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:51:11.513 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.513 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.513 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.513 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:11.513 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441330+0000", 2024-09-11T06:51:11.513 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1930000000000001e-06 2024-09-11T06:51:11.513 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.513 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.513 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:11.513 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441544+0000", 2024-09-11T06:51:11.514 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021405799999999999 2024-09-11T06:51:11.514 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.514 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.514 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:11.514 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441572+0000", 2024-09-11T06:51:11.514 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7317e-05 2024-09-11T06:51:11.514 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.514 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.514 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:11.514 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441593+0000", 2024-09-11T06:51:11.514 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1702999999999999e-05 2024-09-11T06:51:11.514 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.514 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.514 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:11.514 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455891+0000", 2024-09-11T06:51:11.515 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01429708 2024-09-11T06:51:11.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.515 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.515 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:11.515 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456046+0000", 2024-09-11T06:51:11.515 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000155682 2024-09-11T06:51:11.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.515 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.515 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:11.515 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456242+0000", 2024-09-11T06:51:11.515 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000195692 2024-09-11T06:51:11.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.515 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.515 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:11.516 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459876+0000", 2024-09-11T06:51:11.516 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036336480000000002 2024-09-11T06:51:11.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.516 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.516 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:11.516 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460245+0000", 2024-09-11T06:51:11.516 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036901700000000002 2024-09-11T06:51:11.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.516 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.516 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:11.516 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460316+0000", 2024-09-11T06:51:11.516 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1846000000000003e-05 2024-09-11T06:51:11.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.516 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.516 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:11.517 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460366+0000", 2024-09-11T06:51:11.517 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9613000000000001e-05 2024-09-11T06:51:11.517 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.517 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:11.517 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.517 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.517 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.517 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:11.517 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:51:11.517 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 125.03067498599999, 2024-09-11T06:51:11.517 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01904432, 2024-09-11T06:51:11.517 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:11.517 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:11.517 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:11.518 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:11.518 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:11.518 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:11.518 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 116 2024-09-11T06:51:11.518 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.518 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:11.518 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.518 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:11.518 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:51:11.518 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.518 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.518 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.518 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:11.518 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:51:11.518 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.519 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.519 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.519 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:11.519 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441653+0000", 2024-09-11T06:51:11.519 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:51:11.519 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.519 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.519 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:11.519 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441665+0000", 2024-09-11T06:51:11.519 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1653e-05 2024-09-11T06:51:11.519 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.519 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.519 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:11.519 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441684+0000", 2024-09-11T06:51:11.519 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9490000000000001e-05 2024-09-11T06:51:11.520 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.520 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.520 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:11.520 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441699+0000", 2024-09-11T06:51:11.520 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5044e-05 2024-09-11T06:51:11.520 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.520 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.520 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:11.520 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457297+0000", 2024-09-11T06:51:11.520 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015598241000000001 2024-09-11T06:51:11.520 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.520 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.520 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:11.520 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457577+0000", 2024-09-11T06:51:11.520 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027930799999999998 2024-09-11T06:51:11.521 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.521 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.521 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:11.521 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457957+0000", 2024-09-11T06:51:11.521 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00038088 2024-09-11T06:51:11.521 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.521 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.521 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:11.521 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460432+0000", 2024-09-11T06:51:11.521 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024743349999999998 2024-09-11T06:51:11.521 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.521 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.521 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:11.521 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460584+0000", 2024-09-11T06:51:11.522 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015232900000000001 2024-09-11T06:51:11.522 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.522 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.522 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:11.522 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460646+0000", 2024-09-11T06:51:11.522 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1883000000000004e-05 2024-09-11T06:51:11.522 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.522 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.522 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:11.522 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460696+0000", 2024-09-11T06:51:11.522 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9589999999999998e-05 2024-09-11T06:51:11.522 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.522 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:11.522 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.523 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.523 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.523 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:11.523 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:51:11.523 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 125.03060292000001, 2024-09-11T06:51:11.523 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.021834473, 2024-09-11T06:51:11.523 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:11.523 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:11.523 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:11.523 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:11.523 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:11.523 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:11.523 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 117 2024-09-11T06:51:11.524 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.524 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:11.524 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.524 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:11.524 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:51:11.524 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.524 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.524 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.524 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:11.524 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:51:11.524 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.524 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.524 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.524 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:11.524 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441725+0000", 2024-09-11T06:51:11.525 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5069999999999999e-06 2024-09-11T06:51:11.525 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.525 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.525 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:11.525 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441738+0000", 2024-09-11T06:51:11.525 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2813000000000001e-05 2024-09-11T06:51:11.525 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.525 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.525 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:11.525 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441759+0000", 2024-09-11T06:51:11.525 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1246999999999998e-05 2024-09-11T06:51:11.525 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.525 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.526 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:11.526 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441779+0000", 2024-09-11T06:51:11.526 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0392999999999999e-05 2024-09-11T06:51:11.526 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.526 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.526 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:11.526 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458804+0000", 2024-09-11T06:51:11.526 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.017024182999999998 2024-09-11T06:51:11.526 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.526 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.526 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:11.526 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458977+0000", 2024-09-11T06:51:11.526 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000173926 2024-09-11T06:51:11.526 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.526 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.527 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:11.527 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459170+0000", 2024-09-11T06:51:11.527 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000192184 2024-09-11T06:51:11.527 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.527 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.527 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:11.527 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.462300+0000", 2024-09-11T06:51:11.527 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031304969999999999 2024-09-11T06:51:11.527 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.527 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.527 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:11.527 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463408+0000", 2024-09-11T06:51:11.527 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011077839999999999 2024-09-11T06:51:11.528 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.528 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.528 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:11.528 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463497+0000", 2024-09-11T06:51:11.528 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8828999999999999e-05 2024-09-11T06:51:11.528 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.528 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.528 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:11.528 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463558+0000", 2024-09-11T06:51:11.528 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1110000000000003e-05 2024-09-11T06:51:11.528 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.528 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:11.528 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.528 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.529 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.529 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:11.529 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:51:11.529 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 124.65396141799999, 2024-09-11T06:51:11.529 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013136297, 2024-09-11T06:51:11.529 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:11.529 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:11.529 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:11.529 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:11.529 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:11.529 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:11.529 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 138 2024-09-11T06:51:11.529 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.529 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:11.530 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.530 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:11.530 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:51:11.530 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.530 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.530 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.530 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:11.530 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:51:11.530 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.530 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.530 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.530 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:11.530 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:51:11.530 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.89e-07 2024-09-11T06:51:11.531 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.531 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.531 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:11.531 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818369+0000", 2024-09-11T06:51:11.531 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4170000000000001e-06 2024-09-11T06:51:11.531 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.531 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.531 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:11.531 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818376+0000", 2024-09-11T06:51:11.531 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4270000000000002e-06 2024-09-11T06:51:11.531 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.531 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.531 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:11.531 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818383+0000", 2024-09-11T06:51:11.531 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3060000000000003e-06 2024-09-11T06:51:11.532 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.532 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.532 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:11.532 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824783+0000", 2024-09-11T06:51:11.532 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064003020000000001 2024-09-11T06:51:11.532 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.532 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.532 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:11.532 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825013+0000", 2024-09-11T06:51:11.532 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023039499999999999 2024-09-11T06:51:11.532 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.532 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.532 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:11.532 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825484+0000", 2024-09-11T06:51:11.533 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000470562 2024-09-11T06:51:11.533 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.533 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.533 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:11.533 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828633+0000", 2024-09-11T06:51:11.533 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031496689999999999 2024-09-11T06:51:11.533 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.533 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.533 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:11.533 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831413+0000", 2024-09-11T06:51:11.533 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027793219999999999 2024-09-11T06:51:11.533 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.533 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.534 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:11.534 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831450+0000", 2024-09-11T06:51:11.534 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7370999999999998e-05 2024-09-11T06:51:11.534 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.534 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.534 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:11.534 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831501+0000", 2024-09-11T06:51:11.534 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1036999999999999e-05 2024-09-11T06:51:11.534 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.534 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:11.534 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.534 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.534 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.535 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:11.535 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:51:11.535 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 124.653936938, 2024-09-11T06:51:11.535 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013223011, 2024-09-11T06:51:11.535 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:11.535 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:11.535 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:11.535 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:11.535 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:11.535 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:11.535 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 139 2024-09-11T06:51:11.535 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.535 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:11.535 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.535 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:11.535 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:51:11.536 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.536 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.536 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:11.536 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:51:11.536 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.536 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.536 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:11.536 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818390+0000", 2024-09-11T06:51:11.536 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5600000000000001e-07 2024-09-11T06:51:11.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.536 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.536 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:11.536 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818394+0000", 2024-09-11T06:51:11.537 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5929999999999999e-06 2024-09-11T06:51:11.537 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.537 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.537 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:11.537 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818399+0000", 2024-09-11T06:51:11.537 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7740000000000004e-06 2024-09-11T06:51:11.537 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.537 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.537 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:11.537 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818412+0000", 2024-09-11T06:51:11.537 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2625e-05 2024-09-11T06:51:11.537 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.537 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.537 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:11.537 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826205+0000", 2024-09-11T06:51:11.538 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0077930730000000002 2024-09-11T06:51:11.538 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.538 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.538 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:11.538 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826281+0000", 2024-09-11T06:51:11.538 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5876000000000003e-05 2024-09-11T06:51:11.538 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.538 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.538 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:11.538 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826384+0000", 2024-09-11T06:51:11.538 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103838 2024-09-11T06:51:11.538 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.538 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.538 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:11.538 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830642+0000", 2024-09-11T06:51:11.539 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004257083 2024-09-11T06:51:11.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.539 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.539 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:11.539 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831509+0000", 2024-09-11T06:51:11.539 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086730900000000003 2024-09-11T06:51:11.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.539 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.539 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:11.539 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831547+0000", 2024-09-11T06:51:11.539 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7793000000000002e-05 2024-09-11T06:51:11.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.539 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.539 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:11.539 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831612+0000", 2024-09-11T06:51:11.540 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5691000000000005e-05 2024-09-11T06:51:11.540 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.540 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:11.540 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.540 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.540 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.540 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:11.540 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:51:11.540 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 124.65390824399999, 2024-09-11T06:51:11.540 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016098024999999998, 2024-09-11T06:51:11.540 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:11.540 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:11.540 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:11.540 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:11.540 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:11.541 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:11.541 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 140 2024-09-11T06:51:11.541 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.541 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:11.541 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.541 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:11.541 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:51:11.541 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.541 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.541 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.541 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:11.541 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:51:11.541 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.541 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.542 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.542 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:11.542 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:51:11.542 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5199999999999998e-07 2024-09-11T06:51:11.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.542 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.542 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:11.542 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818479+0000", 2024-09-11T06:51:11.542 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0368000000000002e-05 2024-09-11T06:51:11.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.542 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.542 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:11.542 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818487+0000", 2024-09-11T06:51:11.542 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2339999999999994e-06 2024-09-11T06:51:11.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.543 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.543 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:11.543 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818491+0000", 2024-09-11T06:51:11.543 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4329999999999997e-06 2024-09-11T06:51:11.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.543 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.543 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:11.543 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828080+0000", 2024-09-11T06:51:11.543 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0095890449999999992 2024-09-11T06:51:11.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.543 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.543 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:11.543 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828130+0000", 2024-09-11T06:51:11.543 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9239000000000001e-05 2024-09-11T06:51:11.544 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.544 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.544 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:11.544 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828208+0000", 2024-09-11T06:51:11.544 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.852e-05 2024-09-11T06:51:11.544 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.544 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.544 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:11.544 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831625+0000", 2024-09-11T06:51:11.544 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034169410000000002 2024-09-11T06:51:11.544 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.544 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.544 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:11.544 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834411+0000", 2024-09-11T06:51:11.545 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002786111 2024-09-11T06:51:11.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.545 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.545 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:11.545 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834457+0000", 2024-09-11T06:51:11.545 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6041000000000002e-05 2024-09-11T06:51:11.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.545 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.545 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:11.545 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834516+0000", 2024-09-11T06:51:11.545 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8740999999999999e-05 2024-09-11T06:51:11.545 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.545 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:11.545 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.546 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.546 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.546 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:11.546 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:51:11.546 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 124.653827177, 2024-09-11T06:51:11.546 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016588772000000002, 2024-09-11T06:51:11.546 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:11.546 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:11.546 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:11.546 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:11.546 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:11.546 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:11.546 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 141 2024-09-11T06:51:11.546 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.546 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:11.547 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.547 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:11.547 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:51:11.547 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.547 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.547 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.547 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:11.547 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:51:11.547 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.547 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.547 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.547 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:11.547 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818500+0000", 2024-09-11T06:51:11.547 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0800000000000005e-07 2024-09-11T06:51:11.547 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.547 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.548 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:11.548 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818505+0000", 2024-09-11T06:51:11.548 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2429999999999996e-06 2024-09-11T06:51:11.548 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.548 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.548 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:11.548 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818510+0000", 2024-09-11T06:51:11.548 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9339999999999997e-06 2024-09-11T06:51:11.548 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.548 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.548 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:11.548 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818514+0000", 2024-09-11T06:51:11.548 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7270000000000001e-06 2024-09-11T06:51:11.548 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.549 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.549 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:11.549 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828711+0000", 2024-09-11T06:51:11.549 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010197031 2024-09-11T06:51:11.549 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.549 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.549 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:11.549 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828853+0000", 2024-09-11T06:51:11.549 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014201200000000001 2024-09-11T06:51:11.549 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.549 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.549 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:11.549 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.829115+0000", 2024-09-11T06:51:11.549 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026257500000000003 2024-09-11T06:51:11.549 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.550 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:11.550 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831629+0000", 2024-09-11T06:51:11.550 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025138169999999998 2024-09-11T06:51:11.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.550 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:11.550 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834991+0000", 2024-09-11T06:51:11.550 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033624459999999998 2024-09-11T06:51:11.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.550 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:11.550 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835032+0000", 2024-09-11T06:51:11.550 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1001000000000003e-05 2024-09-11T06:51:11.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.551 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.551 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:11.551 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835088+0000", 2024-09-11T06:51:11.551 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5478e-05 2024-09-11T06:51:11.551 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.551 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:11.551 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.551 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.551 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5235.0:1 2.5 2:b2cc3336:::rbd_id.testrbdnbd21607:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-11T06:51:11.551 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:51:11.551 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 36.913025155, 2024-09-11T06:51:11.551 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.68353462499999995, 2024-09-11T06:51:11.551 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:11.552 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:11.552 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:11.552 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:11.552 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5235", 2024-09-11T06:51:11.552 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2008977164", 2024-09-11T06:51:11.552 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:51:11.552 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.552 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:11.552 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.552 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:11.552 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:51:11.552 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.552 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.552 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.552 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:11.552 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:51:11.553 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.553 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:11.553 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559303+0000", 2024-09-11T06:51:11.553 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1830000000000001e-06 2024-09-11T06:51:11.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.553 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:11.553 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559314+0000", 2024-09-11T06:51:11.553 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0396e-05 2024-09-11T06:51:11.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.553 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:11.554 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559335+0000", 2024-09-11T06:51:11.554 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1183e-05 2024-09-11T06:51:11.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.554 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.554 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:11.554 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559354+0000", 2024-09-11T06:51:11.554 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9225999999999999e-05 2024-09-11T06:51:11.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.554 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.554 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:11.554 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559522+0000", 2024-09-11T06:51:11.554 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000168021 2024-09-11T06:51:11.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.554 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.554 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:51:11.555 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559543+0000", 2024-09-11T06:51:11.555 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0616999999999999e-05 2024-09-11T06:51:11.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.555 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.555 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:11.555 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.238584+0000", 2024-09-11T06:51:11.555 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.67904107499999999 2024-09-11T06:51:11.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.555 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.555 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for missing object", 2024-09-11T06:51:11.555 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.238933+0000", 2024-09-11T06:51:11.555 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034879600000000001 2024-09-11T06:51:11.556 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.556 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.556 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:11.556 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.241968+0000", 2024-09-11T06:51:11.556 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030350830000000001 2024-09-11T06:51:11.556 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.556 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.556 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:11.556 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.242281+0000", 2024-09-11T06:51:11.556 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00031299499999999999 2024-09-11T06:51:11.556 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.556 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.556 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:11.556 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.242836+0000", 2024-09-11T06:51:11.557 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00055504999999999997 2024-09-11T06:51:11.557 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.557 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:11.557 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.557 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.557 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.557 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5235.0:18 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 0~1024 [fadvise_sequential]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T06:51:11.557 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:51:11.557 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 36.210532716000003, 2024-09-11T06:51:11.557 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.048993760999999997, 2024-09-11T06:51:11.557 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:11.557 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:11.557 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:11.558 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:11.558 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5235", 2024-09-11T06:51:11.558 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2008977164", 2024-09-11T06:51:11.558 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 18 2024-09-11T06:51:11.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.558 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:11.558 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.558 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:11.558 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:51:11.558 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.558 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.558 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:11.558 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:51:11.558 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.559 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.559 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.559 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:11.559 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261797+0000", 2024-09-11T06:51:11.559 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2059999999999999e-06 2024-09-11T06:51:11.559 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.559 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.559 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:11.559 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261812+0000", 2024-09-11T06:51:11.559 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5159e-05 2024-09-11T06:51:11.559 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.559 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.559 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:11.559 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261852+0000", 2024-09-11T06:51:11.560 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.9906e-05 2024-09-11T06:51:11.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.560 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:11.560 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261865+0000", 2024-09-11T06:51:11.560 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.312e-05 2024-09-11T06:51:11.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.560 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:11.560 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.263166+0000", 2024-09-11T06:51:11.560 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001301134 2024-09-11T06:51:11.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.560 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for missing object", 2024-09-11T06:51:11.561 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.263529+0000", 2024-09-11T06:51:11.561 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036279900000000002 2024-09-11T06:51:11.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.561 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.561 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:11.561 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.300567+0000", 2024-09-11T06:51:11.561 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.037038326000000003 2024-09-11T06:51:11.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.561 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.561 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:11.561 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.300800+0000", 2024-09-11T06:51:11.561 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023323300000000001 2024-09-11T06:51:11.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.562 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:11.562 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.306709+0000", 2024-09-11T06:51:11.562 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0059087690000000003 2024-09-11T06:51:11.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.562 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:11.562 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.306850+0000", 2024-09-11T06:51:11.562 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014106200000000001 2024-09-11T06:51:11.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.562 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:11.562 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.307121+0000", 2024-09-11T06:51:11.562 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027083599999999999 2024-09-11T06:51:11.563 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.563 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:11.563 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.308809+0000", 2024-09-11T06:51:11.563 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0016880759999999999 2024-09-11T06:51:11.563 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.563 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:11.563 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310382+0000", 2024-09-11T06:51:11.563 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0015732840000000001 2024-09-11T06:51:11.563 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.563 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:11.563 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310501+0000", 2024-09-11T06:51:11.563 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000118276 2024-09-11T06:51:11.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.564 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:11.564 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310787+0000", 2024-09-11T06:51:11.564 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00028657500000000001 2024-09-11T06:51:11.564 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.564 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:11.564 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.564 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:70 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 393216~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:11.564 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596118+0000", 2024-09-11T06:51:11.564 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 22.876208313999999, 2024-09-11T06:51:11.565 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012279145, 2024-09-11T06:51:11.565 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:11.565 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:11.565 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:11.565 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:11.565 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:11.565 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:11.565 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 70 2024-09-11T06:51:11.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.565 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:11.565 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.565 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:11.565 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596118+0000", 2024-09-11T06:51:11.565 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.566 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.566 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:11.566 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596118+0000", 2024-09-11T06:51:11.566 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.566 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.566 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:11.566 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596119+0000", 2024-09-11T06:51:11.566 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0299999999999997e-07 2024-09-11T06:51:11.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.566 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.566 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:11.566 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596238+0000", 2024-09-11T06:51:11.566 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000118798 2024-09-11T06:51:11.567 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.567 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.567 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:11.567 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596250+0000", 2024-09-11T06:51:11.567 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2775999999999999e-05 2024-09-11T06:51:11.567 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.567 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.567 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:11.567 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596259+0000", 2024-09-11T06:51:11.567 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3100000000000001e-06 2024-09-11T06:51:11.567 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.567 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.567 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:11.567 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.603389+0000", 2024-09-11T06:51:11.568 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0071300759999999999 2024-09-11T06:51:11.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.568 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.568 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:11.568 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.603540+0000", 2024-09-11T06:51:11.568 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015111700000000001 2024-09-11T06:51:11.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.568 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.568 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:11.568 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.603766+0000", 2024-09-11T06:51:11.568 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022570899999999999 2024-09-11T06:51:11.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.568 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.569 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:11.569 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.606676+0000", 2024-09-11T06:51:11.569 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0029106380000000001 2024-09-11T06:51:11.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.569 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:11.569 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608173+0000", 2024-09-11T06:51:11.569 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014967629999999999 2024-09-11T06:51:11.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.569 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:11.569 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608277+0000", 2024-09-11T06:51:11.569 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000104398 2024-09-11T06:51:11.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.570 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:11.570 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608397+0000", 2024-09-11T06:51:11.570 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000119657 2024-09-11T06:51:11.570 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.570 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:11.570 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.570 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:71 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 524288~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:11.570 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596272+0000", 2024-09-11T06:51:11.570 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 22.876054061000001, 2024-09-11T06:51:11.570 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013390681999999999, 2024-09-11T06:51:11.571 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:11.571 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:11.571 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:11.571 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:11.571 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:11.571 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:11.571 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 71 2024-09-11T06:51:11.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.571 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:11.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.571 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:11.571 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596272+0000", 2024-09-11T06:51:11.571 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.572 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:11.572 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596272+0000", 2024-09-11T06:51:11.572 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.572 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:11.572 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596273+0000", 2024-09-11T06:51:11.572 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.9999999999999996e-07 2024-09-11T06:51:11.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.572 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:11.572 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596395+0000", 2024-09-11T06:51:11.572 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012204799999999999 2024-09-11T06:51:11.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.573 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:11.573 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596411+0000", 2024-09-11T06:51:11.573 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5746000000000001e-05 2024-09-11T06:51:11.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.573 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:11.573 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596419+0000", 2024-09-11T06:51:11.573 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2600000000000005e-06 2024-09-11T06:51:11.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.573 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:11.573 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.605311+0000", 2024-09-11T06:51:11.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0088915179999999993 2024-09-11T06:51:11.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.574 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:11.574 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.605436+0000", 2024-09-11T06:51:11.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000125203 2024-09-11T06:51:11.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.574 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:11.574 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.605625+0000", 2024-09-11T06:51:11.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018961299999999999 2024-09-11T06:51:11.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.574 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:11.574 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609477+0000", 2024-09-11T06:51:11.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0038513549999999999 2024-09-11T06:51:11.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.575 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:11.575 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609499+0000", 2024-09-11T06:51:11.575 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2215e-05 2024-09-11T06:51:11.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.575 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:11.575 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609589+0000", 2024-09-11T06:51:11.575 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0029000000000001e-05 2024-09-11T06:51:11.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.575 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:11.575 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609663+0000", 2024-09-11T06:51:11.575 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3894999999999996e-05 2024-09-11T06:51:11.576 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.576 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:11.576 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.576 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:72 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 655360~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:11.576 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596432+0000", 2024-09-11T06:51:11.576 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 22.875894574, 2024-09-11T06:51:11.576 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01414727, 2024-09-11T06:51:11.576 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:11.576 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:11.576 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:11.576 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:11.576 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:11.577 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:11.577 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 72 2024-09-11T06:51:11.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.577 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:11.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:11.577 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596432+0000", 2024-09-11T06:51:11.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:11.577 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596432+0000", 2024-09-11T06:51:11.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596433+0000", 2024-09-11T06:51:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1999999999999998e-07 2024-09-11T06:51:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596556+0000", 2024-09-11T06:51:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012379000000000001 2024-09-11T06:51:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596571+0000", 2024-09-11T06:51:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4749999999999999e-05 2024-09-11T06:51:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596579+0000", 2024-09-11T06:51:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2500000000000006e-06 2024-09-11T06:51:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.606784+0000", 2024-09-11T06:51:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010204757 2024-09-11T06:51:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.606950+0000", 2024-09-11T06:51:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000166229 2024-09-11T06:51:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.580 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:11.580 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.607176+0000", 2024-09-11T06:51:11.580 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022549899999999999 2024-09-11T06:51:11.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.580 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:11.580 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610363+0000", 2024-09-11T06:51:11.580 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031874690000000001 2024-09-11T06:51:11.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.580 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:11.580 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610490+0000", 2024-09-11T06:51:11.580 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012639899999999999 2024-09-11T06:51:11.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.581 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:11.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610529+0000", 2024-09-11T06:51:11.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.8936000000000001e-05 2024-09-11T06:51:11.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.581 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:11.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610579+0000", 2024-09-11T06:51:11.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0371000000000003e-05 2024-09-11T06:51:11.581 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.581 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:11.581 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.582 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:73 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 786432~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:11.582 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596592+0000", 2024-09-11T06:51:11.582 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 22.875734320999999, 2024-09-11T06:51:11.582 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014914779, 2024-09-11T06:51:11.582 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:11.582 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:11.582 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:11.582 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:11.582 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:11.582 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:11.582 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 73 2024-09-11T06:51:11.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.583 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:11.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:11.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596592+0000", 2024-09-11T06:51:11.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:11.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596592+0000", 2024-09-11T06:51:11.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:11.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596593+0000", 2024-09-11T06:51:11.584 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.0699999999999996e-07 2024-09-11T06:51:11.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.584 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:11.584 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596786+0000", 2024-09-11T06:51:11.584 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019278200000000001 2024-09-11T06:51:11.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.584 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:11.584 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596801+0000", 2024-09-11T06:51:11.584 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5690000000000001e-05 2024-09-11T06:51:11.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.584 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:11.584 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596815+0000", 2024-09-11T06:51:11.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3753e-05 2024-09-11T06:51:11.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.585 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:11.585 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608442+0000", 2024-09-11T06:51:11.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011627385 2024-09-11T06:51:11.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.585 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:11.585 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608570+0000", 2024-09-11T06:51:11.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012740399999999999 2024-09-11T06:51:11.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.585 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608754+0000", 2024-09-11T06:51:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018392899999999999 2024-09-11T06:51:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610370+0000", 2024-09-11T06:51:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001615998 2024-09-11T06:51:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611420+0000", 2024-09-11T06:51:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010502790000000001 2024-09-11T06:51:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:11.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611458+0000", 2024-09-11T06:51:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7749000000000001e-05 2024-09-11T06:51:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611507+0000", 2024-09-11T06:51:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9002999999999998e-05 2024-09-11T06:51:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:74 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 917504~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596849+0000", 2024-09-11T06:51:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 22.875477683, 2024-09-11T06:51:11.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015202963999999999, 2024-09-11T06:51:11.588 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:11.588 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:11.588 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:11.588 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:11.588 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:11.588 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:11.588 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 74 2024-09-11T06:51:11.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.588 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:11.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.588 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:11.588 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596849+0000", 2024-09-11T06:51:11.588 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.589 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:11.589 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596849+0000", 2024-09-11T06:51:11.589 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:11.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.589 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:11.589 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596850+0000", 2024-09-11T06:51:11.589 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3170000000000001e-06 2024-09-11T06:51:11.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.589 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:11.589 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.597118+0000", 2024-09-11T06:51:11.589 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026772999999999999 2024-09-11T06:51:11.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:11.590 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.597136+0000", 2024-09-11T06:51:11.590 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8467000000000001e-05 2024-09-11T06:51:11.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:11.590 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.597148+0000", 2024-09-11T06:51:11.590 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1486e-05 2024-09-11T06:51:11.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:11.590 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609714+0000", 2024-09-11T06:51:11.591 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012566124 2024-09-11T06:51:11.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.591 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:11.591 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609792+0000", 2024-09-11T06:51:11.591 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.8579999999999996e-05 2024-09-11T06:51:11.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.591 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:11.591 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609899+0000", 2024-09-11T06:51:11.591 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00010716499999999999 2024-09-11T06:51:11.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.591 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:11.591 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610865+0000", 2024-09-11T06:51:11.592 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00096540299999999999 2024-09-11T06:51:11.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.592 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:11.592 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611970+0000", 2024-09-11T06:51:11.592 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001105118 2024-09-11T06:51:11.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.592 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:11.592 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.612005+0000", 2024-09-11T06:51:11.592 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4789000000000002e-05 2024-09-11T06:51:11.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:11.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:11.592 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:11.593 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.612052+0000", 2024-09-11T06:51:11.593 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6785e-05 2024-09-11T06:51:11.593 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.593 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:11.593 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.593 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:11.593 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:11.593 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:11.594 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:51:11.595 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:11.590+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:11.695 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:11.690+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:11.764 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:11.764 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:11.764 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:51:11.764 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:11.778 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:51:11.796 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:11.794+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:11.897 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:11.894+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:11.920 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.2 to 1 (10000) 2024-09-11T06:51:11.923 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is set 2024-09-11T06:51:11.946 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-11T06:51:11.946 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:11.946 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:11.946 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:51:11.947 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:51:11.947 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:11.960 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:51:12.004 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:12.002+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:12.099 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:12.094+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:12.131 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:12.131 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:51:12.131 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:51:12.131 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:51:12.131 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.131 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4558.0:1 3.3 3:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-11T06:51:12.131 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:51:12.131 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 55.037281907999997, 2024-09-11T06:51:12.131 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0085520600000000002, 2024-09-11T06:51:12.131 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.131 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.132 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:12.132 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:12.132 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4558", 2024-09-11T06:51:12.132 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1478388898", 2024-09-11T06:51:12.132 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:51:12.132 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.132 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.132 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.132 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.132 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:51:12.132 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.132 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.133 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.133 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.133 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:51:12.133 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.133 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.133 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.133 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.133 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090533+0000", 2024-09-11T06:51:12.133 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2629999999999999e-06 2024-09-11T06:51:12.133 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.133 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.133 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.133 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090546+0000", 2024-09-11T06:51:12.133 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2918999999999999e-05 2024-09-11T06:51:12.134 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.134 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.134 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.134 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090572+0000", 2024-09-11T06:51:12.134 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6412000000000001e-05 2024-09-11T06:51:12.134 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.134 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.134 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.134 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090594+0000", 2024-09-11T06:51:12.134 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2476e-05 2024-09-11T06:51:12.134 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.134 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.134 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.134 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090938+0000", 2024-09-11T06:51:12.135 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034325099999999998 2024-09-11T06:51:12.135 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.135 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.135 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.135 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.091149+0000", 2024-09-11T06:51:12.135 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021151300000000001 2024-09-11T06:51:12.135 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.135 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.135 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:12.135 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.091377+0000", 2024-09-11T06:51:12.135 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022785399999999999 2024-09-11T06:51:12.135 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.135 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.135 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:12.136 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.092912+0000", 2024-09-11T06:51:12.136 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0015346089999999999 2024-09-11T06:51:12.136 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.136 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.136 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:12.136 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098919+0000", 2024-09-11T06:51:12.136 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0060076799999999996 2024-09-11T06:51:12.136 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.136 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.136 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:12.136 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.099009+0000", 2024-09-11T06:51:12.136 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0056000000000006e-05 2024-09-11T06:51:12.136 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.137 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.137 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.137 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.099081+0000", 2024-09-11T06:51:12.137 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2026999999999993e-05 2024-09-11T06:51:12.137 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.137 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.137 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.137 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.137 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.137 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:51:12.137 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:51:12.137 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 48.995792827999999, 2024-09-11T06:51:12.138 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0071924390000000001, 2024-09-11T06:51:12.138 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.138 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.138 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:12.138 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.138 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.138 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.138 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:51:12.138 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.138 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.138 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.138 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.138 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:51:12.138 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.139 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.139 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.139 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.139 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132020+0000", 2024-09-11T06:51:12.139 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.846e-06 2024-09-11T06:51:12.139 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.139 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.139 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.139 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135688+0000", 2024-09-11T06:51:12.139 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036677350000000001 2024-09-11T06:51:12.139 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.140 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.140 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.140 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135747+0000", 2024-09-11T06:51:12.140 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.9314999999999997e-05 2024-09-11T06:51:12.140 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.140 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.140 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.140 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135797+0000", 2024-09-11T06:51:12.140 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9984000000000001e-05 2024-09-11T06:51:12.140 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.140 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.140 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.140 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136143+0000", 2024-09-11T06:51:12.141 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034594299999999998 2024-09-11T06:51:12.141 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.141 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.141 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.141 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136179+0000", 2024-09-11T06:51:12.141 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5636000000000002e-05 2024-09-11T06:51:12.141 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.141 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.141 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.141 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.139211+0000", 2024-09-11T06:51:12.141 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030319800000000001 2024-09-11T06:51:12.141 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.141 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.141 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.141 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.142 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.142 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:51:12.142 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:51:12.142 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 48.993729574, 2024-09-11T06:51:12.142 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064954139999999997, 2024-09-11T06:51:12.142 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.142 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.142 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:12.142 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.142 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.142 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.142 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:51:12.142 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.143 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.143 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.143 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.143 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:51:12.143 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.143 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.143 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.143 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.143 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134084+0000", 2024-09-11T06:51:12.143 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.894e-06 2024-09-11T06:51:12.143 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.143 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.143 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.143 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136513+0000", 2024-09-11T06:51:12.144 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024290969999999999 2024-09-11T06:51:12.144 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.144 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.144 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.144 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136554+0000", 2024-09-11T06:51:12.144 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.0932e-05 2024-09-11T06:51:12.144 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.144 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.144 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.144 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136584+0000", 2024-09-11T06:51:12.144 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0056000000000002e-05 2024-09-11T06:51:12.144 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.145 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.145 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.145 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136805+0000", 2024-09-11T06:51:12.145 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022086100000000001 2024-09-11T06:51:12.145 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.145 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.145 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.145 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136864+0000", 2024-09-11T06:51:12.145 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.9907000000000001e-05 2024-09-11T06:51:12.145 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.145 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.145 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.145 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.140577+0000", 2024-09-11T06:51:12.146 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0037126669999999998 2024-09-11T06:51:12.146 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.146 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.146 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.146 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.146 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.146 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:12.146 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:51:12.146 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 48.985438774000002, 2024-09-11T06:51:12.146 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035763180000000002, 2024-09-11T06:51:12.146 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.146 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.146 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:12.146 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.147 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.147 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.147 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:51:12.147 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.147 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.147 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.147 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.147 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:51:12.147 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.147 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.147 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.147 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.147 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142374+0000", 2024-09-11T06:51:12.147 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5460000000000001e-06 2024-09-11T06:51:12.148 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.148 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.148 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.148 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143902+0000", 2024-09-11T06:51:12.148 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00152808 2024-09-11T06:51:12.148 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.148 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.148 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.148 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143946+0000", 2024-09-11T06:51:12.148 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3874e-05 2024-09-11T06:51:12.148 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.148 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.148 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.148 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143978+0000", 2024-09-11T06:51:12.149 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2425999999999998e-05 2024-09-11T06:51:12.149 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.149 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.149 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.149 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.144177+0000", 2024-09-11T06:51:12.149 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019837500000000001 2024-09-11T06:51:12.149 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.149 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.149 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.149 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.144198+0000", 2024-09-11T06:51:12.149 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1639000000000001e-05 2024-09-11T06:51:12.149 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.149 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.149 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.150 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.145949+0000", 2024-09-11T06:51:12.150 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0017503779999999999 2024-09-11T06:51:12.150 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.150 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.150 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.150 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.150 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.150 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:12.150 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:51:12.150 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 48.981123072000003, 2024-09-11T06:51:12.150 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0057693220000000003, 2024-09-11T06:51:12.150 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.150 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.151 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:12.151 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.151 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.151 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.151 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:51:12.151 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.151 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.151 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.151 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.151 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:51:12.151 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.151 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.151 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.152 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.152 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146690+0000", 2024-09-11T06:51:12.152 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.607e-06 2024-09-11T06:51:12.152 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.152 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.152 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.152 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148693+0000", 2024-09-11T06:51:12.152 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0020029739999999998 2024-09-11T06:51:12.152 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.152 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.152 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.152 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148735+0000", 2024-09-11T06:51:12.152 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1958e-05 2024-09-11T06:51:12.152 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.153 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.153 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.153 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148764+0000", 2024-09-11T06:51:12.153 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8999000000000001e-05 2024-09-11T06:51:12.153 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.153 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.153 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.153 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149069+0000", 2024-09-11T06:51:12.153 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000305032 2024-09-11T06:51:12.153 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.153 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.153 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.153 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149204+0000", 2024-09-11T06:51:12.154 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013495 2024-09-11T06:51:12.154 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.154 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.154 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.154 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.152458+0000", 2024-09-11T06:51:12.154 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0032538020000000001 2024-09-11T06:51:12.154 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.154 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.154 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.154 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.154 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.154 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:12.154 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:51:12.154 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 48.980224530999998, 2024-09-11T06:51:12.155 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0042425329999999997, 2024-09-11T06:51:12.155 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.155 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.155 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:12.155 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.155 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.155 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.155 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:51:12.155 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.155 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.155 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.155 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.155 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:51:12.155 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.156 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.156 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.156 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.156 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147588+0000", 2024-09-11T06:51:12.156 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.277e-06 2024-09-11T06:51:12.156 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.156 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.156 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.156 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149051+0000", 2024-09-11T06:51:12.156 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001462669 2024-09-11T06:51:12.156 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.156 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.156 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.157 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149090+0000", 2024-09-11T06:51:12.157 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.8967999999999997e-05 2024-09-11T06:51:12.157 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.157 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.157 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.157 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149120+0000", 2024-09-11T06:51:12.157 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0766000000000003e-05 2024-09-11T06:51:12.157 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.157 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.157 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.157 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149395+0000", 2024-09-11T06:51:12.157 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027484900000000002 2024-09-11T06:51:12.157 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.158 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.158 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.158 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149439+0000", 2024-09-11T06:51:12.158 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3940999999999998e-05 2024-09-11T06:51:12.158 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.158 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.158 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.158 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.151829+0000", 2024-09-11T06:51:12.158 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002390063 2024-09-11T06:51:12.158 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.158 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.158 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.159 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.159 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.159 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:12.159 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:51:12.159 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 48.974083899999997, 2024-09-11T06:51:12.159 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0046568219999999997, 2024-09-11T06:51:12.159 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.159 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.159 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:12.159 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.159 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.159 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.159 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:51:12.160 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.160 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.160 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.160 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.160 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:51:12.160 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.160 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.160 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.160 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.160 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153729+0000", 2024-09-11T06:51:12.160 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5629999999999999e-06 2024-09-11T06:51:12.160 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.160 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.161 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.161 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155179+0000", 2024-09-11T06:51:12.161 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001450469 2024-09-11T06:51:12.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.161 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.161 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.161 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155221+0000", 2024-09-11T06:51:12.161 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1448e-05 2024-09-11T06:51:12.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.161 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.161 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.161 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155249+0000", 2024-09-11T06:51:12.161 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8433e-05 2024-09-11T06:51:12.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.162 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.162 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.162 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155420+0000", 2024-09-11T06:51:12.162 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017070599999999999 2024-09-11T06:51:12.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.162 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.162 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.162 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155445+0000", 2024-09-11T06:51:12.162 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4813000000000001e-05 2024-09-11T06:51:12.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.162 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.162 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.162 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.158384+0000", 2024-09-11T06:51:12.163 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0029393900000000001 2024-09-11T06:51:12.163 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.163 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.163 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.163 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.163 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.163 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:12.163 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:51:12.163 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 48.972057151000001, 2024-09-11T06:51:12.163 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0046826020000000001, 2024-09-11T06:51:12.163 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.163 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.163 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:12.164 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.164 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.164 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.164 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:51:12.164 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.164 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.164 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.164 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.164 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:51:12.164 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.164 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.164 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.164 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.164 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155756+0000", 2024-09-11T06:51:12.165 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.384e-06 2024-09-11T06:51:12.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.165 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.165 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.165 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157205+0000", 2024-09-11T06:51:12.165 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014496260000000001 2024-09-11T06:51:12.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.165 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.165 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.165 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157252+0000", 2024-09-11T06:51:12.165 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6368000000000001e-05 2024-09-11T06:51:12.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.165 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.165 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.166 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157286+0000", 2024-09-11T06:51:12.166 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4944999999999998e-05 2024-09-11T06:51:12.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.166 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.166 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.166 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157540+0000", 2024-09-11T06:51:12.166 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00025307999999999998 2024-09-11T06:51:12.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.166 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.166 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.166 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157576+0000", 2024-09-11T06:51:12.166 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.6878999999999997e-05 2024-09-11T06:51:12.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.167 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.167 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.167 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.160437+0000", 2024-09-11T06:51:12.167 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00286032 2024-09-11T06:51:12.167 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.167 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.167 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.167 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.167 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.167 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:51:12.167 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.280679+0000", 2024-09-11T06:51:12.167 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 36.847132162000001, 2024-09-11T06:51:12.167 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0040934309999999998, 2024-09-11T06:51:12.167 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.168 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.168 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:51:12.168 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.168 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.168 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.168 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280679+0000", 2024-09-11T06:51:12.168 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.168 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.168 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.168 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.168 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280679+0000", 2024-09-11T06:51:12.168 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.168 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.169 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.169 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.169 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280681+0000", 2024-09-11T06:51:12.169 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2529999999999999e-06 2024-09-11T06:51:12.169 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.169 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.169 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.169 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280713+0000", 2024-09-11T06:51:12.169 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.1936000000000001e-05 2024-09-11T06:51:12.169 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.169 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.169 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.169 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280758+0000", 2024-09-11T06:51:12.169 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5124999999999999e-05 2024-09-11T06:51:12.170 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.170 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.170 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.170 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280779+0000", 2024-09-11T06:51:12.170 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0936000000000001e-05 2024-09-11T06:51:12.170 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.170 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.170 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.170 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280932+0000", 2024-09-11T06:51:12.170 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015220899999999999 2024-09-11T06:51:12.170 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.170 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.171 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.171 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.284773+0000", 2024-09-11T06:51:12.171 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0038409719999999998 2024-09-11T06:51:12.171 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.171 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.171 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.171 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.171 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.171 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:51:12.171 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.293438+0000", 2024-09-11T06:51:12.171 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 36.834373425999999, 2024-09-11T06:51:12.172 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0061479350000000002, 2024-09-11T06:51:12.172 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.172 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.172 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:51:12.172 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.172 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.172 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.172 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293438+0000", 2024-09-11T06:51:12.172 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.172 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.172 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.172 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.172 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293438+0000", 2024-09-11T06:51:12.173 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.173 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.173 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.173 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.173 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293439+0000", 2024-09-11T06:51:12.173 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:51:12.173 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.173 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.173 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.173 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293459+0000", 2024-09-11T06:51:12.173 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9922e-05 2024-09-11T06:51:12.173 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.174 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.174 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.174 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293482+0000", 2024-09-11T06:51:12.174 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2969e-05 2024-09-11T06:51:12.174 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.174 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.174 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.174 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293494+0000", 2024-09-11T06:51:12.174 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.131e-05 2024-09-11T06:51:12.174 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.174 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.175 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.175 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.295901+0000", 2024-09-11T06:51:12.175 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024077949999999999 2024-09-11T06:51:12.175 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.175 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.175 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.175 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.299586+0000", 2024-09-11T06:51:12.175 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036845020000000001 2024-09-11T06:51:12.175 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.175 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.175 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.175 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.175 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.176 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:51:12.176 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.309418+0000", 2024-09-11T06:51:12.176 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 36.818393395000001, 2024-09-11T06:51:12.176 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031185879999999998, 2024-09-11T06:51:12.176 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.176 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.176 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:51:12.176 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.176 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.176 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.177 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.309418+0000", 2024-09-11T06:51:12.177 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.177 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.177 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.177 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.177 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.309418+0000", 2024-09-11T06:51:12.177 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.177 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.177 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.177 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.177 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.309419+0000", 2024-09-11T06:51:12.177 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4699999999999999e-06 2024-09-11T06:51:12.177 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.178 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.178 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.178 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.309441+0000", 2024-09-11T06:51:12.178 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1668999999999999e-05 2024-09-11T06:51:12.178 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.178 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.178 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.178 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.309463+0000", 2024-09-11T06:51:12.178 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2106000000000001e-05 2024-09-11T06:51:12.178 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.178 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.178 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.178 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.309477+0000", 2024-09-11T06:51:12.178 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3482999999999999e-05 2024-09-11T06:51:12.178 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.179 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.179 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.179 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310475+0000", 2024-09-11T06:51:12.179 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00099831099999999999 2024-09-11T06:51:12.179 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.179 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.179 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.179 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.312536+0000", 2024-09-11T06:51:12.179 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0020615490000000002 2024-09-11T06:51:12.179 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.179 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.179 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.179 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.180 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.180 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:51:12.180 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.324823+0000", 2024-09-11T06:51:12.180 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 36.802988526999997, 2024-09-11T06:51:12.180 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033143130000000002, 2024-09-11T06:51:12.180 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.180 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.180 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:51:12.180 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.180 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.180 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.180 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.324823+0000", 2024-09-11T06:51:12.180 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.181 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.181 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.181 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.181 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.324823+0000", 2024-09-11T06:51:12.181 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.181 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.181 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.181 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.181 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.324825+0000", 2024-09-11T06:51:12.181 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.88e-06 2024-09-11T06:51:12.181 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.181 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.181 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.181 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.324851+0000", 2024-09-11T06:51:12.182 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6126e-05 2024-09-11T06:51:12.182 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.182 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.182 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.182 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.324882+0000", 2024-09-11T06:51:12.182 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0778999999999998e-05 2024-09-11T06:51:12.182 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.182 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.182 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.182 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.324902+0000", 2024-09-11T06:51:12.182 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0046000000000001e-05 2024-09-11T06:51:12.182 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.182 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.182 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.183 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.325015+0000", 2024-09-11T06:51:12.183 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000113739 2024-09-11T06:51:12.183 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.183 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.183 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.183 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.328137+0000", 2024-09-11T06:51:12.183 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031217430000000002 2024-09-11T06:51:12.183 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.183 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.183 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.183 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.183 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.183 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.7 e70/69 cost 1048576)", 2024-09-11T06:51:12.183 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.343042+0000", 2024-09-11T06:51:12.183 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 36.784768909, 2024-09-11T06:51:12.184 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031815010000000002, 2024-09-11T06:51:12.184 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.184 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.184 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:51:12.184 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.184 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.184 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.184 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.343042+0000", 2024-09-11T06:51:12.184 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.184 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.184 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.184 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.184 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.343042+0000", 2024-09-11T06:51:12.185 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.185 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.185 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.185 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.185 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.343044+0000", 2024-09-11T06:51:12.185 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.716e-06 2024-09-11T06:51:12.185 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.185 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.185 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.185 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.343071+0000", 2024-09-11T06:51:12.185 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.654e-05 2024-09-11T06:51:12.185 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.185 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.186 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.186 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.343098+0000", 2024-09-11T06:51:12.186 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7529e-05 2024-09-11T06:51:12.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.186 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.186 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.186 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.343115+0000", 2024-09-11T06:51:12.186 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6629e-05 2024-09-11T06:51:12.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.186 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.186 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.186 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.343229+0000", 2024-09-11T06:51:12.186 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011443900000000001 2024-09-11T06:51:12.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.187 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.187 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.187 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.346224+0000", 2024-09-11T06:51:12.187 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0029946479999999999 2024-09-11T06:51:12.187 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.187 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.187 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.187 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.187 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.187 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.7 e70/69 cost 1048576)", 2024-09-11T06:51:12.187 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.352608+0000", 2024-09-11T06:51:12.187 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 36.775203625000003, 2024-09-11T06:51:12.187 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0032189060000000001, 2024-09-11T06:51:12.187 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.188 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.188 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:51:12.188 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.188 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.188 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.188 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.352608+0000", 2024-09-11T06:51:12.188 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.188 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.188 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.188 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.188 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.352608+0000", 2024-09-11T06:51:12.188 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.188 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.189 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.189 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.189 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.352610+0000", 2024-09-11T06:51:12.189 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.627e-06 2024-09-11T06:51:12.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.189 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.189 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.189 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.352641+0000", 2024-09-11T06:51:12.189 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0442000000000001e-05 2024-09-11T06:51:12.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.189 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.189 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.189 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.352673+0000", 2024-09-11T06:51:12.189 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2493000000000002e-05 2024-09-11T06:51:12.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.190 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.190 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.190 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.352693+0000", 2024-09-11T06:51:12.190 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9911999999999998e-05 2024-09-11T06:51:12.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.190 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.190 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.190 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.352785+0000", 2024-09-11T06:51:12.190 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.1539999999999994e-05 2024-09-11T06:51:12.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.190 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.190 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.190 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.355827+0000", 2024-09-11T06:51:12.191 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003041892 2024-09-11T06:51:12.191 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.191 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.191 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.191 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.191 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.191 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.7 e70/69 cost 1048576)", 2024-09-11T06:51:12.191 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.379470+0000", 2024-09-11T06:51:12.191 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 36.748340831999997, 2024-09-11T06:51:12.191 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0029267730000000001, 2024-09-11T06:51:12.191 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.191 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.191 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:51:12.191 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.192 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.192 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.192 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.379470+0000", 2024-09-11T06:51:12.192 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.192 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.192 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.192 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.192 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.379470+0000", 2024-09-11T06:51:12.192 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.192 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.192 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.192 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.193 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.379472+0000", 2024-09-11T06:51:12.193 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6729999999999999e-06 2024-09-11T06:51:12.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.193 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.193 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.193 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.379494+0000", 2024-09-11T06:51:12.193 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1823000000000001e-05 2024-09-11T06:51:12.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.193 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.193 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.193 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.379521+0000", 2024-09-11T06:51:12.193 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7161999999999998e-05 2024-09-11T06:51:12.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.194 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.194 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.194 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.379538+0000", 2024-09-11T06:51:12.194 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7212999999999999e-05 2024-09-11T06:51:12.194 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.194 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.194 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.194 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.379662+0000", 2024-09-11T06:51:12.194 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000123776 2024-09-11T06:51:12.194 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.194 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.194 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.194 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.382397+0000", 2024-09-11T06:51:12.195 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027351260000000001 2024-09-11T06:51:12.195 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.195 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.195 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.195 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.195 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.195 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(client.5239.0:36 2.7 e91/69 2:eab0161e:::rbd_data.1314f0a2bfa2.0000000000000001:head v 91'133, mlcod=91'130)", 2024-09-11T06:51:12.195 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:08.067580+0000", 2024-09-11T06:51:12.195 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.0602312029999998, 2024-09-11T06:51:12.195 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030056929999999998, 2024-09-11T06:51:12.195 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.195 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.195 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:12.195 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.196 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.196 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.196 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.067580+0000", 2024-09-11T06:51:12.196 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.196 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.196 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.196 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.196 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.067580+0000", 2024-09-11T06:51:12.196 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.196 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.196 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.196 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.196 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.067581+0000", 2024-09-11T06:51:12.196 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2900000000000002e-07 2024-09-11T06:51:12.197 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.197 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.197 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.197 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.067688+0000", 2024-09-11T06:51:12.197 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000107278 2024-09-11T06:51:12.197 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.197 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.197 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.197 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.067703+0000", 2024-09-11T06:51:12.197 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4528999999999999e-05 2024-09-11T06:51:12.197 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.197 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.198 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.198 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.067710+0000", 2024-09-11T06:51:12.198 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4719999999999998e-06 2024-09-11T06:51:12.198 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.198 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.198 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.198 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068389+0000", 2024-09-11T06:51:12.198 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00067867000000000003 2024-09-11T06:51:12.198 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.198 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.198 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.198 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068437+0000", 2024-09-11T06:51:12.198 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.8133999999999997e-05 2024-09-11T06:51:12.198 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.198 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.199 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:12.199 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.070541+0000", 2024-09-11T06:51:12.199 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021035139999999999 2024-09-11T06:51:12.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.199 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.199 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.199 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.070586+0000", 2024-09-11T06:51:12.199 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5167000000000003e-05 2024-09-11T06:51:12.199 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.199 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.199 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.199 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.199 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(client.5239.0:37 2.7 e91/69 2:eab0161e:::rbd_data.1314f0a2bfa2.0000000000000001:head v 91'134, mlcod=91'130)", 2024-09-11T06:51:12.199 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:08.068368+0000", 2024-09-11T06:51:12.200 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.0594433480000003, 2024-09-11T06:51:12.200 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003850118, 2024-09-11T06:51:12.200 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.200 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.200 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:12.200 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.200 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.200 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.200 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068368+0000", 2024-09-11T06:51:12.200 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.200 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.200 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.200 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.201 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068368+0000", 2024-09-11T06:51:12.201 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.201 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.201 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.201 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068369+0000", 2024-09-11T06:51:12.201 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2699999999999998e-07 2024-09-11T06:51:12.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.201 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.201 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.201 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068458+0000", 2024-09-11T06:51:12.201 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9075999999999999e-05 2024-09-11T06:51:12.202 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.202 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.202 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.202 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068471+0000", 2024-09-11T06:51:12.202 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.269e-05 2024-09-11T06:51:12.202 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.202 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.202 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.202 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068480+0000", 2024-09-11T06:51:12.202 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.6199999999999994e-06 2024-09-11T06:51:12.202 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.202 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.202 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.202 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.069523+0000", 2024-09-11T06:51:12.202 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001042465 2024-09-11T06:51:12.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.203 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.203 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.203 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.069542+0000", 2024-09-11T06:51:12.203 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9386e-05 2024-09-11T06:51:12.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.203 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.203 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:12.203 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072122+0000", 2024-09-11T06:51:12.203 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025799540000000002 2024-09-11T06:51:12.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.203 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.203 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.203 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072218+0000", 2024-09-11T06:51:12.204 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.6100000000000005e-05 2024-09-11T06:51:12.204 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.204 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.204 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.204 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.204 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.204 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(client.5239.0:38 2.7 e91/69 2:eab0161e:::rbd_data.1314f0a2bfa2.0000000000000001:head v 91'135, mlcod=91'130)", 2024-09-11T06:51:12.204 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:08.069503+0000", 2024-09-11T06:51:12.204 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.0583085729999997, 2024-09-11T06:51:12.204 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003356295, 2024-09-11T06:51:12.204 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.204 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.204 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:12.204 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.205 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.205 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.205 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.069503+0000", 2024-09-11T06:51:12.205 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.205 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.205 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.205 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.069503+0000", 2024-09-11T06:51:12.205 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.205 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.205 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.205 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.069504+0000", 2024-09-11T06:51:12.205 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.178e-06 2024-09-11T06:51:12.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.205 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.206 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.206 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.069638+0000", 2024-09-11T06:51:12.206 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013433499999999999 2024-09-11T06:51:12.206 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.206 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.206 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.206 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.069655+0000", 2024-09-11T06:51:12.206 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6359e-05 2024-09-11T06:51:12.206 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.206 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.206 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.206 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.069664+0000", 2024-09-11T06:51:12.207 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9049999999999992e-06 2024-09-11T06:51:12.207 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.207 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.207 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.207 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.070018+0000", 2024-09-11T06:51:12.207 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00035356100000000001 2024-09-11T06:51:12.207 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.207 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.207 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.207 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.070042+0000", 2024-09-11T06:51:12.207 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4119999999999999e-05 2024-09-11T06:51:12.207 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.207 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.207 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:12.208 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072772+0000", 2024-09-11T06:51:12.208 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027300929999999998 2024-09-11T06:51:12.208 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.208 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.208 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.208 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072859+0000", 2024-09-11T06:51:12.208 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6743999999999995e-05 2024-09-11T06:51:12.208 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.208 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.208 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.208 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.208 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.208 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(client.5239.0:41 2.7 e91/69 2:eab0161e:::rbd_data.1314f0a2bfa2.0000000000000001:head v 91'138, mlcod=91'132)", 2024-09-11T06:51:12.208 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:08.071899+0000", 2024-09-11T06:51:12.209 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.0559125700000003, 2024-09-11T06:51:12.209 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0037511240000000002, 2024-09-11T06:51:12.209 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.209 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.209 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:12.209 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.209 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.209 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.209 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.071899+0000", 2024-09-11T06:51:12.209 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.209 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.209 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.209 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.209 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.071899+0000", 2024-09-11T06:51:12.210 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.210 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.210 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.210 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.210 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.071900+0000", 2024-09-11T06:51:12.210 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3500000000000005e-07 2024-09-11T06:51:12.210 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.210 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.210 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.210 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072027+0000", 2024-09-11T06:51:12.210 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012720299999999999 2024-09-11T06:51:12.210 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.210 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.210 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.211 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072043+0000", 2024-09-11T06:51:12.211 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6475999999999999e-05 2024-09-11T06:51:12.211 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.211 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.211 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.211 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072053+0000", 2024-09-11T06:51:12.211 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2860000000000005e-06 2024-09-11T06:51:12.211 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.211 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.211 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.211 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072256+0000", 2024-09-11T06:51:12.211 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000203876 2024-09-11T06:51:12.211 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.211 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.211 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.212 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072282+0000", 2024-09-11T06:51:12.212 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5811000000000001e-05 2024-09-11T06:51:12.212 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.212 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.212 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:12.212 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.075544+0000", 2024-09-11T06:51:12.212 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0032621500000000001 2024-09-11T06:51:12.212 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.212 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.212 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.212 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.075650+0000", 2024-09-11T06:51:12.212 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000105487 2024-09-11T06:51:12.212 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.213 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.213 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.213 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.213 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.213 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(client.5239.0:42 2.7 e91/69 2:eab0161e:::rbd_data.1314f0a2bfa2.0000000000000001:head v 91'139, mlcod=91'132)", 2024-09-11T06:51:12.213 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:08.072817+0000", 2024-09-11T06:51:12.213 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.0549939210000003, 2024-09-11T06:51:12.213 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003042346, 2024-09-11T06:51:12.213 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.213 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.213 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:12.213 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.213 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.213 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.214 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072817+0000", 2024-09-11T06:51:12.214 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.214 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.214 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.214 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.214 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072817+0000", 2024-09-11T06:51:12.214 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.214 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.214 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.214 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.214 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072819+0000", 2024-09-11T06:51:12.214 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.553e-06 2024-09-11T06:51:12.214 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.215 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.215 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.215 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072965+0000", 2024-09-11T06:51:12.215 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000145678 2024-09-11T06:51:12.215 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.215 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.215 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.215 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072996+0000", 2024-09-11T06:51:12.215 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.1680999999999997e-05 2024-09-11T06:51:12.215 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.215 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.215 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.215 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.073015+0000", 2024-09-11T06:51:12.215 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8529000000000002e-05 2024-09-11T06:51:12.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.216 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.216 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.216 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.073099+0000", 2024-09-11T06:51:12.216 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4406000000000005e-05 2024-09-11T06:51:12.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.216 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.216 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.216 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.073115+0000", 2024-09-11T06:51:12.216 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5835e-05 2024-09-11T06:51:12.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.216 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.216 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:12.216 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.075810+0000", 2024-09-11T06:51:12.217 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0026946700000000001 2024-09-11T06:51:12.217 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.217 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.217 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.217 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.075860+0000", 2024-09-11T06:51:12.217 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9994000000000003e-05 2024-09-11T06:51:12.217 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.217 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.217 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.217 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.217 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.217 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:12.219 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:51:12.219 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:12.214+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:12.319 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:12.314+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:12.382 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd journal inspect --pool rbd --image testrbdnbd21607 2024-09-11T06:51:12.382 INFO:tasks.workunit.client.0.smithi176.stderr:++ awk '/entries inspected/ {print $1}' 2024-09-11T06:51:12.386 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:12.387 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:12.387 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:51:12.387 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:12.402 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:51:12.448 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:12.445+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:12.468 INFO:tasks.workunit.client.0.smithi176.stderr:2024-09-11T06:51:12.463+0000 7fcacb7fe640 -1 JournalMetadata: 0x5599eb5fb1f0 failed to locate client: INSPECT 2024-09-11T06:51:12.472 INFO:tasks.workunit.client.0.smithi176.stderr:2024-09-11T06:51:12.467+0000 7fcaca7fc640 -1 JournalMetadata: 0x5599eb5fb1f0 failed to locate client: INSPECT 2024-09-11T06:51:12.477 INFO:tasks.workunit.client.0.smithi176.stderr:+ queue_length=4 2024-09-11T06:51:12.477 INFO:tasks.workunit.client.0.smithi176.stderr:+ tries=1 2024-09-11T06:51:12.477 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1 -lt 10 ']' 2024-09-11T06:51:12.477 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 4 -gt 0 ']' 2024-09-11T06:51:12.477 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd journal inspect --pool rbd --image testrbdnbd21607 --verbose 2024-09-11T06:51:12.520 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:12.517+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:12.530 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:12.530 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:12.530 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:51:12.530 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:51:12.530 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:12.545 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:51:12.559 INFO:tasks.workunit.client.0.smithi176.stdout:Entry: tag_id=1, commit_tid=1 2024-09-11T06:51:12.559 INFO:tasks.workunit.client.0.smithi176.stdout:{ 2024-09-11T06:51:12.559 INFO:tasks.workunit.client.0.smithi176.stdout: "event_type": "AioDiscard", 2024-09-11T06:51:12.559 INFO:tasks.workunit.client.0.smithi176.stdout: "offset": 0, 2024-09-11T06:51:12.559 INFO:tasks.workunit.client.0.smithi176.stdout: "length": 4096, 2024-09-11T06:51:12.559 INFO:tasks.workunit.client.0.smithi176.stdout: "discard_granularity_bytes": 4096, 2024-09-11T06:51:12.559 INFO:tasks.workunit.client.0.smithi176.stdout: "timestamp": "2024-09-11T06:51:11.164111+0000" 2024-09-11T06:51:12.560 INFO:tasks.workunit.client.0.smithi176.stdout:} 2024-09-11T06:51:12.560 INFO:tasks.workunit.client.0.smithi176.stdout:Entry: tag_id=1, commit_tid=2 2024-09-11T06:51:12.560 INFO:tasks.workunit.client.0.smithi176.stdout:{ 2024-09-11T06:51:12.560 INFO:tasks.workunit.client.0.smithi176.stdout: "event_type": "AioDiscard", 2024-09-11T06:51:12.560 INFO:tasks.workunit.client.0.smithi176.stdout: "offset": 0, 2024-09-11T06:51:12.560 INFO:tasks.workunit.client.0.smithi176.stdout: "length": 4096, 2024-09-11T06:51:12.560 INFO:tasks.workunit.client.0.smithi176.stdout: "discard_granularity_bytes": 4096, 2024-09-11T06:51:12.560 INFO:tasks.workunit.client.0.smithi176.stdout: "timestamp": "2024-09-11T06:51:11.182249+0000" 2024-09-11T06:51:12.560 INFO:tasks.workunit.client.0.smithi176.stdout:} 2024-09-11T06:51:12.560 INFO:tasks.workunit.client.0.smithi176.stdout:Entry: tag_id=1, commit_tid=3 2024-09-11T06:51:12.560 INFO:tasks.workunit.client.0.smithi176.stdout:{ 2024-09-11T06:51:12.561 INFO:tasks.workunit.client.0.smithi176.stdout: "event_type": "AioDiscard", 2024-09-11T06:51:12.561 INFO:tasks.workunit.client.0.smithi176.stdout: "offset": 4096, 2024-09-11T06:51:12.561 INFO:tasks.workunit.client.0.smithi176.stdout: "length": 4096, 2024-09-11T06:51:12.561 INFO:tasks.workunit.client.0.smithi176.stdout: "discard_granularity_bytes": 4096, 2024-09-11T06:51:12.561 INFO:tasks.workunit.client.0.smithi176.stdout: "timestamp": "2024-09-11T06:51:11.209791+0000" 2024-09-11T06:51:12.561 INFO:tasks.workunit.client.0.smithi176.stdout:} 2024-09-11T06:51:12.561 INFO:tasks.workunit.client.0.smithi176.stdout:Entry: tag_id=1, commit_tid=4 2024-09-11T06:51:12.561 INFO:tasks.workunit.client.0.smithi176.stdout:{ 2024-09-11T06:51:12.561 INFO:tasks.workunit.client.0.smithi176.stdout: "event_type": "AioDiscard", 2024-09-11T06:51:12.561 INFO:tasks.workunit.client.0.smithi176.stdout: "offset": 4096, 2024-09-11T06:51:12.561 INFO:tasks.workunit.client.0.smithi176.stdout: "length": 4096, 2024-09-11T06:51:12.561 INFO:tasks.workunit.client.0.smithi176.stdout: "discard_granularity_bytes": 4096, 2024-09-11T06:51:12.562 INFO:tasks.workunit.client.0.smithi176.stdout: "timestamp": "2024-09-11T06:51:11.226109+0000" 2024-09-11T06:51:12.562 INFO:tasks.workunit.client.0.smithi176.stdout:} 2024-09-11T06:51:12.562 INFO:tasks.workunit.client.0.smithi176.stdout:Summary: 2024-09-11T06:51:12.562 INFO:tasks.workunit.client.0.smithi176.stdout: 4 entries inspected, 0 errors 2024-09-11T06:51:12.563 INFO:tasks.workunit.client.0.smithi176.stderr:2024-09-11T06:51:12.559+0000 7fed409bb640 -1 JournalMetadata: 0x55b2d180c9a0 failed to locate client: INSPECT 2024-09-11T06:51:12.569 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 1 2024-09-11T06:51:12.621 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:12.618+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:12.716 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:12.717 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:51:12.717 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:51:12.717 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:51:12.717 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.717 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:51:12.717 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:51:12.717 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 121.48263805800001, 2024-09-11T06:51:12.717 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010553425999999999, 2024-09-11T06:51:12.717 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.717 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.717 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:12.717 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:12.717 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:51:12.718 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:51:12.718 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:51:12.718 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.718 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.718 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.718 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.718 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:51:12.718 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.718 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.718 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.718 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.718 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:51:12.718 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.718 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.719 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.719 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.719 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230274+0000", 2024-09-11T06:51:12.719 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.533e-06 2024-09-11T06:51:12.719 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.719 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.719 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.719 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230285+0000", 2024-09-11T06:51:12.719 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0540000000000001e-05 2024-09-11T06:51:12.719 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.719 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.719 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.719 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230309+0000", 2024-09-11T06:51:12.719 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4223000000000002e-05 2024-09-11T06:51:12.719 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.720 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.720 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.720 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230327+0000", 2024-09-11T06:51:12.720 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:51:12.720 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.720 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.720 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.720 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230478+0000", 2024-09-11T06:51:12.720 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000151459 2024-09-11T06:51:12.720 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.720 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.720 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:12.720 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230490+0000", 2024-09-11T06:51:12.721 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.147e-05 2024-09-11T06:51:12.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.721 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.721 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.721 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231803+0000", 2024-09-11T06:51:12.721 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001313158 2024-09-11T06:51:12.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.721 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.721 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:51:12.721 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231817+0000", 2024-09-11T06:51:12.721 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4197e-05 2024-09-11T06:51:12.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.721 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.722 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.722 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.235945+0000", 2024-09-11T06:51:12.722 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0041282480000000002 2024-09-11T06:51:12.722 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.722 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.722 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.722 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236312+0000", 2024-09-11T06:51:12.722 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036652700000000001 2024-09-11T06:51:12.722 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.722 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.722 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:12.722 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236689+0000", 2024-09-11T06:51:12.722 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000377117 2024-09-11T06:51:12.723 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.723 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.723 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:12.723 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238460+0000", 2024-09-11T06:51:12.723 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001770918 2024-09-11T06:51:12.723 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.723 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.723 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:12.723 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240627+0000", 2024-09-11T06:51:12.723 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021670779999999998 2024-09-11T06:51:12.723 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.723 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.723 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:12.723 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240751+0000", 2024-09-11T06:51:12.723 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012384900000000001 2024-09-11T06:51:12.724 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.724 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.724 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.724 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240821+0000", 2024-09-11T06:51:12.724 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0516e-05 2024-09-11T06:51:12.724 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.724 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.724 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.724 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.724 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.724 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-11T06:51:12.724 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:51:12.724 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 120.480467215, 2024-09-11T06:51:12.724 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069904770000000001, 2024-09-11T06:51:12.725 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.725 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.725 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:12.725 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:12.725 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:51:12.725 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:51:12.725 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 5 2024-09-11T06:51:12.725 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.725 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.725 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.725 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.725 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:51:12.725 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.726 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.726 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.726 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.726 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:51:12.726 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.726 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.726 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.726 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.726 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232442+0000", 2024-09-11T06:51:12.726 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.044e-06 2024-09-11T06:51:12.726 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.726 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.726 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.726 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232460+0000", 2024-09-11T06:51:12.726 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7863000000000002e-05 2024-09-11T06:51:12.727 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.727 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.727 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.727 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232492+0000", 2024-09-11T06:51:12.727 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2812999999999999e-05 2024-09-11T06:51:12.727 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.727 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.727 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.727 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232510+0000", 2024-09-11T06:51:12.727 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7552999999999999e-05 2024-09-11T06:51:12.727 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.727 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.727 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.727 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232640+0000", 2024-09-11T06:51:12.728 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013029900000000001 2024-09-11T06:51:12.728 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.728 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.728 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:12.728 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232650+0000", 2024-09-11T06:51:12.728 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0023999999999999e-05 2024-09-11T06:51:12.728 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.728 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.728 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.728 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235641+0000", 2024-09-11T06:51:12.728 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00299089 2024-09-11T06:51:12.728 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.728 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.728 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.728 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235980+0000", 2024-09-11T06:51:12.729 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000338907 2024-09-11T06:51:12.729 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.729 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.729 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:12.729 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.236351+0000", 2024-09-11T06:51:12.729 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037087699999999998 2024-09-11T06:51:12.729 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.729 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.729 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:12.729 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238074+0000", 2024-09-11T06:51:12.729 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001722859 2024-09-11T06:51:12.729 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.729 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.730 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:12.730 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239160+0000", 2024-09-11T06:51:12.730 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001086074 2024-09-11T06:51:12.730 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.730 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.730 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:12.730 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239299+0000", 2024-09-11T06:51:12.730 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013933500000000001 2024-09-11T06:51:12.730 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.730 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.730 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.730 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239429+0000", 2024-09-11T06:51:12.730 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129939 2024-09-11T06:51:12.730 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.731 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.731 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.731 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.731 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.731 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-11T06:51:12.731 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:51:12.731 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 119.46637926299999, 2024-09-11T06:51:12.731 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069253250000000004, 2024-09-11T06:51:12.731 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.731 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.731 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:12.731 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:12.731 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:51:12.731 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:51:12.731 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 7 2024-09-11T06:51:12.732 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.732 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.732 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.732 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.732 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:51:12.732 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.732 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.732 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.732 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.732 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:51:12.732 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.732 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.732 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.733 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.733 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246530+0000", 2024-09-11T06:51:12.733 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9469999999999999e-06 2024-09-11T06:51:12.733 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.733 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.733 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.733 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246545+0000", 2024-09-11T06:51:12.733 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5973000000000001e-05 2024-09-11T06:51:12.733 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.733 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.733 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.733 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246578+0000", 2024-09-11T06:51:12.733 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2759999999999998e-05 2024-09-11T06:51:12.733 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.734 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.734 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.734 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246595+0000", 2024-09-11T06:51:12.734 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6939000000000002e-05 2024-09-11T06:51:12.734 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.734 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.734 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.734 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246770+0000", 2024-09-11T06:51:12.734 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000174723 2024-09-11T06:51:12.734 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.735 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.735 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:12.735 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246789+0000", 2024-09-11T06:51:12.735 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9159000000000001e-05 2024-09-11T06:51:12.735 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.735 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.735 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.735 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250337+0000", 2024-09-11T06:51:12.735 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035480759999999998 2024-09-11T06:51:12.735 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.735 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.735 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.735 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250549+0000", 2024-09-11T06:51:12.736 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021192199999999999 2024-09-11T06:51:12.736 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.736 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.736 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:12.736 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250750+0000", 2024-09-11T06:51:12.736 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020104199999999999 2024-09-11T06:51:12.736 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.736 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.736 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:12.736 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.252455+0000", 2024-09-11T06:51:12.736 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001705249 2024-09-11T06:51:12.736 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.736 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.737 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:12.737 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253305+0000", 2024-09-11T06:51:12.737 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00084941200000000002 2024-09-11T06:51:12.737 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.737 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.737 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:12.737 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253388+0000", 2024-09-11T06:51:12.737 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3642999999999999e-05 2024-09-11T06:51:12.737 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.737 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.737 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.737 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253452+0000", 2024-09-11T06:51:12.737 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3479999999999995e-05 2024-09-11T06:51:12.737 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.738 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.738 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.738 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.738 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.738 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:360 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:51:12.738 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.298914+0000", 2024-09-11T06:51:12.738 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 51.413991676000002, 2024-09-11T06:51:12.738 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0023571690000000001, 2024-09-11T06:51:12.738 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.738 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.738 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:12.738 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:12.738 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:51:12.738 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:51:12.739 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 360 2024-09-11T06:51:12.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.739 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.739 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.739 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.739 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298914+0000", 2024-09-11T06:51:12.739 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.739 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.739 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.739 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298914+0000", 2024-09-11T06:51:12.739 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.740 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.740 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.740 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298915+0000", 2024-09-11T06:51:12.740 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9300000000000006e-07 2024-09-11T06:51:12.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.740 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.740 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.740 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298921+0000", 2024-09-11T06:51:12.740 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.6030000000000004e-06 2024-09-11T06:51:12.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.740 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.740 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.740 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298932+0000", 2024-09-11T06:51:12.740 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.082e-05 2024-09-11T06:51:12.741 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.741 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.741 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.741 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.298939+0000", 2024-09-11T06:51:12.741 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7919999999999993e-06 2024-09-11T06:51:12.741 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.741 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.741 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.741 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.300011+0000", 2024-09-11T06:51:12.741 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001071179 2024-09-11T06:51:12.741 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.741 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.741 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.742 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.300245+0000", 2024-09-11T06:51:12.742 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000234434 2024-09-11T06:51:12.742 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.742 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.742 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.742 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.301271+0000", 2024-09-11T06:51:12.742 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010263480000000001 2024-09-11T06:51:12.742 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.742 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.742 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.742 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.742 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.742 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4645.0:363 2.3 2:c43241e0:::rbd_data.1314f0a2bfa2.0000000000000009:head [sparse-read 659456~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-11T06:51:12.743 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:21.302938+0000", 2024-09-11T06:51:12.743 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 51.409967760000001, 2024-09-11T06:51:12.743 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00185012, 2024-09-11T06:51:12.743 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.743 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.743 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:12.743 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:12.743 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4645", 2024-09-11T06:51:12.743 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:51:12.743 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 363 2024-09-11T06:51:12.743 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.743 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.743 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.743 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.744 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302938+0000", 2024-09-11T06:51:12.744 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.744 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.744 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.744 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.744 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302938+0000", 2024-09-11T06:51:12.744 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.744 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.744 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.744 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.744 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302939+0000", 2024-09-11T06:51:12.744 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2300000000000001e-06 2024-09-11T06:51:12.745 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.745 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.745 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.745 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302944+0000", 2024-09-11T06:51:12.745 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.493e-06 2024-09-11T06:51:12.745 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.745 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.745 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.745 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302960+0000", 2024-09-11T06:51:12.745 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.632e-05 2024-09-11T06:51:12.745 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.745 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.745 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.745 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.302972+0000", 2024-09-11T06:51:12.746 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1885000000000001e-05 2024-09-11T06:51:12.746 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.746 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.746 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.746 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.303835+0000", 2024-09-11T06:51:12.746 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000863141 2024-09-11T06:51:12.746 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.746 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.746 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.746 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.303965+0000", 2024-09-11T06:51:12.746 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013032199999999999 2024-09-11T06:51:12.746 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.746 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.747 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.747 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:21.304788+0000", 2024-09-11T06:51:12.747 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00082272900000000002 2024-09-11T06:51:12.747 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.747 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.747 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.747 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.747 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.747 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:57 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:12.747 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.568529+0000", 2024-09-11T06:51:12.747 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 24.144376343000001, 2024-09-11T06:51:12.747 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004228229, 2024-09-11T06:51:12.747 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.748 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.748 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:12.748 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:12.748 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:12.748 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:12.748 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 57 2024-09-11T06:51:12.748 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.748 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.748 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.748 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.748 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568529+0000", 2024-09-11T06:51:12.748 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.748 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.748 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.749 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.749 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568529+0000", 2024-09-11T06:51:12.749 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.749 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.749 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.749 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.749 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568532+0000", 2024-09-11T06:51:12.749 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0000000000000001e-06 2024-09-11T06:51:12.749 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.749 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.749 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.749 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568547+0000", 2024-09-11T06:51:12.749 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4219999999999999e-05 2024-09-11T06:51:12.750 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.750 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.750 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.750 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568580+0000", 2024-09-11T06:51:12.750 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.3328999999999999e-05 2024-09-11T06:51:12.750 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.750 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.750 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.750 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568596+0000", 2024-09-11T06:51:12.750 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5990000000000001e-05 2024-09-11T06:51:12.750 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.750 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.750 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.750 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568728+0000", 2024-09-11T06:51:12.750 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013175400000000001 2024-09-11T06:51:12.751 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.751 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.751 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:12.751 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568738+0000", 2024-09-11T06:51:12.751 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0196e-05 2024-09-11T06:51:12.751 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.751 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.751 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.751 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.572464+0000", 2024-09-11T06:51:12.751 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0037264160000000002 2024-09-11T06:51:12.751 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.751 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.751 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.752 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.572584+0000", 2024-09-11T06:51:12.752 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000119438 2024-09-11T06:51:12.752 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.752 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.752 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.752 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.572758+0000", 2024-09-11T06:51:12.752 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017388599999999999 2024-09-11T06:51:12.752 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.752 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.752 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.752 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.752 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.752 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5368.0:10 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:12.752 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:12.438884+0000", 2024-09-11T06:51:12.753 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.27402195099999999, 2024-09-11T06:51:12.753 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0023168360000000001, 2024-09-11T06:51:12.753 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.753 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.753 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:12.753 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:12.753 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5368", 2024-09-11T06:51:12.753 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3389407979", 2024-09-11T06:51:12.753 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 10 2024-09-11T06:51:12.753 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.753 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.753 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.753 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.753 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.438884+0000", 2024-09-11T06:51:12.753 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.754 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.754 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.754 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.754 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.438884+0000", 2024-09-11T06:51:12.754 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.754 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.754 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.754 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.754 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.438886+0000", 2024-09-11T06:51:12.754 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.142e-06 2024-09-11T06:51:12.754 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.754 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.754 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.754 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.438894+0000", 2024-09-11T06:51:12.755 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4300000000000006e-06 2024-09-11T06:51:12.755 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.755 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.755 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.755 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.438910+0000", 2024-09-11T06:51:12.755 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5182e-05 2024-09-11T06:51:12.755 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.755 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.755 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.755 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.438923+0000", 2024-09-11T06:51:12.755 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.289e-05 2024-09-11T06:51:12.755 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.755 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.756 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.756 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.439020+0000", 2024-09-11T06:51:12.756 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7314999999999999e-05 2024-09-11T06:51:12.756 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.756 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.756 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:12.756 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.439024+0000", 2024-09-11T06:51:12.756 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3270000000000002e-06 2024-09-11T06:51:12.756 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.756 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.756 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.756 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.441009+0000", 2024-09-11T06:51:12.757 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0019845819999999999 2024-09-11T06:51:12.757 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.757 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.757 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.757 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.441108+0000", 2024-09-11T06:51:12.757 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9246000000000005e-05 2024-09-11T06:51:12.757 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.757 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.757 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.757 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.441201+0000", 2024-09-11T06:51:12.757 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2721999999999998e-05 2024-09-11T06:51:12.757 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.757 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.757 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.758 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.758 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.758 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5374.0:1 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.client_register in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:12.758 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:12.452549+0000", 2024-09-11T06:51:12.758 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.26035635299999998, 2024-09-11T06:51:12.758 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025512059999999999, 2024-09-11T06:51:12.758 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.758 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.758 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:12.758 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:12.758 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5374", 2024-09-11T06:51:12.758 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1937077541", 2024-09-11T06:51:12.758 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:51:12.759 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.759 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.759 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.759 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.759 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.452549+0000", 2024-09-11T06:51:12.759 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.759 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.759 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.759 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.759 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.452549+0000", 2024-09-11T06:51:12.759 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.759 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.759 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.760 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.760 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.452551+0000", 2024-09-11T06:51:12.760 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.669e-06 2024-09-11T06:51:12.760 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.760 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.760 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.760 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.452559+0000", 2024-09-11T06:51:12.760 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.79e-06 2024-09-11T06:51:12.760 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.760 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.760 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.760 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.452574+0000", 2024-09-11T06:51:12.760 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.49e-05 2024-09-11T06:51:12.761 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.761 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.761 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.761 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.452587+0000", 2024-09-11T06:51:12.761 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3165e-05 2024-09-11T06:51:12.761 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.761 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.761 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.761 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.452657+0000", 2024-09-11T06:51:12.761 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0333000000000006e-05 2024-09-11T06:51:12.761 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.761 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.762 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.762 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.452742+0000", 2024-09-11T06:51:12.762 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4474000000000005e-05 2024-09-11T06:51:12.762 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.762 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.762 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:12.762 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.453017+0000", 2024-09-11T06:51:12.762 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027567899999999997 2024-09-11T06:51:12.762 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.762 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.762 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:12.762 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.453859+0000", 2024-09-11T06:51:12.762 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00084122000000000005 2024-09-11T06:51:12.763 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.763 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.763 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:12.763 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.455014+0000", 2024-09-11T06:51:12.763 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011554219999999999 2024-09-11T06:51:12.763 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.763 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.763 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:12.763 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.455063+0000", 2024-09-11T06:51:12.763 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9107999999999998e-05 2024-09-11T06:51:12.763 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.763 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.763 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.764 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.455101+0000", 2024-09-11T06:51:12.764 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7446e-05 2024-09-11T06:51:12.764 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.764 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.764 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.764 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.764 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.764 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5374.0:3 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [watch watch cookie 94119567275296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:12.764 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:12.456365+0000", 2024-09-11T06:51:12.764 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.256540983, 2024-09-11T06:51:12.764 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024361690000000002, 2024-09-11T06:51:12.764 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.764 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.765 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:12.765 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:12.765 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5374", 2024-09-11T06:51:12.765 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1937077541", 2024-09-11T06:51:12.765 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 3 2024-09-11T06:51:12.765 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.765 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.765 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.765 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.765 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.456365+0000", 2024-09-11T06:51:12.765 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.765 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.765 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.765 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.765 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.456365+0000", 2024-09-11T06:51:12.765 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.766 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.766 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.766 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.766 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.456366+0000", 2024-09-11T06:51:12.766 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0109999999999999e-06 2024-09-11T06:51:12.766 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.766 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.766 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.766 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.456372+0000", 2024-09-11T06:51:12.766 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.122e-06 2024-09-11T06:51:12.766 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.766 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.766 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.766 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.456386+0000", 2024-09-11T06:51:12.767 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4375999999999999e-05 2024-09-11T06:51:12.767 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.767 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.767 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.767 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.456399+0000", 2024-09-11T06:51:12.767 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2971e-05 2024-09-11T06:51:12.767 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.767 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.767 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.767 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.456508+0000", 2024-09-11T06:51:12.767 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00010916699999999999 2024-09-11T06:51:12.767 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.767 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.768 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.768 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.456621+0000", 2024-09-11T06:51:12.768 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000112518 2024-09-11T06:51:12.768 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.768 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.768 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:12.768 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.456831+0000", 2024-09-11T06:51:12.768 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021017400000000001 2024-09-11T06:51:12.768 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.768 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.768 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:12.768 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.457594+0000", 2024-09-11T06:51:12.768 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00076309900000000005 2024-09-11T06:51:12.768 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.768 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.769 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:12.769 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.458604+0000", 2024-09-11T06:51:12.769 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010100770000000001 2024-09-11T06:51:12.769 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.769 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.769 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:12.769 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.458680+0000", 2024-09-11T06:51:12.769 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5935999999999999e-05 2024-09-11T06:51:12.769 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.769 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.769 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.769 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.458801+0000", 2024-09-11T06:51:12.769 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012071800000000001 2024-09-11T06:51:12.769 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.769 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.770 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.770 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.770 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.770 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5826.0:119 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.client_list in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:12.770 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:12.456613+0000", 2024-09-11T06:51:12.770 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.25629277099999997, 2024-09-11T06:51:12.770 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024080009999999999, 2024-09-11T06:51:12.770 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.770 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.770 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:12.770 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:12.770 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5826", 2024-09-11T06:51:12.770 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2236190565", 2024-09-11T06:51:12.770 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 119 2024-09-11T06:51:12.771 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.771 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.771 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.771 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.771 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.456613+0000", 2024-09-11T06:51:12.771 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.771 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.771 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.771 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.771 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.456613+0000", 2024-09-11T06:51:12.771 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.771 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.771 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.771 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.772 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.456614+0000", 2024-09-11T06:51:12.772 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.09e-07 2024-09-11T06:51:12.772 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.772 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.772 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.772 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.456619+0000", 2024-09-11T06:51:12.772 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7049999999999998e-06 2024-09-11T06:51:12.772 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.772 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.772 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.772 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.456628+0000", 2024-09-11T06:51:12.772 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2760000000000006e-06 2024-09-11T06:51:12.772 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.772 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.772 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.773 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.456634+0000", 2024-09-11T06:51:12.773 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.9780000000000002e-06 2024-09-11T06:51:12.773 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.773 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.773 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.773 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.457173+0000", 2024-09-11T06:51:12.773 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00053871500000000005 2024-09-11T06:51:12.773 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.773 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.773 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:12.773 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.457233+0000", 2024-09-11T06:51:12.773 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0748000000000002e-05 2024-09-11T06:51:12.773 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.773 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.773 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.774 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.458831+0000", 2024-09-11T06:51:12.774 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0015980149999999999 2024-09-11T06:51:12.774 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.774 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.774 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.774 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.458881+0000", 2024-09-11T06:51:12.774 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9908999999999999e-05 2024-09-11T06:51:12.774 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.774 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.774 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.774 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.459021+0000", 2024-09-11T06:51:12.774 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000139746 2024-09-11T06:51:12.774 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.774 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.775 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.775 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.775 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.775 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5374.0:12 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.client_unregister in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:12.775 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:12.464342+0000", 2024-09-11T06:51:12.775 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.248563387, 2024-09-11T06:51:12.775 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001898842, 2024-09-11T06:51:12.775 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.775 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.775 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:12.775 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:12.775 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5374", 2024-09-11T06:51:12.775 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1937077541", 2024-09-11T06:51:12.775 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 12 2024-09-11T06:51:12.776 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.776 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.776 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.776 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.776 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.464342+0000", 2024-09-11T06:51:12.776 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.776 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.776 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.776 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.776 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.464342+0000", 2024-09-11T06:51:12.776 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.776 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.776 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.776 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.776 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.464344+0000", 2024-09-11T06:51:12.777 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1990000000000001e-06 2024-09-11T06:51:12.777 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.777 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.777 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.777 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.464350+0000", 2024-09-11T06:51:12.777 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1650000000000003e-06 2024-09-11T06:51:12.777 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.777 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.777 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.777 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.464365+0000", 2024-09-11T06:51:12.777 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5523999999999999e-05 2024-09-11T06:51:12.777 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.777 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.777 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.778 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.464373+0000", 2024-09-11T06:51:12.778 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.8739999999999995e-06 2024-09-11T06:51:12.778 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.778 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.778 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.778 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.464424+0000", 2024-09-11T06:51:12.778 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0513999999999998e-05 2024-09-11T06:51:12.778 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.778 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.778 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.778 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.464487+0000", 2024-09-11T06:51:12.778 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3724000000000002e-05 2024-09-11T06:51:12.778 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.778 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.779 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:12.779 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.464691+0000", 2024-09-11T06:51:12.779 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020360199999999999 2024-09-11T06:51:12.779 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.779 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.779 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:12.779 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.465295+0000", 2024-09-11T06:51:12.779 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00060436400000000003 2024-09-11T06:51:12.779 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.779 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.779 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:12.780 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.466097+0000", 2024-09-11T06:51:12.780 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00080153799999999997 2024-09-11T06:51:12.780 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.780 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.780 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:12.780 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.466183+0000", 2024-09-11T06:51:12.780 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6312999999999998e-05 2024-09-11T06:51:12.780 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.780 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.780 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.780 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.466241+0000", 2024-09-11T06:51:12.780 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8025000000000001e-05 2024-09-11T06:51:12.780 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.781 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.781 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.781 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.781 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.781 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5374.0:16 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [watch reconnect cookie 94119567275296 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:12.781 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:12.467864+0000", 2024-09-11T06:51:12.781 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.245041484, 2024-09-11T06:51:12.781 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024610119999999998, 2024-09-11T06:51:12.781 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.781 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.781 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:12.781 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:12.782 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5374", 2024-09-11T06:51:12.782 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1937077541", 2024-09-11T06:51:12.782 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 16 2024-09-11T06:51:12.782 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.782 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.782 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.782 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.782 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.467864+0000", 2024-09-11T06:51:12.782 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.782 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.782 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.782 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.782 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.467864+0000", 2024-09-11T06:51:12.782 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.782 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.783 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.783 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.783 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.467865+0000", 2024-09-11T06:51:12.783 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3600000000000003e-07 2024-09-11T06:51:12.783 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.783 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.783 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.783 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.467868+0000", 2024-09-11T06:51:12.783 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2499999999999998e-06 2024-09-11T06:51:12.783 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.783 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.783 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.783 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.467879+0000", 2024-09-11T06:51:12.783 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0392e-05 2024-09-11T06:51:12.783 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.784 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.784 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.784 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.467886+0000", 2024-09-11T06:51:12.784 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.6399999999999997e-06 2024-09-11T06:51:12.784 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.784 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.784 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.784 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.468214+0000", 2024-09-11T06:51:12.784 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00032801899999999998 2024-09-11T06:51:12.784 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.784 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.784 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.784 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.468267+0000", 2024-09-11T06:51:12.784 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2617999999999997e-05 2024-09-11T06:51:12.784 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.785 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.785 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.785 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.470325+0000", 2024-09-11T06:51:12.785 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0020583569999999998 2024-09-11T06:51:12.785 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.785 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.785 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.785 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.785 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.785 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5374.0:21 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [watch unwatch cookie 94119567275296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:12.785 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:12.469500+0000", 2024-09-11T06:51:12.785 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.24340551499999999, 2024-09-11T06:51:12.785 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0026485419999999998, 2024-09-11T06:51:12.785 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.786 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.786 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:12.786 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:12.786 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5374", 2024-09-11T06:51:12.786 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1937077541", 2024-09-11T06:51:12.786 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 21 2024-09-11T06:51:12.786 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.786 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.786 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.786 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.786 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469500+0000", 2024-09-11T06:51:12.786 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.786 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.786 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.787 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.787 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469500+0000", 2024-09-11T06:51:12.787 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.787 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.787 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.787 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.787 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469501+0000", 2024-09-11T06:51:12.787 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.33e-07 2024-09-11T06:51:12.787 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.787 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.787 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.787 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469506+0000", 2024-09-11T06:51:12.787 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.8690000000000003e-06 2024-09-11T06:51:12.787 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.787 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.788 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.788 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469519+0000", 2024-09-11T06:51:12.788 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3077e-05 2024-09-11T06:51:12.788 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.788 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.788 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.788 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469529+0000", 2024-09-11T06:51:12.788 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.8060000000000006e-06 2024-09-11T06:51:12.788 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.788 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.788 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.788 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469853+0000", 2024-09-11T06:51:12.788 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00032408399999999998 2024-09-11T06:51:12.789 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.789 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.789 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.789 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469915+0000", 2024-09-11T06:51:12.789 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2379999999999996e-05 2024-09-11T06:51:12.789 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.789 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.789 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:12.789 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.470021+0000", 2024-09-11T06:51:12.789 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00010504400000000001 2024-09-11T06:51:12.789 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.789 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.789 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:12.790 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.471499+0000", 2024-09-11T06:51:12.790 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014781989999999999 2024-09-11T06:51:12.790 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.790 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.790 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:12.790 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.471932+0000", 2024-09-11T06:51:12.790 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00043362499999999998 2024-09-11T06:51:12.790 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.790 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.790 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:12.790 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.472003+0000", 2024-09-11T06:51:12.790 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1060000000000001e-05 2024-09-11T06:51:12.790 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.791 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.791 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.791 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.472149+0000", 2024-09-11T06:51:12.791 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000145465 2024-09-11T06:51:12.791 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.791 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.791 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.791 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.791 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.791 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5826.0:125 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.client_list in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:12.791 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:12.469719+0000", 2024-09-11T06:51:12.791 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.243186861, 2024-09-11T06:51:12.791 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0026625149999999998, 2024-09-11T06:51:12.792 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.792 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.792 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:12.792 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:12.792 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5826", 2024-09-11T06:51:12.792 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2236190565", 2024-09-11T06:51:12.792 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 125 2024-09-11T06:51:12.792 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.792 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.792 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.792 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.792 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469719+0000", 2024-09-11T06:51:12.792 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.792 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.792 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.793 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.793 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469719+0000", 2024-09-11T06:51:12.793 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.793 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.793 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.793 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.793 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469720+0000", 2024-09-11T06:51:12.793 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.097e-06 2024-09-11T06:51:12.793 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.793 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.793 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.793 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469726+0000", 2024-09-11T06:51:12.793 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2870000000000001e-06 2024-09-11T06:51:12.793 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.794 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.794 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.794 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469739+0000", 2024-09-11T06:51:12.794 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2789000000000001e-05 2024-09-11T06:51:12.794 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.794 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.794 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.794 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469749+0000", 2024-09-11T06:51:12.794 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.6879999999999994e-06 2024-09-11T06:51:12.794 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.794 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.794 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.794 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.470359+0000", 2024-09-11T06:51:12.795 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00060988299999999997 2024-09-11T06:51:12.795 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.795 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.795 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:12.795 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.470428+0000", 2024-09-11T06:51:12.795 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9696999999999994e-05 2024-09-11T06:51:12.795 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.795 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.795 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.795 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.472195+0000", 2024-09-11T06:51:12.795 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001767034 2024-09-11T06:51:12.796 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.796 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.796 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.796 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.472244+0000", 2024-09-11T06:51:12.796 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9038e-05 2024-09-11T06:51:12.796 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.796 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.796 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.796 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.472381+0000", 2024-09-11T06:51:12.796 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000137002 2024-09-11T06:51:12.796 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.796 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.796 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.796 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.797 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.797 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5374.0:22 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.client_list in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:12.797 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:12.469984+0000", 2024-09-11T06:51:12.797 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.24292208600000001, 2024-09-11T06:51:12.797 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027625599999999998, 2024-09-11T06:51:12.797 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.797 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.797 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:12.797 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:12.797 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5374", 2024-09-11T06:51:12.797 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1937077541", 2024-09-11T06:51:12.797 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 22 2024-09-11T06:51:12.797 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.797 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.797 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.798 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.798 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469984+0000", 2024-09-11T06:51:12.798 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.798 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.798 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.798 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.798 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469984+0000", 2024-09-11T06:51:12.798 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.798 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.798 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.798 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.798 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469985+0000", 2024-09-11T06:51:12.798 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.569999999999999e-07 2024-09-11T06:51:12.798 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.798 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.799 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.799 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469990+0000", 2024-09-11T06:51:12.799 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5160000000000002e-06 2024-09-11T06:51:12.799 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.799 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.799 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.799 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.470003+0000", 2024-09-11T06:51:12.799 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2933e-05 2024-09-11T06:51:12.799 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.799 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.799 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.799 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.470012+0000", 2024-09-11T06:51:12.799 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2e-06 2024-09-11T06:51:12.799 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.799 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.800 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.800 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.470453+0000", 2024-09-11T06:51:12.800 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000440963 2024-09-11T06:51:12.800 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.800 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.800 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:12.800 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.470509+0000", 2024-09-11T06:51:12.800 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5937999999999999e-05 2024-09-11T06:51:12.800 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.800 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.800 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.800 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.472419+0000", 2024-09-11T06:51:12.800 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0019100289999999999 2024-09-11T06:51:12.800 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.801 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.801 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.801 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.472523+0000", 2024-09-11T06:51:12.801 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103665 2024-09-11T06:51:12.801 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.801 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.801 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.801 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.472746+0000", 2024-09-11T06:51:12.801 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022335899999999999 2024-09-11T06:51:12.801 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.801 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.801 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.801 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.801 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.802 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5386.0:1 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.client_register in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:12.802 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:12.548934+0000", 2024-09-11T06:51:12.802 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.16397207799999999, 2024-09-11T06:51:12.802 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0023054320000000001, 2024-09-11T06:51:12.802 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.802 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.802 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:12.802 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:12.802 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5386", 2024-09-11T06:51:12.802 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3377056437", 2024-09-11T06:51:12.802 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:51:12.802 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.802 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.802 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.803 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.803 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.548934+0000", 2024-09-11T06:51:12.803 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.803 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.803 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.803 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.803 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.548934+0000", 2024-09-11T06:51:12.803 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.803 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.803 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.803 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.803 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.548937+0000", 2024-09-11T06:51:12.803 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.18e-06 2024-09-11T06:51:12.803 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.803 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.804 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.804 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.548964+0000", 2024-09-11T06:51:12.804 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7084999999999999e-05 2024-09-11T06:51:12.804 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.804 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.804 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.804 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.548987+0000", 2024-09-11T06:51:12.804 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2558999999999999e-05 2024-09-11T06:51:12.804 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.804 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.804 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.804 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.549005+0000", 2024-09-11T06:51:12.804 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8649000000000001e-05 2024-09-11T06:51:12.804 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.804 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.805 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.805 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.549123+0000", 2024-09-11T06:51:12.805 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000118057 2024-09-11T06:51:12.805 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.805 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.805 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.805 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.549211+0000", 2024-09-11T06:51:12.805 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.7991999999999994e-05 2024-09-11T06:51:12.805 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.805 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.805 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:12.805 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.549486+0000", 2024-09-11T06:51:12.805 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027448499999999999 2024-09-11T06:51:12.805 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.806 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.806 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:12.806 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.550199+0000", 2024-09-11T06:51:12.806 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00071287200000000003 2024-09-11T06:51:12.806 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.806 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.806 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:12.806 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.551066+0000", 2024-09-11T06:51:12.806 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086760600000000004 2024-09-11T06:51:12.806 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.806 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.806 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:12.806 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.551167+0000", 2024-09-11T06:51:12.806 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000100714 2024-09-11T06:51:12.807 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.807 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.807 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.807 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.551239+0000", 2024-09-11T06:51:12.807 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2232999999999998e-05 2024-09-11T06:51:12.807 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.807 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.807 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.807 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.807 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.807 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5386.0:3 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [watch watch cookie 94226507267584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:12.807 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:12.552545+0000", 2024-09-11T06:51:12.807 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.16036067100000001, 2024-09-11T06:51:12.807 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021251519999999999, 2024-09-11T06:51:12.808 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.808 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.808 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:12.808 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:12.808 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5386", 2024-09-11T06:51:12.808 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3377056437", 2024-09-11T06:51:12.808 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 3 2024-09-11T06:51:12.808 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.808 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.808 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.808 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.809 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.552545+0000", 2024-09-11T06:51:12.809 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.809 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.809 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.809 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.809 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.552545+0000", 2024-09-11T06:51:12.809 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.809 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.809 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.809 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.809 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.552546+0000", 2024-09-11T06:51:12.809 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1399999999999996e-07 2024-09-11T06:51:12.809 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.810 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.810 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.810 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.552549+0000", 2024-09-11T06:51:12.810 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9579999999999999e-06 2024-09-11T06:51:12.810 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.810 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.810 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.810 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.552556+0000", 2024-09-11T06:51:12.810 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.8940000000000004e-06 2024-09-11T06:51:12.810 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.810 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.810 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.810 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.552560+0000", 2024-09-11T06:51:12.810 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.0169999999999998e-06 2024-09-11T06:51:12.811 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.811 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.811 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.811 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.552795+0000", 2024-09-11T06:51:12.811 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023516699999999999 2024-09-11T06:51:12.811 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.811 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.811 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.811 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.552853+0000", 2024-09-11T06:51:12.811 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7887000000000003e-05 2024-09-11T06:51:12.811 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.811 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.811 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:12.811 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.552958+0000", 2024-09-11T06:51:12.812 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000104916 2024-09-11T06:51:12.812 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.812 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.812 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:12.812 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.553434+0000", 2024-09-11T06:51:12.812 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00047627299999999998 2024-09-11T06:51:12.812 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.812 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.812 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:12.812 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.554564+0000", 2024-09-11T06:51:12.812 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011302599999999999 2024-09-11T06:51:12.812 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.812 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.812 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:12.813 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.554604+0000", 2024-09-11T06:51:12.813 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.0083000000000002e-05 2024-09-11T06:51:12.813 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.813 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.813 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.813 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.554670+0000", 2024-09-11T06:51:12.813 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5883000000000006e-05 2024-09-11T06:51:12.813 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.813 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.813 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.813 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.813 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.813 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5826.0:128 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.client_list in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:12.814 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:12.552983+0000", 2024-09-11T06:51:12.814 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.15992331000000001, 2024-09-11T06:51:12.814 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0022091250000000002, 2024-09-11T06:51:12.814 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.814 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.814 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:12.814 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:12.814 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5826", 2024-09-11T06:51:12.814 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2236190565", 2024-09-11T06:51:12.814 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 128 2024-09-11T06:51:12.814 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.814 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.814 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.814 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.815 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.552983+0000", 2024-09-11T06:51:12.815 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.815 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.815 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.815 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.815 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.552983+0000", 2024-09-11T06:51:12.815 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.815 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.815 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.815 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.815 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.552984+0000", 2024-09-11T06:51:12.815 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.223e-06 2024-09-11T06:51:12.815 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.815 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.816 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.816 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.552989+0000", 2024-09-11T06:51:12.816 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.524e-06 2024-09-11T06:51:12.816 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.816 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.816 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.816 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.553003+0000", 2024-09-11T06:51:12.816 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3726999999999999e-05 2024-09-11T06:51:12.816 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.816 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.816 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.816 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.553012+0000", 2024-09-11T06:51:12.816 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9490000000000002e-06 2024-09-11T06:51:12.816 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.817 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.817 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.817 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.553218+0000", 2024-09-11T06:51:12.817 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000205701 2024-09-11T06:51:12.817 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.817 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.817 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:12.817 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.553344+0000", 2024-09-11T06:51:12.817 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000126606 2024-09-11T06:51:12.817 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.817 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.817 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.817 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.554757+0000", 2024-09-11T06:51:12.817 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014129259999999999 2024-09-11T06:51:12.818 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.818 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.818 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.818 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.554887+0000", 2024-09-11T06:51:12.818 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129982 2024-09-11T06:51:12.818 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.818 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.818 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.818 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.555192+0000", 2024-09-11T06:51:12.818 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00030448699999999999 2024-09-11T06:51:12.818 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.818 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.818 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.818 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.819 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.819 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5386.0:17 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [watch unwatch cookie 94226507267584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:12.819 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:12.563293+0000", 2024-09-11T06:51:12.819 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.14961260100000001, 2024-09-11T06:51:12.819 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0028623239999999999, 2024-09-11T06:51:12.819 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.819 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.819 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:12.819 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:12.819 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5386", 2024-09-11T06:51:12.819 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3377056437", 2024-09-11T06:51:12.819 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 17 2024-09-11T06:51:12.819 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.820 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.820 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.820 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.820 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563293+0000", 2024-09-11T06:51:12.820 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.820 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.820 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.820 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.820 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563293+0000", 2024-09-11T06:51:12.820 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.820 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.820 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.820 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.821 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563294+0000", 2024-09-11T06:51:12.821 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5300000000000004e-07 2024-09-11T06:51:12.821 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.821 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.821 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.821 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563296+0000", 2024-09-11T06:51:12.821 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6079999999999998e-06 2024-09-11T06:51:12.821 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.821 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.821 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.821 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563302+0000", 2024-09-11T06:51:12.821 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2549999999999997e-06 2024-09-11T06:51:12.821 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.822 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.822 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.822 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563305+0000", 2024-09-11T06:51:12.822 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2949999999999998e-06 2024-09-11T06:51:12.822 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.822 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.822 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.822 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563792+0000", 2024-09-11T06:51:12.822 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00048731399999999998 2024-09-11T06:51:12.822 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.822 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.822 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.823 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563899+0000", 2024-09-11T06:51:12.823 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000106977 2024-09-11T06:51:12.823 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.823 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.823 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:12.823 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.564145+0000", 2024-09-11T06:51:12.823 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00024587600000000002 2024-09-11T06:51:12.823 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.823 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.823 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:12.823 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.564760+0000", 2024-09-11T06:51:12.823 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00061473099999999998 2024-09-11T06:51:12.823 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.823 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.824 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:12.824 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566046+0000", 2024-09-11T06:51:12.824 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012856810000000001 2024-09-11T06:51:12.824 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.824 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.824 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:12.824 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566085+0000", 2024-09-11T06:51:12.824 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.9499000000000002e-05 2024-09-11T06:51:12.824 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.824 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.824 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.824 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566156+0000", 2024-09-11T06:51:12.824 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0434999999999999e-05 2024-09-11T06:51:12.824 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.824 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.825 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.825 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.825 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.825 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5826.0:131 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.client_list in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:12.825 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:12.563833+0000", 2024-09-11T06:51:12.825 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.14907304499999999, 2024-09-11T06:51:12.825 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0028764329999999999, 2024-09-11T06:51:12.825 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:12.825 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:12.825 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:12.825 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:12.825 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5826", 2024-09-11T06:51:12.825 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2236190565", 2024-09-11T06:51:12.825 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 131 2024-09-11T06:51:12.825 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.826 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:12.826 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.826 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:12.826 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563833+0000", 2024-09-11T06:51:12.826 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.826 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.826 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.826 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:12.826 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563833+0000", 2024-09-11T06:51:12.826 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:12.826 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.826 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.826 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:12.826 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563835+0000", 2024-09-11T06:51:12.826 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8109999999999999e-06 2024-09-11T06:51:12.826 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.827 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.827 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:12.827 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563840+0000", 2024-09-11T06:51:12.827 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7599999999999999e-06 2024-09-11T06:51:12.827 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.827 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.827 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:12.827 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563854+0000", 2024-09-11T06:51:12.827 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3332999999999999e-05 2024-09-11T06:51:12.827 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.827 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.827 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:12.827 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563874+0000", 2024-09-11T06:51:12.827 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0630999999999999e-05 2024-09-11T06:51:12.827 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.828 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.828 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.828 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.564553+0000", 2024-09-11T06:51:12.828 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00067821199999999998 2024-09-11T06:51:12.828 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.828 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.828 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:12.828 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.564610+0000", 2024-09-11T06:51:12.828 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7494999999999998e-05 2024-09-11T06:51:12.828 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.828 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.828 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:12.828 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566240+0000", 2024-09-11T06:51:12.828 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0016295940000000001 2024-09-11T06:51:12.828 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.829 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.829 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:12.829 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566365+0000", 2024-09-11T06:51:12.829 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000124876 2024-09-11T06:51:12.829 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:12.829 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:12.829 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:12.829 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566709+0000", 2024-09-11T06:51:12.829 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034472099999999998 2024-09-11T06:51:12.829 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.829 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.829 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.829 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:12.829 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:12.829 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:12.831 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:12.825+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:12.831 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:51:12.926 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.4 to 1 (10000) 2024-09-11T06:51:12.931 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:12.926+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:12.949 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-11T06:51:12.967 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:12.967 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:12.967 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:51:12.967 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:12.982 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:51:13.073 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:13.070+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:13.132 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:13.129+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:13.145 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:13.145 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:13.145 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:51:13.145 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:51:13.146 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:13.160 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:51:13.233 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:13.229+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:13.318 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:13.318 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:51:13.318 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:51:13.318 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:51:13.318 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.318 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:54 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'32, mlcod=13'29)", 2024-09-11T06:51:13.318 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:51:13.318 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 127.10465454600001, 2024-09-11T06:51:13.319 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034271670000000001, 2024-09-11T06:51:13.319 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:13.319 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:13.319 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:13.319 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:13.319 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.319 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:13.319 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:51:13.319 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.319 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.319 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.319 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:13.319 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:51:13.320 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.320 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:13.320 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210361+0000", 2024-09-11T06:51:13.320 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1260000000000001e-06 2024-09-11T06:51:13.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.320 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:13.320 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210370+0000", 2024-09-11T06:51:13.320 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8200000000000003e-06 2024-09-11T06:51:13.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.321 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:13.321 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210390+0000", 2024-09-11T06:51:13.321 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0126e-05 2024-09-11T06:51:13.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.321 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:13.321 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210404+0000", 2024-09-11T06:51:13.321 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4589999999999999e-05 2024-09-11T06:51:13.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.321 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:13.321 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210538+0000", 2024-09-11T06:51:13.321 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000133719 2024-09-11T06:51:13.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.322 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.322 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:13.322 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210593+0000", 2024-09-11T06:51:13.322 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5016999999999999e-05 2024-09-11T06:51:13.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.322 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.322 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:13.322 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213695+0000", 2024-09-11T06:51:13.322 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003101866 2024-09-11T06:51:13.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.322 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.322 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:13.323 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213787+0000", 2024-09-11T06:51:13.323 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1903000000000004e-05 2024-09-11T06:51:13.323 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.323 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:13.323 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.323 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.323 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:55 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'33, mlcod=13'29)", 2024-09-11T06:51:13.323 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:51:13.323 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 127.10363530799999, 2024-09-11T06:51:13.324 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.005086091, 2024-09-11T06:51:13.324 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:13.324 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:13.324 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:13.324 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:13.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.324 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:13.324 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:51:13.324 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.324 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:13.324 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:51:13.325 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.325 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:13.325 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211380+0000", 2024-09-11T06:51:13.325 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.097e-06 2024-09-11T06:51:13.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.325 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:13.325 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211537+0000", 2024-09-11T06:51:13.325 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000157369 2024-09-11T06:51:13.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.326 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:13.326 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211559+0000", 2024-09-11T06:51:13.326 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000001e-05 2024-09-11T06:51:13.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.326 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:13.326 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211573+0000", 2024-09-11T06:51:13.326 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4946000000000001e-05 2024-09-11T06:51:13.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.326 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:13.326 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211676+0000", 2024-09-11T06:51:13.327 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0001026 2024-09-11T06:51:13.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.327 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.327 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:13.327 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211709+0000", 2024-09-11T06:51:13.327 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3071999999999998e-05 2024-09-11T06:51:13.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.327 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.327 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:13.327 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216379+0000", 2024-09-11T06:51:13.327 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0046693780000000001 2024-09-11T06:51:13.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.327 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.327 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:13.327 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216465+0000", 2024-09-11T06:51:13.328 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6469000000000002e-05 2024-09-11T06:51:13.328 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.328 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:13.328 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.328 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.328 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:56 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'34, mlcod=13'29)", 2024-09-11T06:51:13.328 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:51:13.328 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 127.10220144900001, 2024-09-11T06:51:13.328 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0060310629999999997, 2024-09-11T06:51:13.328 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:13.328 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:13.328 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:13.328 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:13.329 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.329 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:13.329 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:51:13.329 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.329 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.329 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:13.329 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:51:13.329 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.329 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.329 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:13.329 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212814+0000", 2024-09-11T06:51:13.329 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0869999999999999e-06 2024-09-11T06:51:13.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.330 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.330 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:13.330 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212822+0000", 2024-09-11T06:51:13.330 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0870000000000007e-06 2024-09-11T06:51:13.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.330 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.330 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:13.330 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212840+0000", 2024-09-11T06:51:13.330 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:51:13.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.330 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.330 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:13.331 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212854+0000", 2024-09-11T06:51:13.331 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4003e-05 2024-09-11T06:51:13.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.331 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.331 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:13.331 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213026+0000", 2024-09-11T06:51:13.331 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017207499999999999 2024-09-11T06:51:13.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.331 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.331 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:13.331 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213063+0000", 2024-09-11T06:51:13.331 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7187000000000002e-05 2024-09-11T06:51:13.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.331 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.331 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:13.332 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218762+0000", 2024-09-11T06:51:13.332 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0056993290000000004 2024-09-11T06:51:13.332 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.332 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.332 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:13.332 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218844+0000", 2024-09-11T06:51:13.332 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1701999999999998e-05 2024-09-11T06:51:13.332 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.332 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:13.332 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.332 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.332 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.332 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:60 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'38, mlcod=13'36)", 2024-09-11T06:51:13.332 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:51:13.333 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 127.0882601, 2024-09-11T06:51:13.333 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039254270000000004, 2024-09-11T06:51:13.333 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:13.333 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:13.333 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:13.333 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:13.333 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.333 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:13.333 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:51:13.333 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.333 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.333 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.334 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:13.334 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:51:13.334 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.334 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.334 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.334 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:13.334 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226756+0000", 2024-09-11T06:51:13.334 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-11T06:51:13.334 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.334 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.334 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:13.334 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226768+0000", 2024-09-11T06:51:13.334 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1919999999999999e-05 2024-09-11T06:51:13.334 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.335 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.335 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:13.335 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226797+0000", 2024-09-11T06:51:13.335 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9573e-05 2024-09-11T06:51:13.335 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.335 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.335 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:13.335 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226818+0000", 2024-09-11T06:51:13.335 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0432999999999999e-05 2024-09-11T06:51:13.335 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.335 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.335 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:13.335 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226909+0000", 2024-09-11T06:51:13.336 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0920000000000004e-05 2024-09-11T06:51:13.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.336 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.336 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:13.336 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226936+0000", 2024-09-11T06:51:13.336 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7472999999999999e-05 2024-09-11T06:51:13.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.336 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.336 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:13.336 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230596+0000", 2024-09-11T06:51:13.336 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0036602520000000001 2024-09-11T06:51:13.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.336 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.336 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:13.337 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230680+0000", 2024-09-11T06:51:13.337 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3106e-05 2024-09-11T06:51:13.337 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.337 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:13.337 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.337 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.337 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:107 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'69, mlcod=13'64)", 2024-09-11T06:51:13.337 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:51:13.337 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 126.871812643, 2024-09-11T06:51:13.337 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040728830000000002, 2024-09-11T06:51:13.337 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:13.337 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:13.337 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:13.338 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:13.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.338 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:13.338 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:51:13.338 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.338 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:13.338 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:51:13.338 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.338 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:13.338 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443203+0000", 2024-09-11T06:51:13.339 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1480000000000001e-06 2024-09-11T06:51:13.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.339 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:13.339 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443222+0000", 2024-09-11T06:51:13.339 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9040000000000001e-05 2024-09-11T06:51:13.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.339 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:13.339 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443246+0000", 2024-09-11T06:51:13.339 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4606000000000001e-05 2024-09-11T06:51:13.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.339 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:13.340 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443277+0000", 2024-09-11T06:51:13.340 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0207999999999999e-05 2024-09-11T06:51:13.340 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.340 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.340 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:13.340 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443340+0000", 2024-09-11T06:51:13.340 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3602000000000005e-05 2024-09-11T06:51:13.340 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.340 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.340 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:13.340 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443363+0000", 2024-09-11T06:51:13.340 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2331e-05 2024-09-11T06:51:13.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.341 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.341 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:13.341 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447216+0000", 2024-09-11T06:51:13.341 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0038539690000000001 2024-09-11T06:51:13.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.342 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.342 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:13.342 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447274+0000", 2024-09-11T06:51:13.342 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7979000000000002e-05 2024-09-11T06:51:13.342 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.342 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:13.342 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.342 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.342 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.342 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:111 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'73, mlcod=13'67)", 2024-09-11T06:51:13.342 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:51:13.342 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 126.86530101300001, 2024-09-11T06:51:13.343 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034372360000000002, 2024-09-11T06:51:13.343 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:13.343 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:13.343 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:13.343 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:13.343 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.343 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:13.343 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:51:13.343 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.343 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.343 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:13.343 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:51:13.343 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.344 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.344 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:13.344 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449714+0000", 2024-09-11T06:51:13.344 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5000000000000001e-07 2024-09-11T06:51:13.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.344 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.344 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:13.344 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449721+0000", 2024-09-11T06:51:13.344 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8530000000000003e-06 2024-09-11T06:51:13.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.344 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.344 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:13.345 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449736+0000", 2024-09-11T06:51:13.345 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5114e-05 2024-09-11T06:51:13.345 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.345 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:13.345 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449743+0000", 2024-09-11T06:51:13.345 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0210000000000002e-06 2024-09-11T06:51:13.345 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.345 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:13.345 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449804+0000", 2024-09-11T06:51:13.345 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0479999999999997e-05 2024-09-11T06:51:13.345 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.346 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:13.346 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449826+0000", 2024-09-11T06:51:13.346 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1922000000000001e-05 2024-09-11T06:51:13.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.346 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.346 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:13.346 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453104+0000", 2024-09-11T06:51:13.346 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003278652 2024-09-11T06:51:13.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.346 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.346 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:13.346 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453150+0000", 2024-09-11T06:51:13.346 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6244e-05 2024-09-11T06:51:13.346 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.346 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:13.347 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.347 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.347 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:114 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'76, mlcod=13'71)", 2024-09-11T06:51:13.347 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:51:13.347 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 126.86062294200001, 2024-09-11T06:51:13.347 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0042576660000000002, 2024-09-11T06:51:13.347 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:13.347 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:13.347 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:13.347 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:13.347 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.347 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:13.347 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:51:13.348 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.348 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.348 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.348 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:13.348 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:51:13.348 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.348 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.348 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.348 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:13.348 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454392+0000", 2024-09-11T06:51:13.348 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5999999999999991e-07 2024-09-11T06:51:13.348 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.348 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.349 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:13.349 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454399+0000", 2024-09-11T06:51:13.349 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.878e-06 2024-09-11T06:51:13.349 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.349 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:13.349 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454415+0000", 2024-09-11T06:51:13.349 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5659999999999999e-05 2024-09-11T06:51:13.349 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.349 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:13.349 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454422+0000", 2024-09-11T06:51:13.349 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:51:13.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.350 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:13.350 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454474+0000", 2024-09-11T06:51:13.350 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1860999999999997e-05 2024-09-11T06:51:13.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.350 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:13.350 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454497+0000", 2024-09-11T06:51:13.350 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3686e-05 2024-09-11T06:51:13.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.350 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:13.350 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458552+0000", 2024-09-11T06:51:13.350 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040544550000000002 2024-09-11T06:51:13.351 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.351 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.351 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:13.351 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458649+0000", 2024-09-11T06:51:13.351 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7005999999999999e-05 2024-09-11T06:51:13.351 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.351 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:13.351 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.351 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.351 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.351 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:115 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'77, mlcod=13'72)", 2024-09-11T06:51:13.351 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:51:13.351 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 126.85928233, 2024-09-11T06:51:13.352 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003355584, 2024-09-11T06:51:13.352 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:13.352 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:13.352 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:13.352 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:13.352 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.352 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:13.352 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:51:13.352 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.352 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.353 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:13.353 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:51:13.353 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.353 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.353 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:13.353 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:51:13.353 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5999999999999999e-07 2024-09-11T06:51:13.353 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.353 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:13.353 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455803+0000", 2024-09-11T06:51:13.354 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0310999999999998e-05 2024-09-11T06:51:13.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.354 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.354 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:13.354 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455815+0000", 2024-09-11T06:51:13.354 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2787e-05 2024-09-11T06:51:13.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.354 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.354 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:13.354 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455823+0000", 2024-09-11T06:51:13.354 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5959999999999998e-06 2024-09-11T06:51:13.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.354 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.355 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:13.355 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455875+0000", 2024-09-11T06:51:13.355 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2054e-05 2024-09-11T06:51:13.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.355 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:13.355 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455896+0000", 2024-09-11T06:51:13.355 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0554999999999999e-05 2024-09-11T06:51:13.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.355 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:13.355 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459000+0000", 2024-09-11T06:51:13.355 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003104842 2024-09-11T06:51:13.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.356 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:13.356 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459087+0000", 2024-09-11T06:51:13.356 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6978999999999995e-05 2024-09-11T06:51:13.356 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.356 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:13.356 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.356 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:120 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'82, mlcod=13'78)", 2024-09-11T06:51:13.356 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:51:13.356 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 126.848605045, 2024-09-11T06:51:13.357 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0064612549999999999, 2024-09-11T06:51:13.357 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:13.357 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:13.357 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:13.357 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:13.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.357 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:13.357 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:51:13.357 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.358 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:13.358 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:51:13.358 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.359 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:13.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466410+0000", 2024-09-11T06:51:13.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0389999999999999e-06 2024-09-11T06:51:13.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.359 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:13.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466496+0000", 2024-09-11T06:51:13.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5519000000000006e-05 2024-09-11T06:51:13.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.360 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:13.360 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466510+0000", 2024-09-11T06:51:13.360 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3842999999999999e-05 2024-09-11T06:51:13.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.361 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:13.361 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466517+0000", 2024-09-11T06:51:13.361 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5229999999999998e-06 2024-09-11T06:51:13.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.362 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.363 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:13.363 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466595+0000", 2024-09-11T06:51:13.363 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8019e-05 2024-09-11T06:51:13.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.363 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.363 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:13.363 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466612+0000", 2024-09-11T06:51:13.363 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7013000000000001e-05 2024-09-11T06:51:13.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.363 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.364 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:13.364 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472787+0000", 2024-09-11T06:51:13.364 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0061750269999999996 2024-09-11T06:51:13.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.364 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:13.364 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472870+0000", 2024-09-11T06:51:13.364 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3271999999999998e-05 2024-09-11T06:51:13.364 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.364 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:13.364 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.365 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:121 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'83, mlcod=13'78)", 2024-09-11T06:51:13.365 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:51:13.365 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 126.847380249, 2024-09-11T06:51:13.365 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055613069999999997, 2024-09-11T06:51:13.365 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:13.365 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:13.365 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:13.365 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:13.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.365 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:13.366 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:51:13.366 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.366 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:13.366 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:51:13.366 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.366 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:13.366 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467635+0000", 2024-09-11T06:51:13.366 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6899999999999996e-07 2024-09-11T06:51:13.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.367 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.367 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:13.367 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467727+0000", 2024-09-11T06:51:13.367 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1800999999999997e-05 2024-09-11T06:51:13.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.367 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.367 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:13.367 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467740+0000", 2024-09-11T06:51:13.368 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3747999999999999e-05 2024-09-11T06:51:13.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.368 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.368 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:13.368 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467753+0000", 2024-09-11T06:51:13.368 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2807e-05 2024-09-11T06:51:13.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.368 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.368 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:13.368 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467812+0000", 2024-09-11T06:51:13.368 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8381999999999998e-05 2024-09-11T06:51:13.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.368 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.369 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:13.369 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467828+0000", 2024-09-11T06:51:13.369 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6728e-05 2024-09-11T06:51:13.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.369 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:13.369 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473112+0000", 2024-09-11T06:51:13.369 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052833619999999998 2024-09-11T06:51:13.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.369 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:13.369 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473195+0000", 2024-09-11T06:51:13.370 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3609999999999994e-05 2024-09-11T06:51:13.370 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.370 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:13.370 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.370 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.370 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:131 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'89, mlcod=13'85)", 2024-09-11T06:51:13.370 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:51:13.370 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 126.495974534, 2024-09-11T06:51:13.370 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052492260000000001, 2024-09-11T06:51:13.370 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:13.370 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:13.370 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:13.370 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:13.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.371 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:13.371 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:51:13.371 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.371 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:13.371 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:51:13.371 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.371 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:13.371 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819041+0000", 2024-09-11T06:51:13.371 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4360000000000001e-06 2024-09-11T06:51:13.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.372 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:13.372 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819100+0000", 2024-09-11T06:51:13.372 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9225999999999998e-05 2024-09-11T06:51:13.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.372 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:13.372 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819128+0000", 2024-09-11T06:51:13.372 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.779e-05 2024-09-11T06:51:13.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.372 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:13.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819142+0000", 2024-09-11T06:51:13.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3573e-05 2024-09-11T06:51:13.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:13.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819263+0000", 2024-09-11T06:51:13.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000121086 2024-09-11T06:51:13.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:13.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819292+0000", 2024-09-11T06:51:13.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9363e-05 2024-09-11T06:51:13.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:13.374 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824185+0000", 2024-09-11T06:51:13.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0048924620000000002 2024-09-11T06:51:13.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:13.374 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824289+0000", 2024-09-11T06:51:13.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010429000000000001 2024-09-11T06:51:13.374 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.374 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:13.374 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.374 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:132 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'90, mlcod=13'85)", 2024-09-11T06:51:13.375 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:51:13.375 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 126.495199844, 2024-09-11T06:51:13.375 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.004532342, 2024-09-11T06:51:13.375 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:13.375 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:13.375 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:13.375 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:13.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.375 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:13.376 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:51:13.376 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.376 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:13.376 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:51:13.376 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.376 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:13.376 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819816+0000", 2024-09-11T06:51:13.377 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.61e-06 2024-09-11T06:51:13.377 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.377 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.377 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:13.377 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819827+0000", 2024-09-11T06:51:13.377 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1e-05 2024-09-11T06:51:13.377 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.377 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.377 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:13.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819851+0000", 2024-09-11T06:51:13.378 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4267e-05 2024-09-11T06:51:13.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.378 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:13.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819864+0000", 2024-09-11T06:51:13.378 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2767e-05 2024-09-11T06:51:13.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.378 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:13.379 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820071+0000", 2024-09-11T06:51:13.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000207061 2024-09-11T06:51:13.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.379 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:13.379 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820103+0000", 2024-09-11T06:51:13.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2326999999999997e-05 2024-09-11T06:51:13.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.379 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:13.379 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824302+0000", 2024-09-11T06:51:13.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0041986000000000002 2024-09-11T06:51:13.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:13.380 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824347+0000", 2024-09-11T06:51:13.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4709999999999997e-05 2024-09-11T06:51:13.380 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.380 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:13.380 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.380 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:133 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'91, mlcod=13'86)", 2024-09-11T06:51:13.380 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:51:13.380 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 126.49456818500001, 2024-09-11T06:51:13.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039421459999999997, 2024-09-11T06:51:13.380 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:13.381 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:13.381 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:13.381 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:13.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.381 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:13.381 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:51:13.381 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.381 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:51:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820448+0000", 2024-09-11T06:51:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.84e-06 2024-09-11T06:51:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820463+0000", 2024-09-11T06:51:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5356e-05 2024-09-11T06:51:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820499+0000", 2024-09-11T06:51:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.625e-05 2024-09-11T06:51:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820546+0000", 2024-09-11T06:51:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7017000000000001e-05 2024-09-11T06:51:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820882+0000", 2024-09-11T06:51:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033512 2024-09-11T06:51:13.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.384 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:13.384 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820941+0000", 2024-09-11T06:51:13.384 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8922999999999998e-05 2024-09-11T06:51:13.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.384 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:13.384 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824351+0000", 2024-09-11T06:51:13.384 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034101940000000001 2024-09-11T06:51:13.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.384 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:13.384 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824388+0000", 2024-09-11T06:51:13.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7446e-05 2024-09-11T06:51:13.385 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.385 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:13.385 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.385 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:139 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'97, mlcod=13'86)", 2024-09-11T06:51:13.385 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:51:13.385 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 126.489158207, 2024-09-11T06:51:13.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003732221, 2024-09-11T06:51:13.385 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:13.385 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:13.385 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:13.386 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:13.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.386 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:13.386 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:51:13.386 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.386 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:13.386 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:51:13.386 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.386 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:13.386 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825858+0000", 2024-09-11T06:51:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5829999999999999e-06 2024-09-11T06:51:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825874+0000", 2024-09-11T06:51:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5967000000000001e-05 2024-09-11T06:51:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825909+0000", 2024-09-11T06:51:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5692999999999999e-05 2024-09-11T06:51:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:13.388 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825923+0000", 2024-09-11T06:51:13.388 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4080000000000001e-05 2024-09-11T06:51:13.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.388 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:13.388 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826010+0000", 2024-09-11T06:51:13.388 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6248999999999994e-05 2024-09-11T06:51:13.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.388 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:13.388 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826038+0000", 2024-09-11T06:51:13.388 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8010000000000001e-05 2024-09-11T06:51:13.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.389 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:13.389 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829499+0000", 2024-09-11T06:51:13.389 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003461783 2024-09-11T06:51:13.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.389 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:13.389 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829588+0000", 2024-09-11T06:51:13.389 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8856000000000004e-05 2024-09-11T06:51:13.389 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.389 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:13.389 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.389 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:140 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'98, mlcod=13'93)", 2024-09-11T06:51:13.390 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:51:13.390 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 126.48730747800001, 2024-09-11T06:51:13.390 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0057995929999999996, 2024-09-11T06:51:13.390 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:13.390 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:13.390 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:13.390 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:13.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.390 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:13.390 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:51:13.390 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:51:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827708+0000", 2024-09-11T06:51:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.547e-06 2024-09-11T06:51:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827765+0000", 2024-09-11T06:51:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6266e-05 2024-09-11T06:51:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.392 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:13.392 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827791+0000", 2024-09-11T06:51:13.392 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6098999999999999e-05 2024-09-11T06:51:13.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.392 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:13.392 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827806+0000", 2024-09-11T06:51:13.392 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5537e-05 2024-09-11T06:51:13.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.392 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:13.392 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827928+0000", 2024-09-11T06:51:13.393 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000122012 2024-09-11T06:51:13.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.393 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:13.393 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827961+0000", 2024-09-11T06:51:13.393 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2663999999999998e-05 2024-09-11T06:51:13.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.393 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:13.393 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833409+0000", 2024-09-11T06:51:13.393 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054479350000000001 2024-09-11T06:51:13.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.393 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:13.394 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833506+0000", 2024-09-11T06:51:13.394 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7533000000000003e-05 2024-09-11T06:51:13.394 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.394 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:13.394 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.394 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:141 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'99, mlcod=13'93)", 2024-09-11T06:51:13.394 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:51:13.394 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 126.486414095, 2024-09-11T06:51:13.394 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055394709999999998, 2024-09-11T06:51:13.394 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:13.394 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:13.394 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:13.395 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:13.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:13.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:51:13.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:13.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:51:13.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:13.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828602+0000", 2024-09-11T06:51:13.396 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.46e-06 2024-09-11T06:51:13.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.396 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:13.396 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828618+0000", 2024-09-11T06:51:13.396 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6592999999999999e-05 2024-09-11T06:51:13.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.396 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:13.396 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828646+0000", 2024-09-11T06:51:13.396 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7552999999999999e-05 2024-09-11T06:51:13.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.396 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:13.397 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828663+0000", 2024-09-11T06:51:13.397 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.719e-05 2024-09-11T06:51:13.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.397 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.397 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:13.397 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829023+0000", 2024-09-11T06:51:13.397 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036054799999999998 2024-09-11T06:51:13.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.397 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.397 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:13.398 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829062+0000", 2024-09-11T06:51:13.398 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8865999999999997e-05 2024-09-11T06:51:13.398 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.398 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.398 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:13.398 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834078+0000", 2024-09-11T06:51:13.398 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050157480000000004 2024-09-11T06:51:13.398 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.398 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.398 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:13.398 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834140+0000", 2024-09-11T06:51:13.398 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1513000000000005e-05 2024-09-11T06:51:13.398 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.399 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:13.399 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.399 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:145 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'103, mlcod=13'98)", 2024-09-11T06:51:13.399 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:51:13.399 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 126.476461598, 2024-09-11T06:51:13.399 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054455409999999999, 2024-09-11T06:51:13.399 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:13.399 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:13.399 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:13.399 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:13.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.399 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:13.400 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:51:13.400 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.400 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.400 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:13.400 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:51:13.400 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.400 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.400 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:13.400 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838554+0000", 2024-09-11T06:51:13.400 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3430000000000001e-06 2024-09-11T06:51:13.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.400 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.401 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:13.401 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838640+0000", 2024-09-11T06:51:13.401 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5569999999999996e-05 2024-09-11T06:51:13.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.401 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:13.401 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838660+0000", 2024-09-11T06:51:13.401 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0576000000000001e-05 2024-09-11T06:51:13.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.402 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:13.402 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838674+0000", 2024-09-11T06:51:13.402 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3529999999999999e-05 2024-09-11T06:51:13.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.402 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.402 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:13.402 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838838+0000", 2024-09-11T06:51:13.402 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016468799999999999 2024-09-11T06:51:13.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.402 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.402 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:13.402 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838885+0000", 2024-09-11T06:51:13.402 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6884e-05 2024-09-11T06:51:13.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.403 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.403 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:13.403 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843901+0000", 2024-09-11T06:51:13.403 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050162310000000003 2024-09-11T06:51:13.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.403 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.403 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:13.403 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843998+0000", 2024-09-11T06:51:13.403 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.6718999999999993e-05 2024-09-11T06:51:13.404 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.404 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:13.404 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.404 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.404 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4558.0:1 3.3 e56/55 3:c4c92e5a:::rbd_trash:head v 56'1, mlcod=0'0)", 2024-09-11T06:51:13.404 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:51:13.404 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 56.223658536000002, 2024-09-11T06:51:13.404 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0053791810000000002, 2024-09-11T06:51:13.404 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:13.404 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:13.404 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:13.404 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:13.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.405 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:13.405 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:51:13.405 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.405 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:13.405 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:51:13.405 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.405 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:13.405 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091357+0000", 2024-09-11T06:51:13.405 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.54e-07 2024-09-11T06:51:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091376+0000", 2024-09-11T06:51:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9769e-05 2024-09-11T06:51:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091396+0000", 2024-09-11T06:51:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0152999999999998e-05 2024-09-11T06:51:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091416+0000", 2024-09-11T06:51:13.407 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0006000000000001e-05 2024-09-11T06:51:13.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.407 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:13.407 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091506+0000", 2024-09-11T06:51:13.407 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9325000000000003e-05 2024-09-11T06:51:13.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.407 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:13.407 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091523+0000", 2024-09-11T06:51:13.407 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.77e-05 2024-09-11T06:51:13.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.408 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:13.408 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.096685+0000", 2024-09-11T06:51:13.408 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0051618569999999997 2024-09-11T06:51:13.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.408 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.408 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:13.408 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.096735+0000", 2024-09-11T06:51:13.408 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9517e-05 2024-09-11T06:51:13.408 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.408 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:13.408 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.408 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.408 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.5544.0:67 2.5 e82/69 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head v 82'77, mlcod=81'74)", 2024-09-11T06:51:13.409 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.595302+0000", 2024-09-11T06:51:13.409 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.719712120000001, 2024-09-11T06:51:13.409 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054511999999999998, 2024-09-11T06:51:13.409 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:13.409 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:13.409 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:13.409 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:13.409 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.409 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:13.409 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595302+0000", 2024-09-11T06:51:13.409 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.409 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.409 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.410 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:13.410 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595302+0000", 2024-09-11T06:51:13.410 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.410 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.410 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:13.410 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595303+0000", 2024-09-11T06:51:13.410 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2699999999999999e-07 2024-09-11T06:51:13.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.410 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.410 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:13.410 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595522+0000", 2024-09-11T06:51:13.410 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000218946 2024-09-11T06:51:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595544+0000", 2024-09-11T06:51:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2558999999999999e-05 2024-09-11T06:51:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595557+0000", 2024-09-11T06:51:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2653000000000001e-05 2024-09-11T06:51:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595662+0000", 2024-09-11T06:51:13.412 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000104862 2024-09-11T06:51:13.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.412 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:13.412 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595699+0000", 2024-09-11T06:51:13.412 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7376000000000003e-05 2024-09-11T06:51:13.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.412 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:13.412 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.600688+0000", 2024-09-11T06:51:13.412 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0049885279999999999 2024-09-11T06:51:13.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.413 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:13.413 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.600753+0000", 2024-09-11T06:51:13.413 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5549000000000005e-05 2024-09-11T06:51:13.413 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.413 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:13.413 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.413 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.5544.0:90 2.5 e82/69 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head v 82'86, mlcod=82'84)", 2024-09-11T06:51:13.413 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.725934+0000", 2024-09-11T06:51:13.413 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 24.589080359, 2024-09-11T06:51:13.413 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0044823220000000004, 2024-09-11T06:51:13.413 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:13.413 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:13.414 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:13.414 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:13.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.414 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:13.414 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.725934+0000", 2024-09-11T06:51:13.414 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.414 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:13.414 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.725934+0000", 2024-09-11T06:51:13.414 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:13.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.415 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:13.415 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.725936+0000", 2024-09-11T06:51:13.415 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9269999999999999e-06 2024-09-11T06:51:13.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.415 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.415 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:13.415 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.727188+0000", 2024-09-11T06:51:13.415 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012522779999999999 2024-09-11T06:51:13.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.415 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.415 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:13.415 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.727217+0000", 2024-09-11T06:51:13.415 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9373000000000002e-05 2024-09-11T06:51:13.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.416 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:13.416 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.727231+0000", 2024-09-11T06:51:13.416 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3499999999999999e-05 2024-09-11T06:51:13.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.416 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:13.416 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.727335+0000", 2024-09-11T06:51:13.416 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000104064 2024-09-11T06:51:13.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.416 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:13.416 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.727374+0000", 2024-09-11T06:51:13.417 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8967000000000001e-05 2024-09-11T06:51:13.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.417 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:13.417 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.730326+0000", 2024-09-11T06:51:13.417 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0029516519999999999 2024-09-11T06:51:13.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:13.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:13.417 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:13.417 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.730416+0000", 2024-09-11T06:51:13.417 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0561000000000002e-05 2024-09-11T06:51:13.417 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.417 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:13.418 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.418 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:13.418 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:13.418 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:13.418 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:51:13.420 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:51:13.423 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:13.421+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:13.521 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:13.517+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:13.552 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:13.552 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:13.552 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:51:13.552 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:13.567 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:51:13.571 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd journal inspect --pool rbd --image testrbdnbd21607 2024-09-11T06:51:13.572 INFO:tasks.workunit.client.0.smithi176.stderr:++ awk '/entries inspected/ {print $1}' 2024-09-11T06:51:13.579 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:51:13.579 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:51:13.595 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:51:13.595 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:51:13.611 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:51:13.611 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:51:13.621 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:13.618+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:13.631 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:51:13.632 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:51:13.653 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:51:13.653 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:51:13.659 INFO:tasks.workunit.client.0.smithi176.stderr:2024-09-11T06:51:13.654+0000 7fe826ffd640 -1 JournalMetadata: 0x55561ec33900 failed to locate client: INSPECT 2024-09-11T06:51:13.664 INFO:tasks.workunit.client.0.smithi176.stderr:+ queue_length=4 2024-09-11T06:51:13.664 INFO:tasks.workunit.client.0.smithi176.stderr:+ tries=2 2024-09-11T06:51:13.664 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 2 -lt 10 ']' 2024-09-11T06:51:13.664 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 4 -gt 0 ']' 2024-09-11T06:51:13.664 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd journal inspect --pool rbd --image testrbdnbd21607 --verbose 2024-09-11T06:51:13.673 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:51:13.673 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:51:13.677 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:13.677 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:13.692 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:51:13.722 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:13.717+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:13.736 INFO:tasks.workunit.client.0.smithi176.stdout:Entry: tag_id=1, commit_tid=1 2024-09-11T06:51:13.737 INFO:tasks.workunit.client.0.smithi176.stdout:{ 2024-09-11T06:51:13.737 INFO:tasks.workunit.client.0.smithi176.stdout: "event_type": "AioDiscard", 2024-09-11T06:51:13.737 INFO:tasks.workunit.client.0.smithi176.stdout: "offset": 0, 2024-09-11T06:51:13.737 INFO:tasks.workunit.client.0.smithi176.stdout: "length": 4096, 2024-09-11T06:51:13.737 INFO:tasks.workunit.client.0.smithi176.stdout: "discard_granularity_bytes": 4096, 2024-09-11T06:51:13.737 INFO:tasks.workunit.client.0.smithi176.stdout: "timestamp": "2024-09-11T06:51:11.164111+0000" 2024-09-11T06:51:13.737 INFO:tasks.workunit.client.0.smithi176.stdout:} 2024-09-11T06:51:13.737 INFO:tasks.workunit.client.0.smithi176.stdout:Entry: tag_id=1, commit_tid=2 2024-09-11T06:51:13.737 INFO:tasks.workunit.client.0.smithi176.stdout:{ 2024-09-11T06:51:13.737 INFO:tasks.workunit.client.0.smithi176.stdout: "event_type": "AioDiscard", 2024-09-11T06:51:13.737 INFO:tasks.workunit.client.0.smithi176.stdout: "offset": 0, 2024-09-11T06:51:13.737 INFO:tasks.workunit.client.0.smithi176.stdout: "length": 4096, 2024-09-11T06:51:13.738 INFO:tasks.workunit.client.0.smithi176.stdout: "discard_granularity_bytes": 4096, 2024-09-11T06:51:13.738 INFO:tasks.workunit.client.0.smithi176.stdout: "timestamp": "2024-09-11T06:51:11.182249+0000" 2024-09-11T06:51:13.738 INFO:tasks.workunit.client.0.smithi176.stdout:} 2024-09-11T06:51:13.738 INFO:tasks.workunit.client.0.smithi176.stdout:Entry: tag_id=1, commit_tid=3 2024-09-11T06:51:13.738 INFO:tasks.workunit.client.0.smithi176.stdout:{ 2024-09-11T06:51:13.738 INFO:tasks.workunit.client.0.smithi176.stdout: "event_type": "AioDiscard", 2024-09-11T06:51:13.738 INFO:tasks.workunit.client.0.smithi176.stdout: "offset": 4096, 2024-09-11T06:51:13.738 INFO:tasks.workunit.client.0.smithi176.stdout: "length": 4096, 2024-09-11T06:51:13.738 INFO:tasks.workunit.client.0.smithi176.stdout: "discard_granularity_bytes": 4096, 2024-09-11T06:51:13.738 INFO:tasks.workunit.client.0.smithi176.stdout: "timestamp": "2024-09-11T06:51:11.209791+0000" 2024-09-11T06:51:13.738 INFO:tasks.workunit.client.0.smithi176.stdout:} 2024-09-11T06:51:13.738 INFO:tasks.workunit.client.0.smithi176.stdout:Entry: tag_id=1, commit_tid=4 2024-09-11T06:51:13.739 INFO:tasks.workunit.client.0.smithi176.stdout:{ 2024-09-11T06:51:13.739 INFO:tasks.workunit.client.0.smithi176.stdout: "event_type": "AioDiscard", 2024-09-11T06:51:13.739 INFO:tasks.workunit.client.0.smithi176.stdout: "offset": 4096, 2024-09-11T06:51:13.739 INFO:tasks.workunit.client.0.smithi176.stdout: "length": 4096, 2024-09-11T06:51:13.739 INFO:tasks.workunit.client.0.smithi176.stdout: "discard_granularity_bytes": 4096, 2024-09-11T06:51:13.739 INFO:tasks.workunit.client.0.smithi176.stdout: "timestamp": "2024-09-11T06:51:11.226109+0000" 2024-09-11T06:51:13.740 INFO:tasks.workunit.client.0.smithi176.stdout:} 2024-09-11T06:51:13.740 INFO:tasks.workunit.client.0.smithi176.stdout:Summary: 2024-09-11T06:51:13.740 INFO:tasks.workunit.client.0.smithi176.stdout: 4 entries inspected, 0 errors 2024-09-11T06:51:13.742 INFO:tasks.workunit.client.0.smithi176.stderr:2024-09-11T06:51:13.738+0000 7f868affd640 -1 JournalMetadata: 0x55b506fc6c40 failed to locate client: INSPECT 2024-09-11T06:51:13.748 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 1 2024-09-11T06:51:13.823 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:13.817+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:13.849 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:13.849 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:13.862 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:51:13.924 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:13.918+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:13.934 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.1 to 1 (10000) 2024-09-11T06:51:13.949 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T06:51:13.954 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-11T06:51:13.977 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:13.977 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:13.992 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:51:14.033 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:14.030+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:14.126 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:14.122+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:14.147 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:14.147 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:14.160 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:51:14.245 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:14.242+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:14.309 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:14.309 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:14.321 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:51:14.365 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:14.362+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:14.430 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:14.425+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:14.455 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:14.455 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:14.469 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:51:14.531 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:14.526+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:14.632 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:14.626+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:14.638 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:14.638 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:14.652 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:51:14.732 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:14.726+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:14.750 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd journal inspect --pool rbd --image testrbdnbd21607 2024-09-11T06:51:14.750 INFO:tasks.workunit.client.0.smithi176.stderr:++ awk '/entries inspected/ {print $1}' 2024-09-11T06:51:14.770 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:14.770 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:14.784 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:51:14.833 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:14.829+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:14.835 INFO:tasks.workunit.client.0.smithi176.stderr:2024-09-11T06:51:14.830+0000 7fb7ca7fc640 -1 JournalMetadata: 0x55621835f590 failed to locate client: INSPECT 2024-09-11T06:51:14.842 INFO:tasks.workunit.client.0.smithi176.stderr:+ queue_length=4 2024-09-11T06:51:14.842 INFO:tasks.workunit.client.0.smithi176.stderr:+ tries=3 2024-09-11T06:51:14.842 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 3 -lt 10 ']' 2024-09-11T06:51:14.842 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 4 -gt 0 ']' 2024-09-11T06:51:14.842 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd journal inspect --pool rbd --image testrbdnbd21607 --verbose 2024-09-11T06:51:14.912 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:14.912 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:14.921 INFO:tasks.workunit.client.0.smithi176.stdout:Entry: tag_id=1, commit_tid=1 2024-09-11T06:51:14.921 INFO:tasks.workunit.client.0.smithi176.stdout:{ 2024-09-11T06:51:14.921 INFO:tasks.workunit.client.0.smithi176.stdout: "event_type": "AioDiscard", 2024-09-11T06:51:14.921 INFO:tasks.workunit.client.0.smithi176.stdout: "offset": 0, 2024-09-11T06:51:14.922 INFO:tasks.workunit.client.0.smithi176.stdout: "length": 4096, 2024-09-11T06:51:14.922 INFO:tasks.workunit.client.0.smithi176.stdout: "discard_granularity_bytes": 4096, 2024-09-11T06:51:14.922 INFO:tasks.workunit.client.0.smithi176.stdout: "timestamp": "2024-09-11T06:51:11.164111+0000" 2024-09-11T06:51:14.922 INFO:tasks.workunit.client.0.smithi176.stdout:} 2024-09-11T06:51:14.922 INFO:tasks.workunit.client.0.smithi176.stdout:Entry: tag_id=1, commit_tid=2 2024-09-11T06:51:14.922 INFO:tasks.workunit.client.0.smithi176.stdout:{ 2024-09-11T06:51:14.922 INFO:tasks.workunit.client.0.smithi176.stdout: "event_type": "AioDiscard", 2024-09-11T06:51:14.922 INFO:tasks.workunit.client.0.smithi176.stdout: "offset": 0, 2024-09-11T06:51:14.922 INFO:tasks.workunit.client.0.smithi176.stdout: "length": 4096, 2024-09-11T06:51:14.922 INFO:tasks.workunit.client.0.smithi176.stdout: "discard_granularity_bytes": 4096, 2024-09-11T06:51:14.922 INFO:tasks.workunit.client.0.smithi176.stdout: "timestamp": "2024-09-11T06:51:11.182249+0000" 2024-09-11T06:51:14.922 INFO:tasks.workunit.client.0.smithi176.stdout:} 2024-09-11T06:51:14.922 INFO:tasks.workunit.client.0.smithi176.stdout:Entry: tag_id=1, commit_tid=3 2024-09-11T06:51:14.922 INFO:tasks.workunit.client.0.smithi176.stdout:{ 2024-09-11T06:51:14.923 INFO:tasks.workunit.client.0.smithi176.stdout: "event_type": "AioDiscard", 2024-09-11T06:51:14.923 INFO:tasks.workunit.client.0.smithi176.stdout: "offset": 4096, 2024-09-11T06:51:14.923 INFO:tasks.workunit.client.0.smithi176.stdout: "length": 4096, 2024-09-11T06:51:14.923 INFO:tasks.workunit.client.0.smithi176.stdout: "discard_granularity_bytes": 4096, 2024-09-11T06:51:14.923 INFO:tasks.workunit.client.0.smithi176.stdout: "timestamp": "2024-09-11T06:51:11.209791+0000" 2024-09-11T06:51:14.923 INFO:tasks.workunit.client.0.smithi176.stdout:} 2024-09-11T06:51:14.923 INFO:tasks.workunit.client.0.smithi176.stdout:Entry: tag_id=1, commit_tid=4 2024-09-11T06:51:14.923 INFO:tasks.workunit.client.0.smithi176.stdout:{ 2024-09-11T06:51:14.923 INFO:tasks.workunit.client.0.smithi176.stdout: "event_type": "AioDiscard", 2024-09-11T06:51:14.924 INFO:tasks.workunit.client.0.smithi176.stdout: "offset": 4096, 2024-09-11T06:51:14.924 INFO:tasks.workunit.client.0.smithi176.stdout: "length": 4096, 2024-09-11T06:51:14.924 INFO:tasks.workunit.client.0.smithi176.stdout: "discard_granularity_bytes": 4096, 2024-09-11T06:51:14.924 INFO:tasks.workunit.client.0.smithi176.stdout: "timestamp": "2024-09-11T06:51:11.226109+0000" 2024-09-11T06:51:14.924 INFO:tasks.workunit.client.0.smithi176.stdout:} 2024-09-11T06:51:14.924 INFO:tasks.workunit.client.0.smithi176.stdout:Summary: 2024-09-11T06:51:14.924 INFO:tasks.workunit.client.0.smithi176.stdout: 4 entries inspected, 0 errors 2024-09-11T06:51:14.928 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:51:14.931 INFO:tasks.workunit.client.0.smithi176.stderr:2024-09-11T06:51:14.926+0000 7ff9da7fc640 -1 JournalMetadata: 0x561507b50350 failed to locate client: INSPECT 2024-09-11T06:51:14.935 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 1 2024-09-11T06:51:14.942 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.3 to 1 (10000) 2024-09-11T06:51:14.948 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is unset 2024-09-11T06:51:14.972 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-11T06:51:14.973 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:14.969+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:15.035 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:15.029+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:15.093 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:15.094 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:15.107 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:51:15.135 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:15.130+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:15.236 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:15.233+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:15.271 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:15.271 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:15.286 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:51:15.337 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:15.334+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:15.438 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:15.434+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:15.453 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:15.453 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:15.467 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:51:15.539 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:15.533+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:15.590 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:15.590 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:15.604 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:51:15.641 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:15.638+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:15.691 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:51:15.741 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:15.738+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:15.768 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:15.768 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:15.782 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:51:15.842 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:15.838+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:15.842 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:51:15.842 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:51:15.859 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:51:15.859 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:51:15.874 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:51:15.874 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:51:15.894 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:51:15.895 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:51:15.911 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:15.911 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:15.911 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:51:15.911 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:15.914 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:51:15.914 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:51:15.924 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:51:15.934 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:51:15.934 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:51:15.938 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd journal inspect --pool rbd --image testrbdnbd21607 2024-09-11T06:51:15.938 INFO:tasks.workunit.client.0.smithi176.stderr:++ awk '/entries inspected/ {print $1}' 2024-09-11T06:51:15.943 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:15.937+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:15.947 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.5 to 1 (10000) 2024-09-11T06:51:15.968 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-11T06:51:15.968 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:51:16.021 INFO:tasks.workunit.client.0.smithi176.stderr:2024-09-11T06:51:16.018+0000 7f95c5ffb640 -1 JournalMetadata: 0x55774fd9c150 failed to locate client: INSPECT 2024-09-11T06:51:16.029 INFO:tasks.workunit.client.0.smithi176.stderr:+ queue_length=4 2024-09-11T06:51:16.029 INFO:tasks.workunit.client.0.smithi176.stderr:+ tries=4 2024-09-11T06:51:16.029 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 4 -lt 10 ']' 2024-09-11T06:51:16.029 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 4 -gt 0 ']' 2024-09-11T06:51:16.029 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd journal inspect --pool rbd --image testrbdnbd21607 --verbose 2024-09-11T06:51:16.053 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:16.050+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:16.053 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:16.053 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:16.053 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:51:16.054 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:51:16.054 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:16.069 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:51:16.099 INFO:tasks.workunit.client.0.smithi176.stdout:Entry: tag_id=1, commit_tid=1 2024-09-11T06:51:16.099 INFO:tasks.workunit.client.0.smithi176.stdout:{ 2024-09-11T06:51:16.100 INFO:tasks.workunit.client.0.smithi176.stdout: "event_type": "AioDiscard", 2024-09-11T06:51:16.100 INFO:tasks.workunit.client.0.smithi176.stdout: "offset": 0, 2024-09-11T06:51:16.100 INFO:tasks.workunit.client.0.smithi176.stdout: "length": 4096, 2024-09-11T06:51:16.100 INFO:tasks.workunit.client.0.smithi176.stdout: "discard_granularity_bytes": 4096, 2024-09-11T06:51:16.100 INFO:tasks.workunit.client.0.smithi176.stdout: "timestamp": "2024-09-11T06:51:11.164111+0000" 2024-09-11T06:51:16.100 INFO:tasks.workunit.client.0.smithi176.stdout:} 2024-09-11T06:51:16.100 INFO:tasks.workunit.client.0.smithi176.stdout:Entry: tag_id=1, commit_tid=2 2024-09-11T06:51:16.100 INFO:tasks.workunit.client.0.smithi176.stdout:{ 2024-09-11T06:51:16.100 INFO:tasks.workunit.client.0.smithi176.stdout: "event_type": "AioDiscard", 2024-09-11T06:51:16.100 INFO:tasks.workunit.client.0.smithi176.stdout: "offset": 0, 2024-09-11T06:51:16.100 INFO:tasks.workunit.client.0.smithi176.stdout: "length": 4096, 2024-09-11T06:51:16.100 INFO:tasks.workunit.client.0.smithi176.stdout: "discard_granularity_bytes": 4096, 2024-09-11T06:51:16.100 INFO:tasks.workunit.client.0.smithi176.stdout: "timestamp": "2024-09-11T06:51:11.182249+0000" 2024-09-11T06:51:16.101 INFO:tasks.workunit.client.0.smithi176.stdout:} 2024-09-11T06:51:16.101 INFO:tasks.workunit.client.0.smithi176.stdout:Entry: tag_id=1, commit_tid=3 2024-09-11T06:51:16.101 INFO:tasks.workunit.client.0.smithi176.stdout:{ 2024-09-11T06:51:16.101 INFO:tasks.workunit.client.0.smithi176.stdout: "event_type": "AioDiscard", 2024-09-11T06:51:16.101 INFO:tasks.workunit.client.0.smithi176.stdout: "offset": 4096, 2024-09-11T06:51:16.101 INFO:tasks.workunit.client.0.smithi176.stdout: "length": 4096, 2024-09-11T06:51:16.101 INFO:tasks.workunit.client.0.smithi176.stdout: "discard_granularity_bytes": 4096, 2024-09-11T06:51:16.101 INFO:tasks.workunit.client.0.smithi176.stdout: "timestamp": "2024-09-11T06:51:11.209791+0000" 2024-09-11T06:51:16.101 INFO:tasks.workunit.client.0.smithi176.stdout:} 2024-09-11T06:51:16.101 INFO:tasks.workunit.client.0.smithi176.stdout:Entry: tag_id=1, commit_tid=4 2024-09-11T06:51:16.101 INFO:tasks.workunit.client.0.smithi176.stdout:{ 2024-09-11T06:51:16.101 INFO:tasks.workunit.client.0.smithi176.stdout: "event_type": "AioDiscard", 2024-09-11T06:51:16.101 INFO:tasks.workunit.client.0.smithi176.stdout: "offset": 4096, 2024-09-11T06:51:16.101 INFO:tasks.workunit.client.0.smithi176.stdout: "length": 4096, 2024-09-11T06:51:16.102 INFO:tasks.workunit.client.0.smithi176.stdout: "discard_granularity_bytes": 4096, 2024-09-11T06:51:16.102 INFO:tasks.workunit.client.0.smithi176.stdout: "timestamp": "2024-09-11T06:51:11.226109+0000" 2024-09-11T06:51:16.102 INFO:tasks.workunit.client.0.smithi176.stdout:} 2024-09-11T06:51:16.102 INFO:tasks.workunit.client.0.smithi176.stdout:Summary: 2024-09-11T06:51:16.102 INFO:tasks.workunit.client.0.smithi176.stdout: 4 entries inspected, 0 errors 2024-09-11T06:51:16.108 INFO:tasks.workunit.client.0.smithi176.stderr:2024-09-11T06:51:16.102+0000 7ff966ffd640 -1 JournalMetadata: 0x55e7674f4d60 failed to locate client: INSPECT 2024-09-11T06:51:16.112 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 1 2024-09-11T06:51:16.144 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:16.141+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:16.242 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:16.243 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:51:16.243 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:51:16.243 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:51:16.243 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.243 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:54 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'32, mlcod=13'29)", 2024-09-11T06:51:16.243 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:51:16.243 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 130.030230923, 2024-09-11T06:51:16.243 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034271670000000001, 2024-09-11T06:51:16.243 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.243 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.243 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.243 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.243 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.244 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.244 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:51:16.244 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.244 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.244 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.244 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.244 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210360+0000", 2024-09-11T06:51:16.244 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.244 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.244 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.244 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.244 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210361+0000", 2024-09-11T06:51:16.245 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1260000000000001e-06 2024-09-11T06:51:16.245 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.245 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.245 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.245 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210370+0000", 2024-09-11T06:51:16.245 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8200000000000003e-06 2024-09-11T06:51:16.245 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.245 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.245 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.245 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210390+0000", 2024-09-11T06:51:16.245 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0126e-05 2024-09-11T06:51:16.245 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.245 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.245 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.246 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210404+0000", 2024-09-11T06:51:16.246 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4589999999999999e-05 2024-09-11T06:51:16.246 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.246 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.246 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.246 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210538+0000", 2024-09-11T06:51:16.246 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000133719 2024-09-11T06:51:16.246 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.246 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.246 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.246 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.210593+0000", 2024-09-11T06:51:16.246 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5016999999999999e-05 2024-09-11T06:51:16.246 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.246 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.247 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.247 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213695+0000", 2024-09-11T06:51:16.247 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003101866 2024-09-11T06:51:16.247 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.247 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.247 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.247 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213787+0000", 2024-09-11T06:51:16.247 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1903000000000004e-05 2024-09-11T06:51:16.247 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.247 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.247 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.247 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.247 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.248 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:55 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'33, mlcod=13'29)", 2024-09-11T06:51:16.248 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:51:16.248 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 130.02921168500001, 2024-09-11T06:51:16.248 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.005086091, 2024-09-11T06:51:16.248 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.248 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.248 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.248 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.248 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.248 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.248 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:51:16.248 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.249 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.249 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.249 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.249 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:51:16.249 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.249 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.249 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.249 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.249 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211380+0000", 2024-09-11T06:51:16.249 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.097e-06 2024-09-11T06:51:16.249 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.250 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.250 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.250 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211537+0000", 2024-09-11T06:51:16.250 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000157369 2024-09-11T06:51:16.250 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.250 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.250 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.250 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211559+0000", 2024-09-11T06:51:16.250 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000001e-05 2024-09-11T06:51:16.250 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.250 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.250 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.250 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211573+0000", 2024-09-11T06:51:16.251 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4946000000000001e-05 2024-09-11T06:51:16.251 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.251 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.251 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.251 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211676+0000", 2024-09-11T06:51:16.251 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0001026 2024-09-11T06:51:16.251 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.251 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.251 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.251 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211709+0000", 2024-09-11T06:51:16.251 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3071999999999998e-05 2024-09-11T06:51:16.251 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.251 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.252 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.252 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216379+0000", 2024-09-11T06:51:16.252 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0046693780000000001 2024-09-11T06:51:16.252 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.252 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.252 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.252 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216465+0000", 2024-09-11T06:51:16.252 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6469000000000002e-05 2024-09-11T06:51:16.252 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.252 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.252 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.252 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.252 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.253 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:56 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'34, mlcod=13'29)", 2024-09-11T06:51:16.253 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:51:16.253 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 130.027777826, 2024-09-11T06:51:16.253 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0060310629999999997, 2024-09-11T06:51:16.253 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.253 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.253 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.253 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.253 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.253 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:51:16.253 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.254 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.254 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:51:16.254 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.254 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.254 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.254 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212814+0000", 2024-09-11T06:51:16.254 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0869999999999999e-06 2024-09-11T06:51:16.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.254 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.254 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.254 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212822+0000", 2024-09-11T06:51:16.254 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0870000000000007e-06 2024-09-11T06:51:16.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.255 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.255 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.255 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212840+0000", 2024-09-11T06:51:16.255 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:51:16.255 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.255 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.255 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.255 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212854+0000", 2024-09-11T06:51:16.255 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4003e-05 2024-09-11T06:51:16.255 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.255 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.255 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.255 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213026+0000", 2024-09-11T06:51:16.256 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017207499999999999 2024-09-11T06:51:16.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.256 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.256 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.256 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213063+0000", 2024-09-11T06:51:16.256 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7187000000000002e-05 2024-09-11T06:51:16.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.256 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.256 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.256 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218762+0000", 2024-09-11T06:51:16.256 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0056993290000000004 2024-09-11T06:51:16.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.256 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.256 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.256 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218844+0000", 2024-09-11T06:51:16.257 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1701999999999998e-05 2024-09-11T06:51:16.257 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.257 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.257 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.257 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.257 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:60 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'38, mlcod=13'36)", 2024-09-11T06:51:16.257 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:51:16.257 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 130.01383647700001, 2024-09-11T06:51:16.257 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039254270000000004, 2024-09-11T06:51:16.257 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.257 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.257 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.257 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.258 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.258 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:51:16.258 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.258 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.258 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:51:16.258 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.258 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.258 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226756+0000", 2024-09-11T06:51:16.258 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-11T06:51:16.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.259 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.259 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.259 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226768+0000", 2024-09-11T06:51:16.259 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1919999999999999e-05 2024-09-11T06:51:16.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.259 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.259 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.259 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226797+0000", 2024-09-11T06:51:16.259 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9573e-05 2024-09-11T06:51:16.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.259 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.259 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.260 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226818+0000", 2024-09-11T06:51:16.260 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0432999999999999e-05 2024-09-11T06:51:16.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.260 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.260 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.260 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226909+0000", 2024-09-11T06:51:16.260 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0920000000000004e-05 2024-09-11T06:51:16.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.260 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.260 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.260 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226936+0000", 2024-09-11T06:51:16.260 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7472999999999999e-05 2024-09-11T06:51:16.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.261 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.261 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.261 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230596+0000", 2024-09-11T06:51:16.261 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0036602520000000001 2024-09-11T06:51:16.261 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.261 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.261 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.261 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230680+0000", 2024-09-11T06:51:16.261 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3106e-05 2024-09-11T06:51:16.261 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.261 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.261 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.261 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.261 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.262 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:107 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'69, mlcod=13'64)", 2024-09-11T06:51:16.262 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:51:16.262 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 129.79738902, 2024-09-11T06:51:16.262 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040728830000000002, 2024-09-11T06:51:16.262 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.262 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.262 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.262 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.262 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.262 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.262 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:51:16.262 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.262 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.263 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.263 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:51:16.263 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.263 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.263 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.263 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.263 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443203+0000", 2024-09-11T06:51:16.263 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1480000000000001e-06 2024-09-11T06:51:16.263 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.263 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.263 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.263 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443222+0000", 2024-09-11T06:51:16.263 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9040000000000001e-05 2024-09-11T06:51:16.263 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.264 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.264 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443246+0000", 2024-09-11T06:51:16.264 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4606000000000001e-05 2024-09-11T06:51:16.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.264 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.264 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443277+0000", 2024-09-11T06:51:16.264 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0207999999999999e-05 2024-09-11T06:51:16.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.264 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.264 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443340+0000", 2024-09-11T06:51:16.264 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3602000000000005e-05 2024-09-11T06:51:16.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.265 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.265 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443363+0000", 2024-09-11T06:51:16.265 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2331e-05 2024-09-11T06:51:16.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.265 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.265 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447216+0000", 2024-09-11T06:51:16.265 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0038539690000000001 2024-09-11T06:51:16.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.265 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.265 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447274+0000", 2024-09-11T06:51:16.266 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7979000000000002e-05 2024-09-11T06:51:16.266 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.266 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.266 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.266 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.266 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:111 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'73, mlcod=13'67)", 2024-09-11T06:51:16.266 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:51:16.266 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 129.79087738999999, 2024-09-11T06:51:16.266 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034372360000000002, 2024-09-11T06:51:16.266 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.266 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.266 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.266 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.267 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.267 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.267 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:51:16.267 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.267 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.267 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.267 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449713+0000", 2024-09-11T06:51:16.267 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.267 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.267 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.267 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449714+0000", 2024-09-11T06:51:16.267 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5000000000000001e-07 2024-09-11T06:51:16.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.268 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.268 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449721+0000", 2024-09-11T06:51:16.268 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8530000000000003e-06 2024-09-11T06:51:16.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.268 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.268 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449736+0000", 2024-09-11T06:51:16.268 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5114e-05 2024-09-11T06:51:16.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.268 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.268 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449743+0000", 2024-09-11T06:51:16.269 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0210000000000002e-06 2024-09-11T06:51:16.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.269 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.269 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.269 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449804+0000", 2024-09-11T06:51:16.269 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0479999999999997e-05 2024-09-11T06:51:16.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.269 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.269 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.269 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.449826+0000", 2024-09-11T06:51:16.269 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1922000000000001e-05 2024-09-11T06:51:16.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.269 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.269 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.269 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453104+0000", 2024-09-11T06:51:16.270 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003278652 2024-09-11T06:51:16.270 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.270 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.270 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.270 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.453150+0000", 2024-09-11T06:51:16.270 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6244e-05 2024-09-11T06:51:16.270 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.270 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.270 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.270 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.270 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.270 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:114 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'76, mlcod=13'71)", 2024-09-11T06:51:16.270 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:51:16.270 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 129.78619931899999, 2024-09-11T06:51:16.271 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0042576660000000002, 2024-09-11T06:51:16.271 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.271 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.271 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.271 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.271 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.271 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:51:16.271 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.271 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.271 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:51:16.271 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.272 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.272 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.272 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.272 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454392+0000", 2024-09-11T06:51:16.272 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5999999999999991e-07 2024-09-11T06:51:16.272 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.272 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.272 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.273 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454399+0000", 2024-09-11T06:51:16.273 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.878e-06 2024-09-11T06:51:16.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.273 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.273 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454415+0000", 2024-09-11T06:51:16.273 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5659999999999999e-05 2024-09-11T06:51:16.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.273 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.273 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454422+0000", 2024-09-11T06:51:16.273 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:51:16.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.274 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.274 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.274 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454474+0000", 2024-09-11T06:51:16.274 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1860999999999997e-05 2024-09-11T06:51:16.274 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.274 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.274 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.274 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454497+0000", 2024-09-11T06:51:16.274 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3686e-05 2024-09-11T06:51:16.274 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.274 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.274 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.274 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458552+0000", 2024-09-11T06:51:16.274 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040544550000000002 2024-09-11T06:51:16.275 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.275 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.275 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.275 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458649+0000", 2024-09-11T06:51:16.275 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7005999999999999e-05 2024-09-11T06:51:16.275 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.275 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.275 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.275 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.275 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.275 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:115 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'77, mlcod=13'72)", 2024-09-11T06:51:16.275 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:51:16.275 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 129.78485870700001, 2024-09-11T06:51:16.275 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003355584, 2024-09-11T06:51:16.276 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.276 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.276 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.276 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.276 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.276 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.276 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:51:16.276 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.276 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.276 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.276 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:51:16.276 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.277 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.277 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455732+0000", 2024-09-11T06:51:16.277 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5999999999999999e-07 2024-09-11T06:51:16.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.277 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.277 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455803+0000", 2024-09-11T06:51:16.277 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0310999999999998e-05 2024-09-11T06:51:16.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.277 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.277 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455815+0000", 2024-09-11T06:51:16.277 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2787e-05 2024-09-11T06:51:16.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.278 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.278 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.278 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455823+0000", 2024-09-11T06:51:16.278 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5959999999999998e-06 2024-09-11T06:51:16.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.278 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.278 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.278 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455875+0000", 2024-09-11T06:51:16.278 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2054e-05 2024-09-11T06:51:16.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.278 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.278 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.279 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.455896+0000", 2024-09-11T06:51:16.279 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0554999999999999e-05 2024-09-11T06:51:16.279 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.279 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.279 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.279 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459000+0000", 2024-09-11T06:51:16.279 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003104842 2024-09-11T06:51:16.279 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.279 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.279 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.279 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.459087+0000", 2024-09-11T06:51:16.279 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6978999999999995e-05 2024-09-11T06:51:16.279 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.279 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.280 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.280 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.280 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:120 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'82, mlcod=13'78)", 2024-09-11T06:51:16.280 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:51:16.280 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 129.774181422, 2024-09-11T06:51:16.280 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0064612549999999999, 2024-09-11T06:51:16.280 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.280 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.280 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.280 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.280 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.280 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.281 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:51:16.281 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.281 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.281 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.281 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:51:16.281 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.281 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.281 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.281 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466410+0000", 2024-09-11T06:51:16.281 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0389999999999999e-06 2024-09-11T06:51:16.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.282 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.282 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466496+0000", 2024-09-11T06:51:16.282 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5519000000000006e-05 2024-09-11T06:51:16.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.282 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.282 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466510+0000", 2024-09-11T06:51:16.282 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3842999999999999e-05 2024-09-11T06:51:16.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.282 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.282 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466517+0000", 2024-09-11T06:51:16.283 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5229999999999998e-06 2024-09-11T06:51:16.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.283 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.283 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466595+0000", 2024-09-11T06:51:16.283 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8019e-05 2024-09-11T06:51:16.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.283 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.283 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466612+0000", 2024-09-11T06:51:16.283 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7013000000000001e-05 2024-09-11T06:51:16.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.283 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.284 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472787+0000", 2024-09-11T06:51:16.284 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0061750269999999996 2024-09-11T06:51:16.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.284 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.284 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.284 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472870+0000", 2024-09-11T06:51:16.284 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3271999999999998e-05 2024-09-11T06:51:16.284 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.284 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.284 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.284 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.284 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:121 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'83, mlcod=13'78)", 2024-09-11T06:51:16.284 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:51:16.285 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 129.772956626, 2024-09-11T06:51:16.285 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055613069999999997, 2024-09-11T06:51:16.285 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.285 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.285 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.285 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.285 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.285 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:51:16.285 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.285 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.286 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:51:16.286 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.286 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.286 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467635+0000", 2024-09-11T06:51:16.286 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6899999999999996e-07 2024-09-11T06:51:16.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.286 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.286 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467727+0000", 2024-09-11T06:51:16.286 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1800999999999997e-05 2024-09-11T06:51:16.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.287 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.287 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467740+0000", 2024-09-11T06:51:16.287 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3747999999999999e-05 2024-09-11T06:51:16.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.287 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.287 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.287 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467753+0000", 2024-09-11T06:51:16.287 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2807e-05 2024-09-11T06:51:16.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.287 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.287 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.287 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467812+0000", 2024-09-11T06:51:16.287 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8381999999999998e-05 2024-09-11T06:51:16.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.288 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.288 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.288 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467828+0000", 2024-09-11T06:51:16.288 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6728e-05 2024-09-11T06:51:16.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.288 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.288 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.288 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473112+0000", 2024-09-11T06:51:16.288 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052833619999999998 2024-09-11T06:51:16.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.288 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.288 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.289 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473195+0000", 2024-09-11T06:51:16.289 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3609999999999994e-05 2024-09-11T06:51:16.289 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.289 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.289 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.289 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.289 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.289 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:131 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'89, mlcod=13'85)", 2024-09-11T06:51:16.289 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:51:16.289 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 129.421550911, 2024-09-11T06:51:16.289 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052492260000000001, 2024-09-11T06:51:16.289 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.289 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.290 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.290 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.290 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.290 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.290 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:51:16.290 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.290 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.290 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.290 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.290 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:51:16.290 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.290 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.290 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.290 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.290 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819041+0000", 2024-09-11T06:51:16.291 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4360000000000001e-06 2024-09-11T06:51:16.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.291 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.291 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819100+0000", 2024-09-11T06:51:16.291 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9225999999999998e-05 2024-09-11T06:51:16.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.291 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.291 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819128+0000", 2024-09-11T06:51:16.291 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.779e-05 2024-09-11T06:51:16.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.292 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.292 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819142+0000", 2024-09-11T06:51:16.292 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3573e-05 2024-09-11T06:51:16.292 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.292 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.292 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.292 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819263+0000", 2024-09-11T06:51:16.292 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000121086 2024-09-11T06:51:16.292 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.292 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.292 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.292 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819292+0000", 2024-09-11T06:51:16.292 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9363e-05 2024-09-11T06:51:16.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.293 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.293 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.293 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824185+0000", 2024-09-11T06:51:16.293 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0048924620000000002 2024-09-11T06:51:16.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.293 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.293 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.293 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824289+0000", 2024-09-11T06:51:16.293 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010429000000000001 2024-09-11T06:51:16.293 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.293 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.293 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.294 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.294 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:132 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'90, mlcod=13'85)", 2024-09-11T06:51:16.294 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:51:16.294 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 129.42077622100001, 2024-09-11T06:51:16.294 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.004532342, 2024-09-11T06:51:16.294 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.294 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.294 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.294 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.294 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.294 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.294 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:51:16.294 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.294 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.295 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.295 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.295 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:51:16.295 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.295 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.295 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.295 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819816+0000", 2024-09-11T06:51:16.295 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.61e-06 2024-09-11T06:51:16.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.295 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.295 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.295 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819827+0000", 2024-09-11T06:51:16.295 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1e-05 2024-09-11T06:51:16.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.296 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.296 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.296 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819851+0000", 2024-09-11T06:51:16.296 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4267e-05 2024-09-11T06:51:16.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.296 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.296 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.296 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819864+0000", 2024-09-11T06:51:16.296 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2767e-05 2024-09-11T06:51:16.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.296 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.296 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.296 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820071+0000", 2024-09-11T06:51:16.297 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000207061 2024-09-11T06:51:16.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.297 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.297 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.297 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820103+0000", 2024-09-11T06:51:16.297 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2326999999999997e-05 2024-09-11T06:51:16.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.297 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.297 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.297 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824302+0000", 2024-09-11T06:51:16.297 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0041986000000000002 2024-09-11T06:51:16.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.297 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.297 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.298 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824347+0000", 2024-09-11T06:51:16.298 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4709999999999997e-05 2024-09-11T06:51:16.298 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.298 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.298 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.298 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.298 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.298 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:133 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'91, mlcod=13'86)", 2024-09-11T06:51:16.298 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:51:16.298 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 129.42014456199999, 2024-09-11T06:51:16.298 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039421459999999997, 2024-09-11T06:51:16.298 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.298 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.299 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.299 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.299 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.299 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.299 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:51:16.299 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.299 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.299 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.299 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.299 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:51:16.299 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.299 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.299 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.299 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.299 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820448+0000", 2024-09-11T06:51:16.300 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.84e-06 2024-09-11T06:51:16.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.300 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.300 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.300 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820463+0000", 2024-09-11T06:51:16.300 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5356e-05 2024-09-11T06:51:16.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.300 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.300 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.300 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820499+0000", 2024-09-11T06:51:16.300 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.625e-05 2024-09-11T06:51:16.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.300 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.300 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.301 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820546+0000", 2024-09-11T06:51:16.301 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7017000000000001e-05 2024-09-11T06:51:16.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.301 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.301 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.301 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820882+0000", 2024-09-11T06:51:16.301 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033512 2024-09-11T06:51:16.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.301 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.301 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.301 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820941+0000", 2024-09-11T06:51:16.301 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8922999999999998e-05 2024-09-11T06:51:16.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.302 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.302 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.302 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824351+0000", 2024-09-11T06:51:16.302 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034101940000000001 2024-09-11T06:51:16.302 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.302 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.302 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.302 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824388+0000", 2024-09-11T06:51:16.302 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7446e-05 2024-09-11T06:51:16.302 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.302 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.302 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.302 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.302 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.303 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:139 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'97, mlcod=13'86)", 2024-09-11T06:51:16.303 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:51:16.303 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 129.414734584, 2024-09-11T06:51:16.303 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003732221, 2024-09-11T06:51:16.303 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.303 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.303 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.303 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.303 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.303 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.303 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:51:16.303 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.303 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.304 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.304 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.304 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:51:16.304 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.304 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.304 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.304 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.304 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825858+0000", 2024-09-11T06:51:16.304 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5829999999999999e-06 2024-09-11T06:51:16.304 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.304 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.304 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.304 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825874+0000", 2024-09-11T06:51:16.304 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5967000000000001e-05 2024-09-11T06:51:16.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.305 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.305 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.305 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825909+0000", 2024-09-11T06:51:16.305 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5692999999999999e-05 2024-09-11T06:51:16.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.305 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.305 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.305 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825923+0000", 2024-09-11T06:51:16.305 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4080000000000001e-05 2024-09-11T06:51:16.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.305 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.305 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.305 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826010+0000", 2024-09-11T06:51:16.306 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6248999999999994e-05 2024-09-11T06:51:16.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.306 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.306 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826038+0000", 2024-09-11T06:51:16.306 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8010000000000001e-05 2024-09-11T06:51:16.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.306 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.306 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829499+0000", 2024-09-11T06:51:16.306 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003461783 2024-09-11T06:51:16.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.306 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.307 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829588+0000", 2024-09-11T06:51:16.307 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8856000000000004e-05 2024-09-11T06:51:16.307 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.307 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.307 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.307 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.307 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.307 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:140 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'98, mlcod=13'93)", 2024-09-11T06:51:16.307 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:51:16.307 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 129.41288385499999, 2024-09-11T06:51:16.307 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0057995929999999996, 2024-09-11T06:51:16.307 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.307 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.308 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.308 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.308 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.308 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.308 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:51:16.308 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.308 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.308 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.308 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.308 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:51:16.308 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.308 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.308 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.308 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.309 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827708+0000", 2024-09-11T06:51:16.309 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.547e-06 2024-09-11T06:51:16.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.309 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.309 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.309 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827765+0000", 2024-09-11T06:51:16.309 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6266e-05 2024-09-11T06:51:16.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.309 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.309 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.309 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827791+0000", 2024-09-11T06:51:16.309 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6098999999999999e-05 2024-09-11T06:51:16.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.310 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.310 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.310 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827806+0000", 2024-09-11T06:51:16.310 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5537e-05 2024-09-11T06:51:16.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.310 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.310 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.310 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827928+0000", 2024-09-11T06:51:16.310 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000122012 2024-09-11T06:51:16.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.310 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.310 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.310 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827961+0000", 2024-09-11T06:51:16.310 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2663999999999998e-05 2024-09-11T06:51:16.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.311 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.311 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.311 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833409+0000", 2024-09-11T06:51:16.311 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054479350000000001 2024-09-11T06:51:16.311 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.311 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.311 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.311 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833506+0000", 2024-09-11T06:51:16.311 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7533000000000003e-05 2024-09-11T06:51:16.311 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.311 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.311 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.311 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.312 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.312 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:141 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'99, mlcod=13'93)", 2024-09-11T06:51:16.312 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:51:16.312 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 129.41199047200001, 2024-09-11T06:51:16.312 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055394709999999998, 2024-09-11T06:51:16.312 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.312 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.312 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.312 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.312 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.312 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.312 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:51:16.312 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.312 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.313 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.313 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.313 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:51:16.313 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.313 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.313 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.313 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.313 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828602+0000", 2024-09-11T06:51:16.313 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.46e-06 2024-09-11T06:51:16.313 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.313 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.313 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.313 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828618+0000", 2024-09-11T06:51:16.314 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6592999999999999e-05 2024-09-11T06:51:16.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.314 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.314 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.314 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828646+0000", 2024-09-11T06:51:16.314 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7552999999999999e-05 2024-09-11T06:51:16.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.314 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.314 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.314 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828663+0000", 2024-09-11T06:51:16.314 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.719e-05 2024-09-11T06:51:16.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.314 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.314 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.315 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829023+0000", 2024-09-11T06:51:16.315 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036054799999999998 2024-09-11T06:51:16.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.315 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.315 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829062+0000", 2024-09-11T06:51:16.315 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8865999999999997e-05 2024-09-11T06:51:16.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.315 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.315 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834078+0000", 2024-09-11T06:51:16.315 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050157480000000004 2024-09-11T06:51:16.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.316 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.316 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834140+0000", 2024-09-11T06:51:16.316 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1513000000000005e-05 2024-09-11T06:51:16.316 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.316 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.316 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.316 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.316 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.316 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:145 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'103, mlcod=13'98)", 2024-09-11T06:51:16.316 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:51:16.316 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 129.40203797500001, 2024-09-11T06:51:16.316 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054455409999999999, 2024-09-11T06:51:16.316 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.317 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.317 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.317 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.317 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.317 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.317 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:51:16.317 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.317 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.317 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.317 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.317 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:51:16.317 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.317 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.317 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.318 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.318 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838554+0000", 2024-09-11T06:51:16.318 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3430000000000001e-06 2024-09-11T06:51:16.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.320 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.320 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838640+0000", 2024-09-11T06:51:16.320 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5569999999999996e-05 2024-09-11T06:51:16.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.321 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.321 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838660+0000", 2024-09-11T06:51:16.321 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0576000000000001e-05 2024-09-11T06:51:16.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.321 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.321 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838674+0000", 2024-09-11T06:51:16.321 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3529999999999999e-05 2024-09-11T06:51:16.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.321 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.321 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838838+0000", 2024-09-11T06:51:16.321 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016468799999999999 2024-09-11T06:51:16.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.322 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.322 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.322 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838885+0000", 2024-09-11T06:51:16.322 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6884e-05 2024-09-11T06:51:16.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.322 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.322 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.322 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843901+0000", 2024-09-11T06:51:16.322 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050162310000000003 2024-09-11T06:51:16.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.322 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.322 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.323 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843998+0000", 2024-09-11T06:51:16.323 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.6718999999999993e-05 2024-09-11T06:51:16.323 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.323 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.323 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.323 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.323 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4558.0:1 3.3 e56/55 3:c4c92e5a:::rbd_trash:head v 56'1, mlcod=0'0)", 2024-09-11T06:51:16.323 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:51:16.323 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 59.149234913000001, 2024-09-11T06:51:16.323 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0053791810000000002, 2024-09-11T06:51:16.323 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.323 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.323 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.324 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.324 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.324 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:51:16.324 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.324 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.324 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:51:16.324 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.324 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.324 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091357+0000", 2024-09-11T06:51:16.325 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.54e-07 2024-09-11T06:51:16.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.325 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.325 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091376+0000", 2024-09-11T06:51:16.325 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9769e-05 2024-09-11T06:51:16.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.325 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.325 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091396+0000", 2024-09-11T06:51:16.325 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0152999999999998e-05 2024-09-11T06:51:16.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.325 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.326 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091416+0000", 2024-09-11T06:51:16.326 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0006000000000001e-05 2024-09-11T06:51:16.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.326 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.326 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091506+0000", 2024-09-11T06:51:16.326 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9325000000000003e-05 2024-09-11T06:51:16.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.326 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.326 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091523+0000", 2024-09-11T06:51:16.326 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.77e-05 2024-09-11T06:51:16.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.327 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.327 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.096685+0000", 2024-09-11T06:51:16.327 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0051618569999999997 2024-09-11T06:51:16.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.327 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.327 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.327 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.096735+0000", 2024-09-11T06:51:16.327 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9517e-05 2024-09-11T06:51:16.327 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.327 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.327 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.327 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.327 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.5544.0:67 2.5 e82/69 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head v 82'77, mlcod=81'74)", 2024-09-11T06:51:16.328 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.595302+0000", 2024-09-11T06:51:16.328 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.645288496999999, 2024-09-11T06:51:16.328 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054511999999999998, 2024-09-11T06:51:16.328 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.328 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.328 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.328 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.328 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.328 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.328 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595302+0000", 2024-09-11T06:51:16.328 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.328 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.329 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.329 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595302+0000", 2024-09-11T06:51:16.329 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.329 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.329 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.329 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595303+0000", 2024-09-11T06:51:16.329 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2699999999999999e-07 2024-09-11T06:51:16.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.329 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.329 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.329 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595522+0000", 2024-09-11T06:51:16.329 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000218946 2024-09-11T06:51:16.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.330 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.330 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.330 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595544+0000", 2024-09-11T06:51:16.330 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2558999999999999e-05 2024-09-11T06:51:16.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.330 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.330 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.330 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595557+0000", 2024-09-11T06:51:16.330 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2653000000000001e-05 2024-09-11T06:51:16.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.330 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.330 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.330 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595662+0000", 2024-09-11T06:51:16.330 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000104862 2024-09-11T06:51:16.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.331 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.331 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.331 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595699+0000", 2024-09-11T06:51:16.331 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7376000000000003e-05 2024-09-11T06:51:16.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.331 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.331 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.331 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.600688+0000", 2024-09-11T06:51:16.331 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0049885279999999999 2024-09-11T06:51:16.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.331 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.331 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.331 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.600753+0000", 2024-09-11T06:51:16.332 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5549000000000005e-05 2024-09-11T06:51:16.332 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.332 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.332 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.332 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.332 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.332 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.5544.0:90 2.5 e82/69 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head v 82'86, mlcod=82'84)", 2024-09-11T06:51:16.332 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.725934+0000", 2024-09-11T06:51:16.332 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.514656735999999, 2024-09-11T06:51:16.332 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0044823220000000004, 2024-09-11T06:51:16.332 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.332 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.332 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.332 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.333 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.333 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.333 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.725934+0000", 2024-09-11T06:51:16.333 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.333 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.333 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.333 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.333 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.725934+0000", 2024-09-11T06:51:16.333 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.333 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.333 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.333 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.333 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.725936+0000", 2024-09-11T06:51:16.333 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9269999999999999e-06 2024-09-11T06:51:16.334 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.334 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.334 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.334 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.727188+0000", 2024-09-11T06:51:16.334 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012522779999999999 2024-09-11T06:51:16.334 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.334 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.334 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.334 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.727217+0000", 2024-09-11T06:51:16.334 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9373000000000002e-05 2024-09-11T06:51:16.334 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.334 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.334 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.334 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.727231+0000", 2024-09-11T06:51:16.335 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3499999999999999e-05 2024-09-11T06:51:16.335 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.335 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.335 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.335 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.727335+0000", 2024-09-11T06:51:16.335 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000104064 2024-09-11T06:51:16.335 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.335 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.335 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.335 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.727374+0000", 2024-09-11T06:51:16.335 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8967000000000001e-05 2024-09-11T06:51:16.335 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.335 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.335 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.335 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.730326+0000", 2024-09-11T06:51:16.336 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0029516519999999999 2024-09-11T06:51:16.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.336 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.336 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.336 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.730416+0000", 2024-09-11T06:51:16.336 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0561000000000002e-05 2024-09-11T06:51:16.336 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.336 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.336 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.336 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.336 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.336 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:16.338 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:51:16.339 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:51:16.339 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:51:16.340 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-09-11T06:51:15.948751+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":151655643,"num_objects":82,"num_object_clones":20,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":12,"num_read":5144,"num_read_kb":115731,"num_write":978,"num_write_kb":275318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":186911175,"num_keys_recovered":22,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":941,"ondisk_log_size":941,"up":34,"acting":34,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":469624,"kb_used_data":286632,"kb_used_omap":9,"kb_used_meta":182838,"kb_avail":565761416,"statfs":{"total":579820584960,"available":579339689984,"internally_reserved":0,"allocated":293511168,"data_stored":310280204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9565,"internal_metadata":187226787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":312,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":992,"num_read_kb":1725,"num_write":45,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.027673"},"pg_stats":[{"pgid":"1.0","version":"14'117","reported_seq":330,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-11T06:51:13.938455+0000","last_change":"2024-09-11T06:50:37.941212+0000","last_active":"2024-09-11T06:51:13.938455+0000","last_peered":"2024-09-11T06:51:13.938455+0000","last_clean":"2024-09-11T06:51:13.938455+0000","last_became_active":"2024-09-11T06:50:37.940448+0000","last_became_peered":"2024-09-11T06:50:37.940448+0000","last_unstale":"2024-09-11T06:51:13.938455+0000","last_undegraded":"2024-09-11T06:51:13.938455+0000","last_fullsized":"2024-09-11T06:51:13.938455+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_clean_scrub_stamp":"2024-09-11T06:49:05.088839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:48:59.833858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"95'73","reported_seq":192,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-11T06:51:13.744555+0000","last_change":"2024-09-11T06:50:47.401354+0000","last_active":"2024-09-11T06:51:13.744555+0000","last_peered":"2024-09-11T06:51:13.744555+0000","last_clean":"2024-09-11T06:51:13.744555+0000","last_became_active":"2024-09-11T06:50:47.306211+0000","last_became_peered":"2024-09-11T06:50:47.306211+0000","last_unstale":"2024-09-11T06:51:13.744555+0000","last_undegraded":"2024-09-11T06:51:13.744555+0000","last_fullsized":"2024-09-11T06:51:13.744555+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:06:28.664588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011212469999999999,"stat_sum":{"num_bytes":12582950,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":227,"num_read_kb":8351,"num_write":65,"num_write_kb":28692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":25165862,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"58'2","reported_seq":126,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-11T06:51:12.925546+0000","last_change":"2024-09-11T06:50:47.310511+0000","last_active":"2024-09-11T06:51:12.925546+0000","last_peered":"2024-09-11T06:51:12.925546+0000","last_clean":"2024-09-11T06:51:12.925546+0000","last_became_active":"2024-09-11T06:50:47.304328+0000","last_became_peered":"2024-09-11T06:50:47.304328+0000","last_unstale":"2024-09-11T06:51:12.925546+0000","last_undegraded":"2024-09-11T06:51:12.925546+0000","last_fullsized":"2024-09-11T06:51:12.925546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:21:15.806857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011764950000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"89'63","reported_seq":229,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-11T06:51:13.938458+0000","last_change":"2024-09-11T06:50:35.668519+0000","last_active":"2024-09-11T06:51:13.938458+0000","last_peered":"2024-09-11T06:51:13.938458+0000","last_clean":"2024-09-11T06:51:13.938458+0000","last_became_active":"2024-09-11T06:50:35.238279+0000","last_became_peered":"2024-09-11T06:50:35.238279+0000","last_unstale":"2024-09-11T06:51:13.938458+0000","last_undegraded":"2024-09-11T06:51:13.938458+0000","last_fullsized":"2024-09-11T06:51:13.938458+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:31:59.366688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077615599999999996,"stat_sum":{"num_bytes":20971520,"num_objects":9,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":150,"num_read_kb":23060,"num_write":59,"num_write_kb":45056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":41943040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":94,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-11T06:51:14.016299+0000","last_change":"2024-09-11T06:50:19.703255+0000","last_active":"2024-09-11T06:51:14.016299+0000","last_peered":"2024-09-11T06:51:14.016299+0000","last_clean":"2024-09-11T06:51:14.016299+0000","last_became_active":"2024-09-11T06:50:17.089296+0000","last_became_peered":"2024-09-11T06:50:17.089296+0000","last_unstale":"2024-09-11T06:51:14.016299+0000","last_undegraded":"2024-09-11T06:51:14.016299+0000","last_fullsized":"2024-09-11T06:51:14.016299+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:42:18.684413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001159373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"93'173","reported_seq":1653,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-11T06:51:13.936592+0000","last_change":"2024-09-11T06:49:55.948809+0000","last_active":"2024-09-11T06:51:13.936592+0000","last_peered":"2024-09-11T06:51:13.936592+0000","last_clean":"2024-09-11T06:51:13.936592+0000","last_became_active":"2024-09-11T06:49:55.948499+0000","last_became_peered":"2024-09-11T06:49:55.948499+0000","last_unstale":"2024-09-11T06:51:13.936592+0000","last_undegraded":"2024-09-11T06:51:13.936592+0000","last_fullsized":"2024-09-11T06:51:13.936592+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:02:59.359427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010163780000000001,"stat_sum":{"num_bytes":29360244,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":2879,"num_read_kb":25749,"num_write":89,"num_write_kb":41039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"79'1","reported_seq":94,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-11T06:51:12.925427+0000","last_change":"2024-09-11T06:50:19.097182+0000","last_active":"2024-09-11T06:51:12.925427+0000","last_peered":"2024-09-11T06:51:12.925427+0000","last_clean":"2024-09-11T06:51:12.925427+0000","last_became_active":"2024-09-11T06:50:17.097783+0000","last_became_peered":"2024-09-11T06:50:17.097783+0000","last_unstale":"2024-09-11T06:51:12.925427+0000","last_undegraded":"2024-09-11T06:51:12.925427+0000","last_fullsized":"2024-09-11T06:51:12.925427+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:35:49.244377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088028100000000001,"stat_sum":{"num_bytes":35,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"93'6","reported_seq":98,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-11T06:51:13.716821+0000","last_change":"2024-09-11T06:50:47.453549+0000","last_active":"2024-09-11T06:51:13.716821+0000","last_peered":"2024-09-11T06:51:13.716821+0000","last_clean":"2024-09-11T06:51:13.716821+0000","last_became_active":"2024-09-11T06:50:47.306692+0000","last_became_peered":"2024-09-11T06:50:47.306692+0000","last_unstale":"2024-09-11T06:51:13.716821+0000","last_undegraded":"2024-09-11T06:51:13.716821+0000","last_fullsized":"2024-09-11T06:51:13.716821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:18:21.306703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001910244,"stat_sum":{"num_bytes":8388743,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":54,"num_read_kb":53,"num_write":6,"num_write_kb":8196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":8388684,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"56'1","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-11T06:51:13.936662+0000","last_change":"2024-09-11T06:51:05.893192+0000","last_active":"2024-09-11T06:51:13.936662+0000","last_peered":"2024-09-11T06:51:13.936662+0000","last_clean":"2024-09-11T06:51:13.936662+0000","last_became_active":"2024-09-11T06:51:05.889184+0000","last_became_peered":"2024-09-11T06:51:05.889184+0000","last_unstale":"2024-09-11T06:51:13.936662+0000","last_undegraded":"2024-09-11T06:51:13.936662+0000","last_fullsized":"2024-09-11T06:51:13.936662+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:44:46.365158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-11T06:51:12.925756+0000","last_change":"2024-09-11T06:50:58.852200+0000","last_active":"2024-09-11T06:51:12.925756+0000","last_peered":"2024-09-11T06:51:12.925756+0000","last_clean":"2024-09-11T06:51:12.925756+0000","last_became_active":"2024-09-11T06:50:58.843758+0000","last_became_peered":"2024-09-11T06:50:58.843758+0000","last_unstale":"2024-09-11T06:51:12.925756+0000","last_undegraded":"2024-09-11T06:51:12.925756+0000","last_fullsized":"2024-09-11T06:51:12.925756+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:08.232430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"93'168","reported_seq":378,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-11T06:51:13.648420+0000","last_change":"2024-09-11T06:49:56.122331+0000","last_active":"2024-09-11T06:51:13.648420+0000","last_peered":"2024-09-11T06:51:13.648420+0000","last_clean":"2024-09-11T06:51:13.648420+0000","last_became_active":"2024-09-11T06:49:56.121998+0000","last_became_peered":"2024-09-11T06:49:56.121998+0000","last_unstale":"2024-09-11T06:51:13.648420+0000","last_undegraded":"2024-09-11T06:51:13.648420+0000","last_fullsized":"2024-09-11T06:51:13.648420+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:41:04.465351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583028,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":797,"num_read_kb":8824,"num_write":272,"num_write_kb":20700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"79'4","reported_seq":107,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-11T06:51:12.925610+0000","last_change":"2024-09-11T06:50:19.097504+0000","last_active":"2024-09-11T06:51:12.925610+0000","last_peered":"2024-09-11T06:51:12.925610+0000","last_clean":"2024-09-11T06:51:12.925610+0000","last_became_active":"2024-09-11T06:50:17.096243+0000","last_became_peered":"2024-09-11T06:50:17.096243+0000","last_unstale":"2024-09-11T06:51:12.925610+0000","last_undegraded":"2024-09-11T06:51:12.925610+0000","last_fullsized":"2024-09-11T06:51:12.925610+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:44:45.211005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085501399999999997,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":25,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"93'145","reported_seq":448,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-11T06:51:13.938473+0000","last_change":"2024-09-11T06:50:35.337013+0000","last_active":"2024-09-11T06:51:13.938473+0000","last_peered":"2024-09-11T06:51:13.938473+0000","last_clean":"2024-09-11T06:51:13.938473+0000","last_became_active":"2024-09-11T06:50:35.236333+0000","last_became_peered":"2024-09-11T06:50:35.236333+0000","last_unstale":"2024-09-11T06:51:13.938473+0000","last_undegraded":"2024-09-11T06:51:13.938473+0000","last_fullsized":"2024-09-11T06:51:13.938473+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:36:17.699073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095172900000000001,"stat_sum":{"num_bytes":29425734,"num_objects":13,"num_object_clones":5,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":477,"num_read_kb":19488,"num_write":140,"num_write_kb":45180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":41943104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":95,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-11T06:51:13.936522+0000","last_change":"2024-09-11T06:50:19.096344+0000","last_active":"2024-09-11T06:51:13.936522+0000","last_peered":"2024-09-11T06:51:13.936522+0000","last_clean":"2024-09-11T06:51:13.936522+0000","last_became_active":"2024-09-11T06:50:17.096560+0000","last_became_peered":"2024-09-11T06:50:17.096560+0000","last_unstale":"2024-09-11T06:51:13.936522+0000","last_undegraded":"2024-09-11T06:51:13.936522+0000","last_fullsized":"2024-09-11T06:51:13.936522+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:03:19.807744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053029399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"89'45","reported_seq":237,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-11T06:51:12.925752+0000","last_change":"2024-09-11T06:50:35.320796+0000","last_active":"2024-09-11T06:51:12.925752+0000","last_peered":"2024-09-11T06:51:12.925752+0000","last_clean":"2024-09-11T06:51:12.925752+0000","last_became_active":"2024-09-11T06:50:35.241589+0000","last_became_peered":"2024-09-11T06:50:35.241589+0000","last_unstale":"2024-09-11T06:51:12.925752+0000","last_undegraded":"2024-09-11T06:51:12.925752+0000","last_fullsized":"2024-09-11T06:51:12.925752+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:44:18.225009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388646,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":118,"num_read_kb":8286,"num_write":60,"num_write_kb":16420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":16777292,"num_keys_recovered":6,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":95,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-11T06:51:13.936794+0000","last_change":"2024-09-11T06:50:19.688101+0000","last_active":"2024-09-11T06:51:13.936794+0000","last_peered":"2024-09-11T06:51:13.936794+0000","last_clean":"2024-09-11T06:51:13.936794+0000","last_became_active":"2024-09-11T06:50:17.096959+0000","last_became_peered":"2024-09-11T06:50:17.096959+0000","last_unstale":"2024-09-11T06:51:13.936794+0000","last_undegraded":"2024-09-11T06:51:13.936794+0000","last_fullsized":"2024-09-11T06:51:13.936794+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:54:30.491263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059833200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"93'142","reported_seq":234,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-11T06:51:13.938494+0000","last_change":"2024-09-11T06:50:35.519269+0000","last_active":"2024-09-11T06:51:13.938494+0000","last_peered":"2024-09-11T06:51:13.938494+0000","last_clean":"2024-09-11T06:51:13.938494+0000","last_became_active":"2024-09-11T06:50:35.238011+0000","last_became_peered":"2024-09-11T06:50:35.238011+0000","last_unstale":"2024-09-11T06:51:13.938494+0000","last_undegraded":"2024-09-11T06:51:13.938494+0000","last_fullsized":"2024-09-11T06:51:13.938494+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:49:49.483725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001403348,"stat_sum":{"num_bytes":29364340,"num_objects":15,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":295,"num_read_kb":21795,"num_write":122,"num_write_kb":67613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":50331648,"num_keys_recovered":16,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":70,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":31,"num_read_kb":25,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":73,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":16,"acting":16,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":151065205,"num_objects":74,"num_object_clones":20,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":12,"num_read":4997,"num_read_kb":115606,"num_write":813,"num_write_kb":272896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":184549630,"num_keys_recovered":22,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":285614080,"data_stored":306529514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":815,"ondisk_log_size":815,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640284,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48624,"kb_used_data":21592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94323216,"statfs":{"total":96636764160,"available":96586973184,"internally_reserved":0,"allocated":22110208,"data_stored":21401686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640285,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58920,"kb_used_data":27728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94312920,"statfs":{"total":96636764160,"available":96576430080,"internally_reserved":0,"allocated":28393472,"data_stored":27753587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640285,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":114892,"kb_used_data":83696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94256948,"statfs":{"total":96636764160,"available":96519114752,"internally_reserved":0,"allocated":85704704,"data_stored":95498918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640286,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69228,"kb_used_data":38036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94302612,"statfs":{"total":96636764160,"available":96565874688,"internally_reserved":0,"allocated":38948864,"data_stored":38244435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640286,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":106120,"kb_used_data":74928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94265720,"statfs":{"total":96636764160,"available":96528097280,"internally_reserved":0,"allocated":76726272,"data_stored":75993457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640287,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71840,"kb_used_data":40652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94300000,"statfs":{"total":96636764160,"available":96563200000,"internally_reserved":0,"allocated":41627648,"data_stored":51388121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":39927808,"data_stored":50372794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":75595776,"data_stored":75563413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":37826560,"data_stored":37814426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":83988480,"data_stored":94478560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":27287552,"data_stored":27328628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20987904,"data_stored":20971693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":35,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:51:16.341 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:51:16.342 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:16.337+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:16.439 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:16.433+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:16.502 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:16.502 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:16.502 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:51:16.502 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:16.516 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:51:16.561 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:16.558+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:16.620 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:16.620 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:16.620 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:51:16.620 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:51:16.620 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:16.632 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:51:16.641 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:16.637+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:16.682 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:51:16.683 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:51:16.701 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-09-11T06:51:15.948751+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":151655643,"num_objects":82,"num_object_clones":20,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":12,"num_read":5144,"num_read_kb":115731,"num_write":978,"num_write_kb":275318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":186911175,"num_keys_recovered":22,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":941,"ondisk_log_size":941,"up":34,"acting":34,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":469624,"kb_used_data":286632,"kb_used_omap":9,"kb_used_meta":182838,"kb_avail":565761416,"statfs":{"total":579820584960,"available":579339689984,"internally_reserved":0,"allocated":293511168,"data_stored":310280204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9565,"internal_metadata":187226787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":312,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":992,"num_read_kb":1725,"num_write":45,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.027673"},"pg_stats":[{"pgid":"1.0","version":"14'117","reported_seq":330,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-11T06:51:13.938455+0000","last_change":"2024-09-11T06:50:37.941212+0000","last_active":"2024-09-11T06:51:13.938455+0000","last_peered":"2024-09-11T06:51:13.938455+0000","last_clean":"2024-09-11T06:51:13.938455+0000","last_became_active":"2024-09-11T06:50:37.940448+0000","last_became_peered":"2024-09-11T06:50:37.940448+0000","last_unstale":"2024-09-11T06:51:13.938455+0000","last_undegraded":"2024-09-11T06:51:13.938455+0000","last_fullsized":"2024-09-11T06:51:13.938455+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_clean_scrub_stamp":"2024-09-11T06:49:05.088839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:48:59.833858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"95'73","reported_seq":192,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-11T06:51:13.744555+0000","last_change":"2024-09-11T06:50:47.401354+0000","last_active":"2024-09-11T06:51:13.744555+0000","last_peered":"2024-09-11T06:51:13.744555+0000","last_clean":"2024-09-11T06:51:13.744555+0000","last_became_active":"2024-09-11T06:50:47.306211+0000","last_became_peered":"2024-09-11T06:50:47.306211+0000","last_unstale":"2024-09-11T06:51:13.744555+0000","last_undegraded":"2024-09-11T06:51:13.744555+0000","last_fullsized":"2024-09-11T06:51:13.744555+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:06:28.664588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011212469999999999,"stat_sum":{"num_bytes":12582950,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":227,"num_read_kb":8351,"num_write":65,"num_write_kb":28692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":25165862,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"58'2","reported_seq":126,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-11T06:51:12.925546+0000","last_change":"2024-09-11T06:50:47.310511+0000","last_active":"2024-09-11T06:51:12.925546+0000","last_peered":"2024-09-11T06:51:12.925546+0000","last_clean":"2024-09-11T06:51:12.925546+0000","last_became_active":"2024-09-11T06:50:47.304328+0000","last_became_peered":"2024-09-11T06:50:47.304328+0000","last_unstale":"2024-09-11T06:51:12.925546+0000","last_undegraded":"2024-09-11T06:51:12.925546+0000","last_fullsized":"2024-09-11T06:51:12.925546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:21:15.806857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011764950000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"89'63","reported_seq":229,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-11T06:51:13.938458+0000","last_change":"2024-09-11T06:50:35.668519+0000","last_active":"2024-09-11T06:51:13.938458+0000","last_peered":"2024-09-11T06:51:13.938458+0000","last_clean":"2024-09-11T06:51:13.938458+0000","last_became_active":"2024-09-11T06:50:35.238279+0000","last_became_peered":"2024-09-11T06:50:35.238279+0000","last_unstale":"2024-09-11T06:51:13.938458+0000","last_undegraded":"2024-09-11T06:51:13.938458+0000","last_fullsized":"2024-09-11T06:51:13.938458+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:31:59.366688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077615599999999996,"stat_sum":{"num_bytes":20971520,"num_objects":9,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":150,"num_read_kb":23060,"num_write":59,"num_write_kb":45056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":41943040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":94,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-11T06:51:14.016299+0000","last_change":"2024-09-11T06:50:19.703255+0000","last_active":"2024-09-11T06:51:14.016299+0000","last_peered":"2024-09-11T06:51:14.016299+0000","last_clean":"2024-09-11T06:51:14.016299+0000","last_became_active":"2024-09-11T06:50:17.089296+0000","last_became_peered":"2024-09-11T06:50:17.089296+0000","last_unstale":"2024-09-11T06:51:14.016299+0000","last_undegraded":"2024-09-11T06:51:14.016299+0000","last_fullsized":"2024-09-11T06:51:14.016299+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:42:18.684413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001159373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"93'173","reported_seq":1653,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-11T06:51:13.936592+0000","last_change":"2024-09-11T06:49:55.948809+0000","last_active":"2024-09-11T06:51:13.936592+0000","last_peered":"2024-09-11T06:51:13.936592+0000","last_clean":"2024-09-11T06:51:13.936592+0000","last_became_active":"2024-09-11T06:49:55.948499+0000","last_became_peered":"2024-09-11T06:49:55.948499+0000","last_unstale":"2024-09-11T06:51:13.936592+0000","last_undegraded":"2024-09-11T06:51:13.936592+0000","last_fullsized":"2024-09-11T06:51:13.936592+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:02:59.359427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010163780000000001,"stat_sum":{"num_bytes":29360244,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":2879,"num_read_kb":25749,"num_write":89,"num_write_kb":41039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"79'1","reported_seq":94,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-11T06:51:12.925427+0000","last_change":"2024-09-11T06:50:19.097182+0000","last_active":"2024-09-11T06:51:12.925427+0000","last_peered":"2024-09-11T06:51:12.925427+0000","last_clean":"2024-09-11T06:51:12.925427+0000","last_became_active":"2024-09-11T06:50:17.097783+0000","last_became_peered":"2024-09-11T06:50:17.097783+0000","last_unstale":"2024-09-11T06:51:12.925427+0000","last_undegraded":"2024-09-11T06:51:12.925427+0000","last_fullsized":"2024-09-11T06:51:12.925427+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:35:49.244377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088028100000000001,"stat_sum":{"num_bytes":35,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"93'6","reported_seq":98,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-11T06:51:13.716821+0000","last_change":"2024-09-11T06:50:47.453549+0000","last_active":"2024-09-11T06:51:13.716821+0000","last_peered":"2024-09-11T06:51:13.716821+0000","last_clean":"2024-09-11T06:51:13.716821+0000","last_became_active":"2024-09-11T06:50:47.306692+0000","last_became_peered":"2024-09-11T06:50:47.306692+0000","last_unstale":"2024-09-11T06:51:13.716821+0000","last_undegraded":"2024-09-11T06:51:13.716821+0000","last_fullsized":"2024-09-11T06:51:13.716821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:18:21.306703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001910244,"stat_sum":{"num_bytes":8388743,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":54,"num_read_kb":53,"num_write":6,"num_write_kb":8196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":8388684,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"56'1","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-11T06:51:13.936662+0000","last_change":"2024-09-11T06:51:05.893192+0000","last_active":"2024-09-11T06:51:13.936662+0000","last_peered":"2024-09-11T06:51:13.936662+0000","last_clean":"2024-09-11T06:51:13.936662+0000","last_became_active":"2024-09-11T06:51:05.889184+0000","last_became_peered":"2024-09-11T06:51:05.889184+0000","last_unstale":"2024-09-11T06:51:13.936662+0000","last_undegraded":"2024-09-11T06:51:13.936662+0000","last_fullsized":"2024-09-11T06:51:13.936662+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:44:46.365158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-11T06:51:12.925756+0000","last_change":"2024-09-11T06:50:58.852200+0000","last_active":"2024-09-11T06:51:12.925756+0000","last_peered":"2024-09-11T06:51:12.925756+0000","last_clean":"2024-09-11T06:51:12.925756+0000","last_became_active":"2024-09-11T06:50:58.843758+0000","last_became_peered":"2024-09-11T06:50:58.843758+0000","last_unstale":"2024-09-11T06:51:12.925756+0000","last_undegraded":"2024-09-11T06:51:12.925756+0000","last_fullsized":"2024-09-11T06:51:12.925756+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:08.232430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"93'168","reported_seq":378,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-11T06:51:13.648420+0000","last_change":"2024-09-11T06:49:56.122331+0000","last_active":"2024-09-11T06:51:13.648420+0000","last_peered":"2024-09-11T06:51:13.648420+0000","last_clean":"2024-09-11T06:51:13.648420+0000","last_became_active":"2024-09-11T06:49:56.121998+0000","last_became_peered":"2024-09-11T06:49:56.121998+0000","last_unstale":"2024-09-11T06:51:13.648420+0000","last_undegraded":"2024-09-11T06:51:13.648420+0000","last_fullsized":"2024-09-11T06:51:13.648420+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:41:04.465351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583028,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":797,"num_read_kb":8824,"num_write":272,"num_write_kb":20700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"79'4","reported_seq":107,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-11T06:51:12.925610+0000","last_change":"2024-09-11T06:50:19.097504+0000","last_active":"2024-09-11T06:51:12.925610+0000","last_peered":"2024-09-11T06:51:12.925610+0000","last_clean":"2024-09-11T06:51:12.925610+0000","last_became_active":"2024-09-11T06:50:17.096243+0000","last_became_peered":"2024-09-11T06:50:17.096243+0000","last_unstale":"2024-09-11T06:51:12.925610+0000","last_undegraded":"2024-09-11T06:51:12.925610+0000","last_fullsized":"2024-09-11T06:51:12.925610+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:44:45.211005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085501399999999997,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":25,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"93'145","reported_seq":448,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-11T06:51:13.938473+0000","last_change":"2024-09-11T06:50:35.337013+0000","last_active":"2024-09-11T06:51:13.938473+0000","last_peered":"2024-09-11T06:51:13.938473+0000","last_clean":"2024-09-11T06:51:13.938473+0000","last_became_active":"2024-09-11T06:50:35.236333+0000","last_became_peered":"2024-09-11T06:50:35.236333+0000","last_unstale":"2024-09-11T06:51:13.938473+0000","last_undegraded":"2024-09-11T06:51:13.938473+0000","last_fullsized":"2024-09-11T06:51:13.938473+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:36:17.699073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095172900000000001,"stat_sum":{"num_bytes":29425734,"num_objects":13,"num_object_clones":5,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":477,"num_read_kb":19488,"num_write":140,"num_write_kb":45180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":41943104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":95,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-11T06:51:13.936522+0000","last_change":"2024-09-11T06:50:19.096344+0000","last_active":"2024-09-11T06:51:13.936522+0000","last_peered":"2024-09-11T06:51:13.936522+0000","last_clean":"2024-09-11T06:51:13.936522+0000","last_became_active":"2024-09-11T06:50:17.096560+0000","last_became_peered":"2024-09-11T06:50:17.096560+0000","last_unstale":"2024-09-11T06:51:13.936522+0000","last_undegraded":"2024-09-11T06:51:13.936522+0000","last_fullsized":"2024-09-11T06:51:13.936522+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:03:19.807744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053029399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"89'45","reported_seq":237,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-11T06:51:12.925752+0000","last_change":"2024-09-11T06:50:35.320796+0000","last_active":"2024-09-11T06:51:12.925752+0000","last_peered":"2024-09-11T06:51:12.925752+0000","last_clean":"2024-09-11T06:51:12.925752+0000","last_became_active":"2024-09-11T06:50:35.241589+0000","last_became_peered":"2024-09-11T06:50:35.241589+0000","last_unstale":"2024-09-11T06:51:12.925752+0000","last_undegraded":"2024-09-11T06:51:12.925752+0000","last_fullsized":"2024-09-11T06:51:12.925752+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:44:18.225009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388646,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":118,"num_read_kb":8286,"num_write":60,"num_write_kb":16420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":16777292,"num_keys_recovered":6,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":95,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-11T06:51:13.936794+0000","last_change":"2024-09-11T06:50:19.688101+0000","last_active":"2024-09-11T06:51:13.936794+0000","last_peered":"2024-09-11T06:51:13.936794+0000","last_clean":"2024-09-11T06:51:13.936794+0000","last_became_active":"2024-09-11T06:50:17.096959+0000","last_became_peered":"2024-09-11T06:50:17.096959+0000","last_unstale":"2024-09-11T06:51:13.936794+0000","last_undegraded":"2024-09-11T06:51:13.936794+0000","last_fullsized":"2024-09-11T06:51:13.936794+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:54:30.491263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059833200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"93'142","reported_seq":234,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-11T06:51:13.938494+0000","last_change":"2024-09-11T06:50:35.519269+0000","last_active":"2024-09-11T06:51:13.938494+0000","last_peered":"2024-09-11T06:51:13.938494+0000","last_clean":"2024-09-11T06:51:13.938494+0000","last_became_active":"2024-09-11T06:50:35.238011+0000","last_became_peered":"2024-09-11T06:50:35.238011+0000","last_unstale":"2024-09-11T06:51:13.938494+0000","last_undegraded":"2024-09-11T06:51:13.938494+0000","last_fullsized":"2024-09-11T06:51:13.938494+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:49:49.483725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001403348,"stat_sum":{"num_bytes":29364340,"num_objects":15,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":295,"num_read_kb":21795,"num_write":122,"num_write_kb":67613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":50331648,"num_keys_recovered":16,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":70,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":31,"num_read_kb":25,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":73,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":16,"acting":16,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":151065205,"num_objects":74,"num_object_clones":20,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":12,"num_read":4997,"num_read_kb":115606,"num_write":813,"num_write_kb":272896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":184549630,"num_keys_recovered":22,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":285614080,"data_stored":306529514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":815,"ondisk_log_size":815,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640284,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48624,"kb_used_data":21592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94323216,"statfs":{"total":96636764160,"available":96586973184,"internally_reserved":0,"allocated":22110208,"data_stored":21401686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640285,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58920,"kb_used_data":27728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94312920,"statfs":{"total":96636764160,"available":96576430080,"internally_reserved":0,"allocated":28393472,"data_stored":27753587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640285,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":114892,"kb_used_data":83696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94256948,"statfs":{"total":96636764160,"available":96519114752,"internally_reserved":0,"allocated":85704704,"data_stored":95498918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640286,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69228,"kb_used_data":38036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94302612,"statfs":{"total":96636764160,"available":96565874688,"internally_reserved":0,"allocated":38948864,"data_stored":38244435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640286,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":106120,"kb_used_data":74928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94265720,"statfs":{"total":96636764160,"available":96528097280,"internally_reserved":0,"allocated":76726272,"data_stored":75993457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640287,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71840,"kb_used_data":40652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94300000,"statfs":{"total":96636764160,"available":96563200000,"internally_reserved":0,"allocated":41627648,"data_stored":51388121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":39927808,"data_stored":50372794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":75595776,"data_stored":75563413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":37826560,"data_stored":37814426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":83988480,"data_stored":94478560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":27287552,"data_stored":27328628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20987904,"data_stored":20971693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":35,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:51:16.702 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-11T06:51:16.702 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T06:51:16.702 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [0, 2, 4, 1, 3, 5] 2024-09-11T06:51:16.702 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-11T06:51:16.745 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:16.742+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:16.794 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:16.794 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:51:16.794 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:51:16.794 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:51:16.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.794 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4558.0:5 3.2 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-11T06:51:16.795 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:51:16.795 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 58.703491796000002, 2024-09-11T06:51:16.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0062353959999999998, 2024-09-11T06:51:16.795 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.795 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.795 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.795 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:16.795 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4558", 2024-09-11T06:51:16.795 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1478388898", 2024-09-11T06:51:16.795 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 5 2024-09-11T06:51:16.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.795 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:51:16.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:51:16.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088422+0000", 2024-09-11T06:51:16.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7859999999999998e-06 2024-09-11T06:51:16.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088440+0000", 2024-09-11T06:51:16.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8185999999999998e-05 2024-09-11T06:51:16.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088479+0000", 2024-09-11T06:51:16.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8248999999999999e-05 2024-09-11T06:51:16.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088497+0000", 2024-09-11T06:51:16.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8889000000000002e-05 2024-09-11T06:51:16.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088658+0000", 2024-09-11T06:51:16.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000160271 2024-09-11T06:51:16.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op must wait for map", 2024-09-11T06:51:16.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088668+0000", 2024-09-11T06:51:16.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0409e-05 2024-09-11T06:51:16.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.799 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.799 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092347+0000", 2024-09-11T06:51:16.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00367915 2024-09-11T06:51:16.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.799 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.799 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092541+0000", 2024-09-11T06:51:16.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000193833 2024-09-11T06:51:16.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.800 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-11T06:51:16.800 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092732+0000", 2024-09-11T06:51:16.800 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000190772 2024-09-11T06:51:16.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.800 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.800 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:16.800 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094136+0000", 2024-09-11T06:51:16.800 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014040809999999999 2024-09-11T06:51:16.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.800 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.800 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:16.800 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094506+0000", 2024-09-11T06:51:16.800 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000369959 2024-09-11T06:51:16.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.801 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.801 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094590+0000", 2024-09-11T06:51:16.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4033e-05 2024-09-11T06:51:16.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.801 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.801 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094654+0000", 2024-09-11T06:51:16.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3777999999999998e-05 2024-09-11T06:51:16.801 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.801 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.801 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.802 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:587 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-11T06:51:16.802 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:51:16.802 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 54.562659140999997, 2024-09-11T06:51:16.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0023436289999999999, 2024-09-11T06:51:16.802 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.802 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.802 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.802 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:16.802 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:51:16.802 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:51:16.802 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 587 2024-09-11T06:51:16.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.803 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.803 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.803 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:51:16.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.803 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.803 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:51:16.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.803 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.803 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229252+0000", 2024-09-11T06:51:16.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3799999999999999e-06 2024-09-11T06:51:16.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.804 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.804 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229259+0000", 2024-09-11T06:51:16.804 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4760000000000003e-06 2024-09-11T06:51:16.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.804 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.804 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229271+0000", 2024-09-11T06:51:16.804 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2313000000000001e-05 2024-09-11T06:51:16.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.804 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.804 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229279+0000", 2024-09-11T06:51:16.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1610000000000003e-06 2024-09-11T06:51:16.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.805 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.805 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229344+0000", 2024-09-11T06:51:16.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4375000000000006e-05 2024-09-11T06:51:16.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.805 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.805 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229404+0000", 2024-09-11T06:51:16.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0517999999999999e-05 2024-09-11T06:51:16.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:16.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229516+0000", 2024-09-11T06:51:16.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011129900000000001 2024-09-11T06:51:16.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:16.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231502+0000", 2024-09-11T06:51:16.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001986238 2024-09-11T06:51:16.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:16.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231537+0000", 2024-09-11T06:51:16.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4968000000000001e-05 2024-09-11T06:51:16.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.807 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.807 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231569+0000", 2024-09-11T06:51:16.807 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2459000000000002e-05 2024-09-11T06:51:16.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.807 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.807 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231595+0000", 2024-09-11T06:51:16.807 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5442000000000001e-05 2024-09-11T06:51:16.807 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.807 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.807 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.808 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:15 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:16.808 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.709418+0000", 2024-09-11T06:51:16.808 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 46.082491988000001, 2024-09-11T06:51:16.808 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027504280000000001, 2024-09-11T06:51:16.808 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.808 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.808 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.808 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:16.808 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:16.808 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:16.808 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 15 2024-09-11T06:51:16.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.808 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.808 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.809 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.809 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709418+0000", 2024-09-11T06:51:16.809 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.809 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.809 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709418+0000", 2024-09-11T06:51:16.809 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.809 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.809 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709420+0000", 2024-09-11T06:51:16.809 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2299999999999998e-06 2024-09-11T06:51:16.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709432+0000", 2024-09-11T06:51:16.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1897e-05 2024-09-11T06:51:16.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709464+0000", 2024-09-11T06:51:16.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1819000000000001e-05 2024-09-11T06:51:16.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709479+0000", 2024-09-11T06:51:16.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4853e-05 2024-09-11T06:51:16.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.811 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.811 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709624+0000", 2024-09-11T06:51:16.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144912 2024-09-11T06:51:16.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.811 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.811 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709773+0000", 2024-09-11T06:51:16.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000149529 2024-09-11T06:51:16.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.811 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:16.811 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.710055+0000", 2024-09-11T06:51:16.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000281692 2024-09-11T06:51:16.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.812 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:16.812 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.711412+0000", 2024-09-11T06:51:16.812 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001356931 2024-09-11T06:51:16.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.812 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:16.812 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.711890+0000", 2024-09-11T06:51:16.812 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00047768700000000002 2024-09-11T06:51:16.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.812 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.813 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.712050+0000", 2024-09-11T06:51:16.813 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016041499999999999 2024-09-11T06:51:16.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.813 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.813 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.712169+0000", 2024-09-11T06:51:16.813 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011846299999999999 2024-09-11T06:51:16.813 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.813 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.813 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.813 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:18 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:16.813 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.713502+0000", 2024-09-11T06:51:16.813 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 46.078408562, 2024-09-11T06:51:16.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032323069999999998, 2024-09-11T06:51:16.814 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.814 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.814 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.814 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:16.814 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:16.814 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:16.814 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18 2024-09-11T06:51:16.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.814 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.814 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.814 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713502+0000", 2024-09-11T06:51:16.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713502+0000", 2024-09-11T06:51:16.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713503+0000", 2024-09-11T06:51:16.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8199999999999999e-06 2024-09-11T06:51:16.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713513+0000", 2024-09-11T06:51:16.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8539999999999992e-06 2024-09-11T06:51:16.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.816 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.816 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713540+0000", 2024-09-11T06:51:16.816 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6594999999999999e-05 2024-09-11T06:51:16.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.816 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.816 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713553+0000", 2024-09-11T06:51:16.816 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2843999999999999e-05 2024-09-11T06:51:16.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.816 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.817 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713686+0000", 2024-09-11T06:51:16.817 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013294899999999999 2024-09-11T06:51:16.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.817 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.817 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.817 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713937+0000", 2024-09-11T06:51:16.817 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025110600000000002 2024-09-11T06:51:16.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.817 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.817 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:16.817 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.714412+0000", 2024-09-11T06:51:16.817 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00047562 2024-09-11T06:51:16.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.817 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.818 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:16.818 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.715867+0000", 2024-09-11T06:51:16.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001455004 2024-09-11T06:51:16.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.818 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:16.818 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.716569+0000", 2024-09-11T06:51:16.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070159400000000005 2024-09-11T06:51:16.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.818 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.818 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.716660+0000", 2024-09-11T06:51:16.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1403000000000006e-05 2024-09-11T06:51:16.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.716734+0000", 2024-09-11T06:51:16.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3517999999999996e-05 2024-09-11T06:51:16.819 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.819 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.819 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.819 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:23 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:16.819 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.723392+0000", 2024-09-11T06:51:16.819 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 46.068517966000002, 2024-09-11T06:51:16.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027098489999999999, 2024-09-11T06:51:16.819 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.820 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.820 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.820 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:16.820 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:16.820 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:16.820 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23 2024-09-11T06:51:16.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.820 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723392+0000", 2024-09-11T06:51:16.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.821 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.821 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723392+0000", 2024-09-11T06:51:16.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.821 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.821 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723394+0000", 2024-09-11T06:51:16.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0669999999999999e-06 2024-09-11T06:51:16.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.821 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.821 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723406+0000", 2024-09-11T06:51:16.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1367e-05 2024-09-11T06:51:16.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.822 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.822 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.822 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723436+0000", 2024-09-11T06:51:16.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0874999999999998e-05 2024-09-11T06:51:16.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.822 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.822 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.822 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723451+0000", 2024-09-11T06:51:16.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4192999999999999e-05 2024-09-11T06:51:16.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.823 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.823 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723577+0000", 2024-09-11T06:51:16.823 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000126267 2024-09-11T06:51:16.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.823 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.823 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723718+0000", 2024-09-11T06:51:16.823 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000141206 2024-09-11T06:51:16.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.823 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:16.823 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723982+0000", 2024-09-11T06:51:16.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026338499999999999 2024-09-11T06:51:16.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.824 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:16.824 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.725377+0000", 2024-09-11T06:51:16.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013956999999999999 2024-09-11T06:51:16.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.824 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:16.824 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.725908+0000", 2024-09-11T06:51:16.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00053036200000000004 2024-09-11T06:51:16.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.725999+0000", 2024-09-11T06:51:16.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1016999999999999e-05 2024-09-11T06:51:16.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726102+0000", 2024-09-11T06:51:16.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010341 2024-09-11T06:51:16.825 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.825 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.825 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.826 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:15 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:16.826 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.868921+0000", 2024-09-11T06:51:16.826 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.922988924999999, 2024-09-11T06:51:16.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0026879, 2024-09-11T06:51:16.826 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.826 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.826 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.826 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:16.826 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:16.826 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:16.826 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 15 2024-09-11T06:51:16.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.826 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.827 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.827 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.827 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868921+0000", 2024-09-11T06:51:16.827 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.827 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.827 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.827 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.827 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868921+0000", 2024-09-11T06:51:16.827 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.827 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.827 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.827 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.827 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868923+0000", 2024-09-11T06:51:16.827 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8959999999999999e-06 2024-09-11T06:51:16.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868934+0000", 2024-09-11T06:51:16.828 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1083e-05 2024-09-11T06:51:16.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868965+0000", 2024-09-11T06:51:16.828 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0833000000000001e-05 2024-09-11T06:51:16.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868990+0000", 2024-09-11T06:51:16.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4545999999999998e-05 2024-09-11T06:51:16.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.869124+0000", 2024-09-11T06:51:16.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013489599999999999 2024-09-11T06:51:16.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.869270+0000", 2024-09-11T06:51:16.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000145976 2024-09-11T06:51:16.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:16.830 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.869538+0000", 2024-09-11T06:51:16.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026780599999999999 2024-09-11T06:51:16.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.830 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:16.830 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.870691+0000", 2024-09-11T06:51:16.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00115277 2024-09-11T06:51:16.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.830 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:16.830 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.871456+0000", 2024-09-11T06:51:16.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00076473800000000005 2024-09-11T06:51:16.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.871543+0000", 2024-09-11T06:51:16.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7040999999999996e-05 2024-09-11T06:51:16.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.871609+0000", 2024-09-11T06:51:16.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6315000000000005e-05 2024-09-11T06:51:16.831 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.831 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.831 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.831 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:20 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:16.832 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.877429+0000", 2024-09-11T06:51:16.832 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.914480636, 2024-09-11T06:51:16.832 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032382629999999999, 2024-09-11T06:51:16.832 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.832 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.832 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.832 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:16.832 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:16.832 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:16.832 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 20 2024-09-11T06:51:16.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.832 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877429+0000", 2024-09-11T06:51:16.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.833 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877429+0000", 2024-09-11T06:51:16.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.833 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877431+0000", 2024-09-11T06:51:16.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9429999999999999e-06 2024-09-11T06:51:16.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.834 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.834 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877444+0000", 2024-09-11T06:51:16.834 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2396e-05 2024-09-11T06:51:16.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.834 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.834 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877474+0000", 2024-09-11T06:51:16.834 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0239999999999998e-05 2024-09-11T06:51:16.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.834 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.834 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877496+0000", 2024-09-11T06:51:16.834 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1914999999999999e-05 2024-09-11T06:51:16.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877616+0000", 2024-09-11T06:51:16.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000119814 2024-09-11T06:51:16.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877836+0000", 2024-09-11T06:51:16.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022029600000000001 2024-09-11T06:51:16.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:16.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.878482+0000", 2024-09-11T06:51:16.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00064605200000000004 2024-09-11T06:51:16.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:16.836 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.879804+0000", 2024-09-11T06:51:16.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013216460000000001 2024-09-11T06:51:16.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:16.836 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.880519+0000", 2024-09-11T06:51:16.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00071547600000000005 2024-09-11T06:51:16.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.836 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.880606+0000", 2024-09-11T06:51:16.837 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6907e-05 2024-09-11T06:51:16.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.837 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.837 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.837 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.880668+0000", 2024-09-11T06:51:16.837 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1577999999999999e-05 2024-09-11T06:51:16.837 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.837 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.837 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.837 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.837 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:23 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:16.837 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.885067+0000", 2024-09-11T06:51:16.837 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.906842982000001, 2024-09-11T06:51:16.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028127320000000001, 2024-09-11T06:51:16.838 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.838 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.838 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.838 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:16.838 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:16.838 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:16.838 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23 2024-09-11T06:51:16.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.838 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885067+0000", 2024-09-11T06:51:16.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.839 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.839 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885067+0000", 2024-09-11T06:51:16.839 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.839 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.839 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885069+0000", 2024-09-11T06:51:16.839 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2000000000000001e-06 2024-09-11T06:51:16.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.839 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.839 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885081+0000", 2024-09-11T06:51:16.839 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1209999999999999e-05 2024-09-11T06:51:16.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.840 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.840 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885111+0000", 2024-09-11T06:51:16.840 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0318999999999999e-05 2024-09-11T06:51:16.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.840 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.840 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885132+0000", 2024-09-11T06:51:16.840 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0649000000000002e-05 2024-09-11T06:51:16.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885243+0000", 2024-09-11T06:51:16.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000111284 2024-09-11T06:51:16.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885387+0000", 2024-09-11T06:51:16.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144222 2024-09-11T06:51:16.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:16.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885699+0000", 2024-09-11T06:51:16.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00031239800000000003 2024-09-11T06:51:16.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.842 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:16.842 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.886817+0000", 2024-09-11T06:51:16.842 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001117191 2024-09-11T06:51:16.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.842 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:16.842 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.887684+0000", 2024-09-11T06:51:16.842 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00086699499999999996 2024-09-11T06:51:16.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.842 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.843 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.887772+0000", 2024-09-11T06:51:16.843 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8224000000000001e-05 2024-09-11T06:51:16.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.843 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.843 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.843 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.887880+0000", 2024-09-11T06:51:16.843 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010804 2024-09-11T06:51:16.843 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.843 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.843 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.843 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.843 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "MOSDPGPull(2.0 e70/69 cost 1048576)", 2024-09-11T06:51:16.843 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:35.528435+0000", 2024-09-11T06:51:16.844 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.263475245999999, 2024-09-11T06:51:16.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027390520000000001, 2024-09-11T06:51:16.844 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.844 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.844 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-11T06:51:16.844 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.844 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.844 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528435+0000", 2024-09-11T06:51:16.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.844 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.844 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528435+0000", 2024-09-11T06:51:16.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.845 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528437+0000", 2024-09-11T06:51:16.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5959999999999998e-06 2024-09-11T06:51:16.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.845 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528465+0000", 2024-09-11T06:51:16.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7436e-05 2024-09-11T06:51:16.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.845 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528499+0000", 2024-09-11T06:51:16.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3599000000000001e-05 2024-09-11T06:51:16.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528515+0000", 2024-09-11T06:51:16.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.677e-05 2024-09-11T06:51:16.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528652+0000", 2024-09-11T06:51:16.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000136546 2024-09-11T06:51:16.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.531174+0000", 2024-09-11T06:51:16.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0025221050000000002 2024-09-11T06:51:16.847 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.847 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.847 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.847 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4828.0:15 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-11T06:51:16.847 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:36.253730+0000", 2024-09-11T06:51:16.847 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 40.538179903, 2024-09-11T06:51:16.847 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0025257589999999998, 2024-09-11T06:51:16.847 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.847 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.847 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.847 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:16.847 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4828", 2024-09-11T06:51:16.847 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2410288650", 2024-09-11T06:51:16.848 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 15 2024-09-11T06:51:16.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.848 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.848 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.848 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253730+0000", 2024-09-11T06:51:16.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.848 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.848 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253730+0000", 2024-09-11T06:51:16.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.849 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253732+0000", 2024-09-11T06:51:16.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2400000000000002e-06 2024-09-11T06:51:16.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.849 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253744+0000", 2024-09-11T06:51:16.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1316e-05 2024-09-11T06:51:16.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.849 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253771+0000", 2024-09-11T06:51:16.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.724e-05 2024-09-11T06:51:16.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.850 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.850 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253785+0000", 2024-09-11T06:51:16.850 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3786e-05 2024-09-11T06:51:16.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.850 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.850 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253886+0000", 2024-09-11T06:51:16.850 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000100981 2024-09-11T06:51:16.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.850 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.850 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.254029+0000", 2024-09-11T06:51:16.850 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014311900000000001 2024-09-11T06:51:16.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:16.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.254317+0000", 2024-09-11T06:51:16.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00028809300000000001 2024-09-11T06:51:16.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:16.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.255403+0000", 2024-09-11T06:51:16.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001085639 2024-09-11T06:51:16.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:16.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.256107+0000", 2024-09-11T06:51:16.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070468799999999997 2024-09-11T06:51:16.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.852 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.852 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.852 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.256191+0000", 2024-09-11T06:51:16.852 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3714999999999994e-05 2024-09-11T06:51:16.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.852 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.852 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.852 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.256256+0000", 2024-09-11T06:51:16.852 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4942000000000003e-05 2024-09-11T06:51:16.852 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.852 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.852 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.853 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.853 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5313.0:26 2.6 2:7b712441:ns::rbd_object_map.11adc335555f:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:51:16.853 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.260230+0000", 2024-09-11T06:51:16.853 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.531680495000003, 2024-09-11T06:51:16.853 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002457201, 2024-09-11T06:51:16.853 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.853 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.853 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.853 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:16.853 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5313", 2024-09-11T06:51:16.853 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/624314229", 2024-09-11T06:51:16.853 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:51:16.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.854 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.854 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.854 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260230+0000", 2024-09-11T06:51:16.854 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.854 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.854 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260230+0000", 2024-09-11T06:51:16.854 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.854 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.854 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260232+0000", 2024-09-11T06:51:16.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.956e-06 2024-09-11T06:51:16.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.855 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.855 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260243+0000", 2024-09-11T06:51:16.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1687e-05 2024-09-11T06:51:16.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.855 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.855 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260269+0000", 2024-09-11T06:51:16.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6211999999999999e-05 2024-09-11T06:51:16.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.855 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.856 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260284+0000", 2024-09-11T06:51:16.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4053000000000001e-05 2024-09-11T06:51:16.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.856 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.856 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260391+0000", 2024-09-11T06:51:16.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000107278 2024-09-11T06:51:16.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.856 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.856 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260541+0000", 2024-09-11T06:51:16.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015033000000000001 2024-09-11T06:51:16.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.857 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 3", 2024-09-11T06:51:16.857 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260951+0000", 2024-09-11T06:51:16.857 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040981299999999997 2024-09-11T06:51:16.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.857 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:16.857 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262399+0000", 2024-09-11T06:51:16.857 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014475930000000001 2024-09-11T06:51:16.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.857 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:16.857 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262531+0000", 2024-09-11T06:51:16.857 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013239 2024-09-11T06:51:16.858 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.858 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.858 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262623+0000", 2024-09-11T06:51:16.858 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2361000000000006e-05 2024-09-11T06:51:16.858 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.858 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.858 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262687+0000", 2024-09-11T06:51:16.858 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3528000000000006e-05 2024-09-11T06:51:16.858 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.858 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.858 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.858 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.859 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.859 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5313.0:31 2.6 2:7b712441:ns::rbd_object_map.11adc335555f:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:51:16.859 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.266495+0000", 2024-09-11T06:51:16.859 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.525414720999997, 2024-09-11T06:51:16.859 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0026763870000000001, 2024-09-11T06:51:16.859 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.859 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.859 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.859 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:16.859 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5313", 2024-09-11T06:51:16.859 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/624314229", 2024-09-11T06:51:16.859 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 31 2024-09-11T06:51:16.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.859 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.860 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.860 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266495+0000", 2024-09-11T06:51:16.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.860 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.860 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266495+0000", 2024-09-11T06:51:16.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.860 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.860 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266498+0000", 2024-09-11T06:51:16.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2369999999999999e-06 2024-09-11T06:51:16.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.861 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.861 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266509+0000", 2024-09-11T06:51:16.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1459e-05 2024-09-11T06:51:16.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.861 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.861 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266540+0000", 2024-09-11T06:51:16.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1189999999999998e-05 2024-09-11T06:51:16.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.861 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.862 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266563+0000", 2024-09-11T06:51:16.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2529000000000001e-05 2024-09-11T06:51:16.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.862 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.862 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266708+0000", 2024-09-11T06:51:16.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014511 2024-09-11T06:51:16.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.862 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.862 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266962+0000", 2024-09-11T06:51:16.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000253897 2024-09-11T06:51:16.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 3", 2024-09-11T06:51:16.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.267430+0000", 2024-09-11T06:51:16.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00046799899999999998 2024-09-11T06:51:16.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:16.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.268765+0000", 2024-09-11T06:51:16.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013350359999999999 2024-09-11T06:51:16.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:16.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.269019+0000", 2024-09-11T06:51:16.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025397000000000001 2024-09-11T06:51:16.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.864 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.864 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.269105+0000", 2024-09-11T06:51:16.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5952000000000006e-05 2024-09-11T06:51:16.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.864 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.864 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.269172+0000", 2024-09-11T06:51:16.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7008000000000006e-05 2024-09-11T06:51:16.864 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.864 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.864 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.865 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:5 3.4 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:16.865 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.588582+0000", 2024-09-11T06:51:16.865 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.203328568, 2024-09-11T06:51:16.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002889857, 2024-09-11T06:51:16.865 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.865 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.865 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.865 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:16.865 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:51:16.865 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:51:16.865 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 5 2024-09-11T06:51:16.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.865 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.866 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.866 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588582+0000", 2024-09-11T06:51:16.866 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.866 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.866 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588582+0000", 2024-09-11T06:51:16.866 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.866 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.866 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588583+0000", 2024-09-11T06:51:16.866 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.223e-06 2024-09-11T06:51:16.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.867 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.867 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588588+0000", 2024-09-11T06:51:16.867 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4299999999999997e-06 2024-09-11T06:51:16.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.867 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.867 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588606+0000", 2024-09-11T06:51:16.867 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7652999999999999e-05 2024-09-11T06:51:16.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.868 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.868 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588615+0000", 2024-09-11T06:51:16.868 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4399999999999994e-06 2024-09-11T06:51:16.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.868 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.868 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588725+0000", 2024-09-11T06:51:16.868 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010945799999999999 2024-09-11T06:51:16.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.868 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.869 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.589031+0000", 2024-09-11T06:51:16.869 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030590000000000001 2024-09-11T06:51:16.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.869 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:16.869 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.589521+0000", 2024-09-11T06:51:16.869 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00049061299999999999 2024-09-11T06:51:16.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.869 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:16.869 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.590792+0000", 2024-09-11T06:51:16.869 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012708039999999999 2024-09-11T06:51:16.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.870 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:16.870 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591339+0000", 2024-09-11T06:51:16.870 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00054710499999999999 2024-09-11T06:51:16.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.870 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.870 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.870 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591412+0000", 2024-09-11T06:51:16.870 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2811999999999993e-05 2024-09-11T06:51:16.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.870 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.870 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.870 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591471+0000", 2024-09-11T06:51:16.870 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9419000000000001e-05 2024-09-11T06:51:16.871 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.871 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.871 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.871 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.871 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.871 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:8 3.4 3:3f0de396:::rbd_header.1415bf940188:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:16.871 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.596120+0000", 2024-09-11T06:51:16.871 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.195789684000001, 2024-09-11T06:51:16.871 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027911020000000002, 2024-09-11T06:51:16.871 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.871 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.871 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.871 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:16.871 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:51:16.871 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:51:16.872 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 8 2024-09-11T06:51:16.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.872 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.872 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.872 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596120+0000", 2024-09-11T06:51:16.872 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.872 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.872 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596120+0000", 2024-09-11T06:51:16.872 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.873 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.873 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.873 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596122+0000", 2024-09-11T06:51:16.873 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.513e-06 2024-09-11T06:51:16.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.873 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.873 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.873 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596130+0000", 2024-09-11T06:51:16.873 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4999999999999999e-06 2024-09-11T06:51:16.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.874 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.874 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596152+0000", 2024-09-11T06:51:16.874 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1846000000000001e-05 2024-09-11T06:51:16.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.874 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.874 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596163+0000", 2024-09-11T06:51:16.874 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1029999999999999e-05 2024-09-11T06:51:16.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.874 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.875 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596242+0000", 2024-09-11T06:51:16.875 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8275000000000005e-05 2024-09-11T06:51:16.875 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.875 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.875 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.875 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596439+0000", 2024-09-11T06:51:16.875 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000196999 2024-09-11T06:51:16.875 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.875 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.875 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:16.875 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596755+0000", 2024-09-11T06:51:16.875 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00031626100000000003 2024-09-11T06:51:16.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.876 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:16.876 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.598043+0000", 2024-09-11T06:51:16.876 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001288389 2024-09-11T06:51:16.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.876 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:16.876 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.598774+0000", 2024-09-11T06:51:16.876 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00073080499999999995 2024-09-11T06:51:16.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.876 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.876 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.598855+0000", 2024-09-11T06:51:16.877 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0615e-05 2024-09-11T06:51:16.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.877 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.877 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.598912+0000", 2024-09-11T06:51:16.877 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6869e-05 2024-09-11T06:51:16.877 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.877 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.877 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.877 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:9 3.0 3:00834abc:::rbd_object_map.1415bf940188:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:16.877 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.599161+0000", 2024-09-11T06:51:16.878 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.192748667, 2024-09-11T06:51:16.878 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002826051, 2024-09-11T06:51:16.878 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.878 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.878 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.878 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:16.878 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:51:16.878 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:51:16.878 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 9 2024-09-11T06:51:16.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.878 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.878 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.878 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.878 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599161+0000", 2024-09-11T06:51:16.879 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.879 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.879 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.879 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599161+0000", 2024-09-11T06:51:16.879 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.879 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.879 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.879 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599163+0000", 2024-09-11T06:51:16.879 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8700000000000001e-06 2024-09-11T06:51:16.879 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.880 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.880 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.880 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599174+0000", 2024-09-11T06:51:16.880 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0416e-05 2024-09-11T06:51:16.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.880 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.880 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.880 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599202+0000", 2024-09-11T06:51:16.880 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7946e-05 2024-09-11T06:51:16.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.880 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.880 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.880 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599216+0000", 2024-09-11T06:51:16.881 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3934000000000001e-05 2024-09-11T06:51:16.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.881 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.881 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599345+0000", 2024-09-11T06:51:16.881 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012954000000000001 2024-09-11T06:51:16.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.881 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.881 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599540+0000", 2024-09-11T06:51:16.881 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019516299999999999 2024-09-11T06:51:16.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.881 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:16.882 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599796+0000", 2024-09-11T06:51:16.882 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025556500000000002 2024-09-11T06:51:16.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.882 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:16.882 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.601071+0000", 2024-09-11T06:51:16.882 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001275396 2024-09-11T06:51:16.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.882 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:16.882 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.601852+0000", 2024-09-11T06:51:16.882 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000780524 2024-09-11T06:51:16.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.883 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.883 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.601925+0000", 2024-09-11T06:51:16.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3574999999999999e-05 2024-09-11T06:51:16.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.883 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.883 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.601988+0000", 2024-09-11T06:51:16.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2121999999999999e-05 2024-09-11T06:51:16.883 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.883 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.883 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.884 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5038.0:12 3.4 3:3f0de396:::rbd_header.1415bf940188:head [watch watch cookie 140693532068064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:16.884 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.718024+0000", 2024-09-11T06:51:16.884 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.073885729000001, 2024-09-11T06:51:16.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002724583, 2024-09-11T06:51:16.884 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.884 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.884 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.884 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:16.884 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5038", 2024-09-11T06:51:16.884 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/945190275", 2024-09-11T06:51:16.884 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:51:16.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.885 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718024+0000", 2024-09-11T06:51:16.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718024+0000", 2024-09-11T06:51:16.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718027+0000", 2024-09-11T06:51:16.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1430000000000001e-06 2024-09-11T06:51:16.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718037+0000", 2024-09-11T06:51:16.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0253e-05 2024-09-11T06:51:16.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718068+0000", 2024-09-11T06:51:16.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1050000000000003e-05 2024-09-11T06:51:16.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.887 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.887 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718086+0000", 2024-09-11T06:51:16.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8629000000000001e-05 2024-09-11T06:51:16.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.887 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.887 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718206+0000", 2024-09-11T06:51:16.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000119358 2024-09-11T06:51:16.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.887 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.887 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718425+0000", 2024-09-11T06:51:16.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000218975 2024-09-11T06:51:16.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.888 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:16.888 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718830+0000", 2024-09-11T06:51:16.888 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040538199999999998 2024-09-11T06:51:16.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.888 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:16.888 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.719849+0000", 2024-09-11T06:51:16.888 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010187589999999999 2024-09-11T06:51:16.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.888 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:16.888 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.720548+0000", 2024-09-11T06:51:16.888 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000699112 2024-09-11T06:51:16.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.889 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.889 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.720632+0000", 2024-09-11T06:51:16.889 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4314999999999995e-05 2024-09-11T06:51:16.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.889 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.889 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.720749+0000", 2024-09-11T06:51:16.889 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116607 2024-09-11T06:51:16.889 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.889 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.889 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.890 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:47 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:16.890 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.553228+0000", 2024-09-11T06:51:16.890 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 28.238681638999999, 2024-09-11T06:51:16.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040683100000000003, 2024-09-11T06:51:16.890 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.890 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.890 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.890 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:16.890 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:16.890 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:16.890 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 47 2024-09-11T06:51:16.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.890 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553228+0000", 2024-09-11T06:51:16.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553228+0000", 2024-09-11T06:51:16.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553231+0000", 2024-09-11T06:51:16.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3130000000000001e-06 2024-09-11T06:51:16.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553242+0000", 2024-09-11T06:51:16.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1345999999999999e-05 2024-09-11T06:51:16.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553272+0000", 2024-09-11T06:51:16.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0179999999999999e-05 2024-09-11T06:51:16.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.893 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.893 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553287+0000", 2024-09-11T06:51:16.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4283e-05 2024-09-11T06:51:16.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.893 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.893 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553415+0000", 2024-09-11T06:51:16.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000128491 2024-09-11T06:51:16.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.893 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.894 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553651+0000", 2024-09-11T06:51:16.894 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000236328 2024-09-11T06:51:16.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.894 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.894 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:16.894 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.554356+0000", 2024-09-11T06:51:16.894 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070436299999999995 2024-09-11T06:51:16.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.894 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.894 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:16.894 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.556064+0000", 2024-09-11T06:51:16.894 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0017084439999999999 2024-09-11T06:51:16.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:16.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.556993+0000", 2024-09-11T06:51:16.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00092912400000000003 2024-09-11T06:51:16.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.557173+0000", 2024-09-11T06:51:16.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017940400000000001 2024-09-11T06:51:16.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.557297+0000", 2024-09-11T06:51:16.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000124034 2024-09-11T06:51:16.895 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.896 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.896 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.896 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:66 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:16.896 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.589465+0000", 2024-09-11T06:51:16.896 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 28.202444834000001, 2024-09-11T06:51:16.896 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039855489999999997, 2024-09-11T06:51:16.896 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.896 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.896 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.896 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:16.896 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:16.896 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:16.897 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 66 2024-09-11T06:51:16.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.897 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589465+0000", 2024-09-11T06:51:16.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589465+0000", 2024-09-11T06:51:16.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.898 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.898 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589468+0000", 2024-09-11T06:51:16.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3539999999999998e-06 2024-09-11T06:51:16.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.898 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.898 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589480+0000", 2024-09-11T06:51:16.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1946e-05 2024-09-11T06:51:16.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.898 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.898 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589512+0000", 2024-09-11T06:51:16.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2712999999999997e-05 2024-09-11T06:51:16.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.899 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.899 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589527+0000", 2024-09-11T06:51:16.899 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4686e-05 2024-09-11T06:51:16.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.899 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.899 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589673+0000", 2024-09-11T06:51:16.899 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014630700000000001 2024-09-11T06:51:16.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.899 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.899 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589936+0000", 2024-09-11T06:51:16.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026234499999999998 2024-09-11T06:51:16.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:16.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.591001+0000", 2024-09-11T06:51:16.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001065023 2024-09-11T06:51:16.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:16.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.592757+0000", 2024-09-11T06:51:16.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0017560849999999999 2024-09-11T06:51:16.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:16.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.593297+0000", 2024-09-11T06:51:16.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00054007600000000001 2024-09-11T06:51:16.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.593385+0000", 2024-09-11T06:51:16.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8327999999999999e-05 2024-09-11T06:51:16.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.593451+0000", 2024-09-11T06:51:16.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5685999999999994e-05 2024-09-11T06:51:16.901 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.901 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.902 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.902 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5315.0:79 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-11T06:51:16.902 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:07.271557+0000", 2024-09-11T06:51:16.902 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.5203530339999993, 2024-09-11T06:51:16.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0023454019999999999, 2024-09-11T06:51:16.902 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.902 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.902 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.902 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:16.902 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5315", 2024-09-11T06:51:16.903 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2243844038", 2024-09-11T06:51:16.903 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 79 2024-09-11T06:51:16.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.903 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.903 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.903 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271557+0000", 2024-09-11T06:51:16.903 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.903 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.903 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271557+0000", 2024-09-11T06:51:16.903 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.904 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.904 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271560+0000", 2024-09-11T06:51:16.904 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3560000000000001e-06 2024-09-11T06:51:16.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.904 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.904 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271596+0000", 2024-09-11T06:51:16.904 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5593000000000003e-05 2024-09-11T06:51:16.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.904 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.904 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271640+0000", 2024-09-11T06:51:16.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4125999999999997e-05 2024-09-11T06:51:16.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.905 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.905 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271652+0000", 2024-09-11T06:51:16.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1853e-05 2024-09-11T06:51:16.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.905 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.905 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271785+0000", 2024-09-11T06:51:16.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013291299999999999 2024-09-11T06:51:16.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.906 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271948+0000", 2024-09-11T06:51:16.906 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000163191 2024-09-11T06:51:16.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:16.906 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.272260+0000", 2024-09-11T06:51:16.906 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00031196700000000002 2024-09-11T06:51:16.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:16.906 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.273293+0000", 2024-09-11T06:51:16.906 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010324139999999999 2024-09-11T06:51:16.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.907 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:16.907 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.273805+0000", 2024-09-11T06:51:16.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00051249399999999995 2024-09-11T06:51:16.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.907 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.907 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.273858+0000", 2024-09-11T06:51:16.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2975e-05 2024-09-11T06:51:16.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.907 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.907 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.273902+0000", 2024-09-11T06:51:16.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4520000000000001e-05 2024-09-11T06:51:16.908 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.908 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.908 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.908 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5239.0:52 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-11T06:51:16.908 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:08.176534+0000", 2024-09-11T06:51:16.908 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.6153761089999996, 2024-09-11T06:51:16.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0022800960000000001, 2024-09-11T06:51:16.908 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:16.908 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:16.908 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:16.908 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:16.908 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5239", 2024-09-11T06:51:16.909 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3565361812", 2024-09-11T06:51:16.909 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 52 2024-09-11T06:51:16.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.909 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:16.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.909 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:16.909 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176534+0000", 2024-09-11T06:51:16.909 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.909 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:16.909 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176534+0000", 2024-09-11T06:51:16.909 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:16.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.910 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:16.910 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176537+0000", 2024-09-11T06:51:16.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5600000000000001e-06 2024-09-11T06:51:16.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.910 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:16.910 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176551+0000", 2024-09-11T06:51:16.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3926e-05 2024-09-11T06:51:16.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.910 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:16.911 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176584+0000", 2024-09-11T06:51:16.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3105999999999998e-05 2024-09-11T06:51:16.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.911 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:16.911 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176600+0000", 2024-09-11T06:51:16.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6327000000000001e-05 2024-09-11T06:51:16.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.911 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:16.911 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176753+0000", 2024-09-11T06:51:16.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015280900000000001 2024-09-11T06:51:16.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.912 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.912 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:16.912 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176911+0000", 2024-09-11T06:51:16.912 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000158145 2024-09-11T06:51:16.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.912 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.912 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:16.912 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.177216+0000", 2024-09-11T06:51:16.912 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030545000000000002 2024-09-11T06:51:16.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.912 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.912 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:16.912 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.178461+0000", 2024-09-11T06:51:16.912 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012451210000000001 2024-09-11T06:51:16.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:16.913 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.178706+0000", 2024-09-11T06:51:16.913 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024469999999999998 2024-09-11T06:51:16.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:16.913 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.178763+0000", 2024-09-11T06:51:16.913 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7339000000000001e-05 2024-09-11T06:51:16.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:16.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:16.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:16.913 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.178814+0000", 2024-09-11T06:51:16.914 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0612999999999998e-05 2024-09-11T06:51:16.914 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.914 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.914 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.914 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:16.914 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:16.914 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:16.915 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:51:16.916 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:16.909+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:16.973 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T06:51:17.016 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:17.013+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:17.029 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:17.029 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:17.029 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:51:17.029 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:17.035 INFO:tasks.ceph.mon.a.smithi071.stderr:2024-09-11T06:51:17.030+0000 7f1296554640 -1 mon.a@0(leader).osd e98 definitely_dead 0 2024-09-11T06:51:17.043 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:51:17.114 INFO:tasks.workunit.client.0.smithi176.stderr:++ awk '/entries inspected/ {print $1}' 2024-09-11T06:51:17.114 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd journal inspect --pool rbd --image testrbdnbd21607 2024-09-11T06:51:17.117 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:17.113+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:17.153 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:17.153 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:17.153 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:51:17.154 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:51:17.154 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:17.168 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:51:17.201 INFO:tasks.workunit.client.0.smithi176.stderr:2024-09-11T06:51:17.194+0000 7f62f6ffd640 -1 JournalMetadata: 0x55aec6802dc0 failed to locate client: INSPECT 2024-09-11T06:51:17.207 INFO:tasks.workunit.client.0.smithi176.stderr:+ queue_length=0 2024-09-11T06:51:17.207 INFO:tasks.workunit.client.0.smithi176.stderr:+ tries=5 2024-09-11T06:51:17.207 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 5 -lt 10 ']' 2024-09-11T06:51:17.207 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 0 -gt 0 ']' 2024-09-11T06:51:17.207 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 0 -eq 0 ']' 2024-09-11T06:51:17.207 INFO:tasks.workunit.client.0.smithi176.stderr:+ unmap_device /dev/nbd2 24474 2024-09-11T06:51:17.207 INFO:tasks.workunit.client.0.smithi176.stderr:+ local args=/dev/nbd2 2024-09-11T06:51:17.207 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pid=24474 2024-09-11T06:51:17.207 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd2 2024-09-11T06:51:17.207 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:17.207 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:17.208 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:17.208 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:51:17.209 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:51:17.209 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:17.210 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd2 2024-09-11T06:51:17.218 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:17.213+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:17.280 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:17.280 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:51:17.281 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:51:17.281 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:51:17.281 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.281 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:26 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 140250278007648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:17.281 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:51:17.281 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 46.546829197999998, 2024-09-11T06:51:17.281 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034645140000000001, 2024-09-11T06:51:17.281 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:17.281 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:17.281 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.281 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:17.281 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:17.282 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:17.282 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:51:17.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.282 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:17.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.282 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:17.282 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:51:17.282 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.282 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:17.282 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:51:17.283 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.283 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:17.283 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729634+0000", 2024-09-11T06:51:17.283 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0470000000000001e-06 2024-09-11T06:51:17.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.283 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:17.283 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729640+0000", 2024-09-11T06:51:17.283 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8860000000000001e-06 2024-09-11T06:51:17.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.284 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:17.284 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729654+0000", 2024-09-11T06:51:17.284 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:51:17.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.284 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.284 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.284 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729664+0000", 2024-09-11T06:51:17.284 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.594e-06 2024-09-11T06:51:17.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.284 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.284 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:17.284 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730422+0000", 2024-09-11T06:51:17.284 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000758648 2024-09-11T06:51:17.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.285 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:17.285 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730571+0000", 2024-09-11T06:51:17.285 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000148932 2024-09-11T06:51:17.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.285 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:17.285 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730936+0000", 2024-09-11T06:51:17.285 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036510900000000002 2024-09-11T06:51:17.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.285 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:17.285 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732159+0000", 2024-09-11T06:51:17.286 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001222302 2024-09-11T06:51:17.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.286 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.286 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732763+0000", 2024-09-11T06:51:17.286 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00060421599999999998 2024-09-11T06:51:17.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.286 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:17.286 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732916+0000", 2024-09-11T06:51:17.286 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015264 2024-09-11T06:51:17.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.287 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:17.287 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733097+0000", 2024-09-11T06:51:17.287 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000181504 2024-09-11T06:51:17.287 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.287 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:17.287 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.287 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.287 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:27 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:17.287 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:51:17.287 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 46.545855295999999, 2024-09-11T06:51:17.287 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003072852, 2024-09-11T06:51:17.287 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:17.288 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:17.288 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:17.288 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:17.288 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:17.288 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:17.288 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27 2024-09-11T06:51:17.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.288 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:17.288 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.288 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:17.288 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:51:17.288 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.288 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.289 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:17.289 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:51:17.289 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.289 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.289 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.289 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:17.289 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730608+0000", 2024-09-11T06:51:17.289 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8369999999999999e-06 2024-09-11T06:51:17.289 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.289 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.289 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:17.289 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730617+0000", 2024-09-11T06:51:17.290 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1859999999999992e-06 2024-09-11T06:51:17.290 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.290 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.290 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:17.290 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730636+0000", 2024-09-11T06:51:17.290 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9296e-05 2024-09-11T06:51:17.290 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.290 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.290 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.290 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730674+0000", 2024-09-11T06:51:17.290 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8609000000000003e-05 2024-09-11T06:51:17.290 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.290 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.290 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:17.291 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731496+0000", 2024-09-11T06:51:17.291 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00082112299999999999 2024-09-11T06:51:17.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.291 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:17.291 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731722+0000", 2024-09-11T06:51:17.291 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022651400000000001 2024-09-11T06:51:17.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.291 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:17.291 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733215+0000", 2024-09-11T06:51:17.291 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001493257 2024-09-11T06:51:17.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.292 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:17.292 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733452+0000", 2024-09-11T06:51:17.292 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023631600000000001 2024-09-11T06:51:17.292 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.292 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.292 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:17.292 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733679+0000", 2024-09-11T06:51:17.292 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022771400000000001 2024-09-11T06:51:17.292 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.292 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:17.292 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.292 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.292 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.293 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:73 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:17.293 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:51:17.293 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 46.545771012000003, 2024-09-11T06:51:17.293 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033454439999999999, 2024-09-11T06:51:17.293 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:17.293 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:17.293 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:17.294 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:17.294 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:51:17.294 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:51:17.294 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73 2024-09-11T06:51:17.294 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.294 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:17.294 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.294 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:17.294 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:51:17.294 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.294 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.294 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.294 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:17.294 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:51:17.295 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.295 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.295 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:17.295 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730694+0000", 2024-09-11T06:51:17.295 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5099999999999999e-06 2024-09-11T06:51:17.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.295 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.295 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:17.295 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730708+0000", 2024-09-11T06:51:17.295 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3237e-05 2024-09-11T06:51:17.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.295 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.295 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:17.296 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730738+0000", 2024-09-11T06:51:17.296 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0688999999999997e-05 2024-09-11T06:51:17.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.296 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.296 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.296 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730759+0000", 2024-09-11T06:51:17.296 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0818999999999998e-05 2024-09-11T06:51:17.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.296 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.296 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:17.296 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731841+0000", 2024-09-11T06:51:17.296 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010821819999999999 2024-09-11T06:51:17.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.297 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.297 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:17.297 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732092+0000", 2024-09-11T06:51:17.297 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025086299999999999 2024-09-11T06:51:17.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.297 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.297 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:17.297 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733730+0000", 2024-09-11T06:51:17.297 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0016374600000000001 2024-09-11T06:51:17.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.297 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.297 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:17.297 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733866+0000", 2024-09-11T06:51:17.298 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000136666 2024-09-11T06:51:17.298 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.298 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.298 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:17.298 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.734036+0000", 2024-09-11T06:51:17.298 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000170018 2024-09-11T06:51:17.298 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.298 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:17.298 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.298 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.298 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.298 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:18 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:17.298 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:51:17.298 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 46.402353822999999, 2024-09-11T06:51:17.299 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0031646579999999999, 2024-09-11T06:51:17.299 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:17.299 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:17.299 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.299 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:17.299 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:17.299 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:17.299 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18 2024-09-11T06:51:17.299 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.299 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:17.299 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.299 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:17.299 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:51:17.300 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.300 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.300 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:17.300 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:51:17.300 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.300 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.300 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:17.300 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874109+0000", 2024-09-11T06:51:17.300 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.426e-06 2024-09-11T06:51:17.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.300 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.301 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:17.301 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874118+0000", 2024-09-11T06:51:17.301 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6929999999999996e-06 2024-09-11T06:51:17.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.301 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.301 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:17.301 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874134+0000", 2024-09-11T06:51:17.301 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5503e-05 2024-09-11T06:51:17.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.301 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.301 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.301 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874144+0000", 2024-09-11T06:51:17.301 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0207000000000001e-05 2024-09-11T06:51:17.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.302 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.302 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:17.302 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874545+0000", 2024-09-11T06:51:17.302 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040120800000000001 2024-09-11T06:51:17.302 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.302 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.302 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:17.302 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874686+0000", 2024-09-11T06:51:17.302 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014128599999999999 2024-09-11T06:51:17.302 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.302 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.302 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:17.302 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875023+0000", 2024-09-11T06:51:17.303 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033706600000000001 2024-09-11T06:51:17.303 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.303 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.303 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:17.303 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.876217+0000", 2024-09-11T06:51:17.303 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011938999999999999 2024-09-11T06:51:17.303 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.303 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.303 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.303 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877042+0000", 2024-09-11T06:51:17.303 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00082499899999999998 2024-09-11T06:51:17.303 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.303 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.303 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:17.304 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877128+0000", 2024-09-11T06:51:17.304 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5903999999999996e-05 2024-09-11T06:51:17.304 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.304 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.304 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:17.304 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877273+0000", 2024-09-11T06:51:17.304 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144466 2024-09-11T06:51:17.304 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.304 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:17.304 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.304 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.304 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.304 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:24 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:17.304 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:51:17.304 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 46.387848116999997, 2024-09-11T06:51:17.305 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002980271, 2024-09-11T06:51:17.305 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:17.305 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:17.305 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.305 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:17.305 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:17.305 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:17.305 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 24 2024-09-11T06:51:17.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.305 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:17.305 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.305 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:17.305 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:51:17.305 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.306 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:17.306 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:51:17.306 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.306 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:17.306 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888616+0000", 2024-09-11T06:51:17.306 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0030000000000001e-06 2024-09-11T06:51:17.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.306 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:17.306 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888626+0000", 2024-09-11T06:51:17.307 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0502999999999999e-05 2024-09-11T06:51:17.307 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.307 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.307 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:17.307 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888655+0000", 2024-09-11T06:51:17.307 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8419999999999999e-05 2024-09-11T06:51:17.307 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.307 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.307 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.307 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888688+0000", 2024-09-11T06:51:17.307 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3212e-05 2024-09-11T06:51:17.307 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.307 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.307 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:17.308 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888829+0000", 2024-09-11T06:51:17.308 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014114299999999999 2024-09-11T06:51:17.308 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.308 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.308 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:17.308 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.889082+0000", 2024-09-11T06:51:17.308 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025279499999999999 2024-09-11T06:51:17.308 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.308 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.308 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:17.308 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.889577+0000", 2024-09-11T06:51:17.308 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00049536299999999997 2024-09-11T06:51:17.308 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.309 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.309 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:17.309 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.890811+0000", 2024-09-11T06:51:17.309 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001233858 2024-09-11T06:51:17.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.309 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.309 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.309 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891368+0000", 2024-09-11T06:51:17.309 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00055677400000000001 2024-09-11T06:51:17.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.309 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.310 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:17.310 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891456+0000", 2024-09-11T06:51:17.310 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8246999999999997e-05 2024-09-11T06:51:17.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.310 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.310 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:17.310 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891594+0000", 2024-09-11T06:51:17.310 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013795300000000001 2024-09-11T06:51:17.310 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.310 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:17.310 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.311 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.311 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:26 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 139740787512064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:17.311 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:51:17.311 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 46.384145472999997, 2024-09-11T06:51:17.311 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003613733, 2024-09-11T06:51:17.311 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:17.311 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:17.311 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.311 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:17.311 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:17.311 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:17.311 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:51:17.312 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.312 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:17.312 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.312 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:17.312 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:51:17.312 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.312 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.312 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.312 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:17.312 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:51:17.313 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.313 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.313 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.313 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:17.313 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892318+0000", 2024-09-11T06:51:17.313 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.623e-06 2024-09-11T06:51:17.313 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.313 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.313 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:17.313 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892325+0000", 2024-09-11T06:51:17.313 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:51:17.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.314 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.314 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:17.314 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892346+0000", 2024-09-11T06:51:17.314 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1246e-05 2024-09-11T06:51:17.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.314 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.315 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.315 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892362+0000", 2024-09-11T06:51:17.315 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5409000000000001e-05 2024-09-11T06:51:17.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.315 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:17.315 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892883+0000", 2024-09-11T06:51:17.315 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00052087599999999998 2024-09-11T06:51:17.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.315 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:17.316 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893123+0000", 2024-09-11T06:51:17.316 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000240769 2024-09-11T06:51:17.316 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.316 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.316 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:17.316 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893526+0000", 2024-09-11T06:51:17.316 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040300200000000002 2024-09-11T06:51:17.316 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.316 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.316 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:17.316 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894820+0000", 2024-09-11T06:51:17.317 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012934159999999999 2024-09-11T06:51:17.317 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.317 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.317 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.317 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895608+0000", 2024-09-11T06:51:17.317 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00078788100000000004 2024-09-11T06:51:17.317 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.317 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.317 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:17.317 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895724+0000", 2024-09-11T06:51:17.317 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011595999999999999 2024-09-11T06:51:17.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.318 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.318 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:17.318 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895930+0000", 2024-09-11T06:51:17.318 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000206391 2024-09-11T06:51:17.318 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.318 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:17.318 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.318 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.318 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:133 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:17.319 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:51:17.319 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 46.383347456000003, 2024-09-11T06:51:17.319 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035017360000000001, 2024-09-11T06:51:17.319 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:17.319 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:17.319 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:17.319 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:17.319 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:51:17.319 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:51:17.319 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 133 2024-09-11T06:51:17.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.320 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:17.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.320 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:17.320 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:51:17.320 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.320 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:17.320 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:51:17.320 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.321 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:17.321 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893116+0000", 2024-09-11T06:51:17.321 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1629999999999999e-06 2024-09-11T06:51:17.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.321 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:17.321 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893128+0000", 2024-09-11T06:51:17.321 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.11e-05 2024-09-11T06:51:17.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.322 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.322 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:17.322 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893156+0000", 2024-09-11T06:51:17.322 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8266e-05 2024-09-11T06:51:17.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.322 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.322 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.322 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893171+0000", 2024-09-11T06:51:17.322 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5426000000000001e-05 2024-09-11T06:51:17.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.322 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.323 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:17.323 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894561+0000", 2024-09-11T06:51:17.323 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013899959999999999 2024-09-11T06:51:17.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.323 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.323 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:17.323 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894781+0000", 2024-09-11T06:51:17.323 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021987100000000001 2024-09-11T06:51:17.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.323 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.323 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:17.324 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896349+0000", 2024-09-11T06:51:17.324 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015681779999999999 2024-09-11T06:51:17.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.324 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:17.324 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896458+0000", 2024-09-11T06:51:17.324 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000108187 2024-09-11T06:51:17.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.324 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:17.324 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896616+0000", 2024-09-11T06:51:17.324 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015854899999999999 2024-09-11T06:51:17.324 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.325 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:17.325 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.325 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:27 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:17.325 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:51:17.325 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 46.383336075999999, 2024-09-11T06:51:17.325 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003151989, 2024-09-11T06:51:17.325 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:17.325 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:17.325 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:17.325 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:17.325 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:17.325 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:17.326 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27 2024-09-11T06:51:17.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.326 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:17.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.326 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:17.326 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:51:17.326 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.326 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:17.326 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:51:17.326 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.327 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:17.327 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893128+0000", 2024-09-11T06:51:17.327 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.013e-06 2024-09-11T06:51:17.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.327 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.327 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:17.327 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893136+0000", 2024-09-11T06:51:17.327 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4190000000000002e-06 2024-09-11T06:51:17.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.327 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.327 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:17.327 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893158+0000", 2024-09-11T06:51:17.327 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1506999999999999e-05 2024-09-11T06:51:17.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.328 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.328 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.328 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893170+0000", 2024-09-11T06:51:17.328 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2741999999999999e-05 2024-09-11T06:51:17.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.328 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.328 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:17.328 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894255+0000", 2024-09-11T06:51:17.328 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001084453 2024-09-11T06:51:17.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.328 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.328 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:17.328 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894461+0000", 2024-09-11T06:51:17.329 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00020586700000000001 2024-09-11T06:51:17.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.329 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.329 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:17.329 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896033+0000", 2024-09-11T06:51:17.329 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001572092 2024-09-11T06:51:17.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.329 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.329 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:17.329 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896149+0000", 2024-09-11T06:51:17.329 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116636 2024-09-11T06:51:17.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.329 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.329 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:17.330 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896278+0000", 2024-09-11T06:51:17.330 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012826 2024-09-11T06:51:17.330 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.330 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:17.330 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.330 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.330 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:176 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch reconnect cookie 139735284585312 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:17.330 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:51:17.330 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 46.351969257, 2024-09-11T06:51:17.330 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033620960000000002, 2024-09-11T06:51:17.330 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:17.330 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:17.330 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:17.331 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:17.331 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:51:17.331 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:51:17.331 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 176 2024-09-11T06:51:17.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.331 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:17.331 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.331 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:17.331 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:51:17.331 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.331 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.331 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:17.331 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:51:17.332 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.332 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.332 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.332 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:17.332 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924494+0000", 2024-09-11T06:51:17.332 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.967e-06 2024-09-11T06:51:17.332 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.332 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.332 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:17.332 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924501+0000", 2024-09-11T06:51:17.332 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2659999999999998e-06 2024-09-11T06:51:17.332 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.332 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.332 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:17.333 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924520+0000", 2024-09-11T06:51:17.333 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9038999999999998e-05 2024-09-11T06:51:17.333 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.333 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.333 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.333 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924535+0000", 2024-09-11T06:51:17.333 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5359999999999999e-05 2024-09-11T06:51:17.333 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.333 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.333 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:17.333 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.925544+0000", 2024-09-11T06:51:17.333 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010085109999999999 2024-09-11T06:51:17.333 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.333 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.334 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:17.334 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.925671+0000", 2024-09-11T06:51:17.334 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012756599999999999 2024-09-11T06:51:17.334 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.334 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.334 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:17.334 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.927855+0000", 2024-09-11T06:51:17.334 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0021833870000000002 2024-09-11T06:51:17.334 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.334 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:17.334 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.334 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.334 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.335 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4828.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch watch cookie 140587969811264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-11T06:51:17.335 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:51:17.335 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.030266298999997, 2024-09-11T06:51:17.335 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032778859999999998, 2024-09-11T06:51:17.335 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:17.335 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:17.335 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.335 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:17.335 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4828", 2024-09-11T06:51:17.335 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2410288650", 2024-09-11T06:51:17.335 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:51:17.335 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.335 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:17.335 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.336 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:17.336 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:51:17.336 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.336 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.336 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:17.336 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:51:17.336 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.336 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.336 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:17.336 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246198+0000", 2024-09-11T06:51:17.336 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.667e-06 2024-09-11T06:51:17.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.337 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.337 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:17.337 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246209+0000", 2024-09-11T06:51:17.337 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0409e-05 2024-09-11T06:51:17.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.337 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.337 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:17.337 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246241+0000", 2024-09-11T06:51:17.337 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2592999999999998e-05 2024-09-11T06:51:17.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.337 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.337 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.337 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246260+0000", 2024-09-11T06:51:17.338 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9343000000000001e-05 2024-09-11T06:51:17.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.338 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:17.338 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246466+0000", 2024-09-11T06:51:17.338 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000205701 2024-09-11T06:51:17.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.338 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:17.338 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246713+0000", 2024-09-11T06:51:17.338 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024646999999999999 2024-09-11T06:51:17.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.338 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:17.339 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.247133+0000", 2024-09-11T06:51:17.339 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000420536 2024-09-11T06:51:17.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.339 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:17.339 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.248541+0000", 2024-09-11T06:51:17.339 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001407801 2024-09-11T06:51:17.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.339 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.339 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249241+0000", 2024-09-11T06:51:17.339 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070043500000000001 2024-09-11T06:51:17.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.340 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:17.340 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249326+0000", 2024-09-11T06:51:17.340 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5037999999999995e-05 2024-09-11T06:51:17.340 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.340 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.340 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:17.340 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249473+0000", 2024-09-11T06:51:17.340 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014689299999999999 2024-09-11T06:51:17.340 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.340 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:17.340 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.340 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.340 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.340 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5313.0:25 2.1 2:967e1786:ns::rbd_object_map.11adc335555f.0000000000000004:head [writefull 0~38] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:51:17.341 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.256788+0000", 2024-09-11T06:51:17.341 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 38.019673300000001, 2024-09-11T06:51:17.341 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0030446449999999999, 2024-09-11T06:51:17.341 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:17.341 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:17.341 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.341 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:17.341 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5313", 2024-09-11T06:51:17.341 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/624314229", 2024-09-11T06:51:17.341 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 25 2024-09-11T06:51:17.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.341 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:17.341 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.342 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:17.342 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256788+0000", 2024-09-11T06:51:17.342 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.342 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.342 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.342 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:17.342 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256788+0000", 2024-09-11T06:51:17.342 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.342 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.342 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.342 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:17.342 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256791+0000", 2024-09-11T06:51:17.342 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5770000000000001e-06 2024-09-11T06:51:17.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.343 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.343 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:17.343 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256802+0000", 2024-09-11T06:51:17.343 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0742e-05 2024-09-11T06:51:17.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.343 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.343 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:17.343 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256823+0000", 2024-09-11T06:51:17.343 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0999999999999999e-05 2024-09-11T06:51:17.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.343 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.343 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.343 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256841+0000", 2024-09-11T06:51:17.344 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8099000000000001e-05 2024-09-11T06:51:17.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.344 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.344 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:17.344 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256988+0000", 2024-09-11T06:51:17.344 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014756400000000001 2024-09-11T06:51:17.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.344 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.344 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:17.344 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.257234+0000", 2024-09-11T06:51:17.344 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024524399999999999 2024-09-11T06:51:17.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.344 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.344 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:17.345 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.257446+0000", 2024-09-11T06:51:17.345 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000212241 2024-09-11T06:51:17.345 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.345 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:17.345 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259292+0000", 2024-09-11T06:51:17.345 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001845937 2024-09-11T06:51:17.345 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.345 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.345 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259681+0000", 2024-09-11T06:51:17.345 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00038940699999999998 2024-09-11T06:51:17.345 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.346 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:17.346 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259770+0000", 2024-09-11T06:51:17.346 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8385000000000002e-05 2024-09-11T06:51:17.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.346 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.346 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:17.346 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259833+0000", 2024-09-11T06:51:17.346 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3448999999999995e-05 2024-09-11T06:51:17.346 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.346 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:17.346 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.346 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.347 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5328.0:14 2.1 2:967e1786:ns::rbd_object_map.11adc335555f.0000000000000004:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:51:17.347 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.396257+0000", 2024-09-11T06:51:17.347 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.880204851000002, 2024-09-11T06:51:17.347 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032125370000000001, 2024-09-11T06:51:17.347 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:17.347 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:17.347 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:17.347 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:17.347 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5328", 2024-09-11T06:51:17.347 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2386493698", 2024-09-11T06:51:17.347 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 14 2024-09-11T06:51:17.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.347 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:17.347 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.348 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:17.348 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396257+0000", 2024-09-11T06:51:17.348 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.348 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.348 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.348 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:17.348 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396257+0000", 2024-09-11T06:51:17.348 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.348 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.348 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.348 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:17.348 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396259+0000", 2024-09-11T06:51:17.348 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4830000000000002e-06 2024-09-11T06:51:17.348 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.349 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:17.349 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396271+0000", 2024-09-11T06:51:17.349 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.115e-05 2024-09-11T06:51:17.349 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.349 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:17.349 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396292+0000", 2024-09-11T06:51:17.349 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1761999999999999e-05 2024-09-11T06:51:17.349 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.349 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.349 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396314+0000", 2024-09-11T06:51:17.349 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1982999999999999e-05 2024-09-11T06:51:17.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.350 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:17.350 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396409+0000", 2024-09-11T06:51:17.350 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4250000000000001e-05 2024-09-11T06:51:17.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.350 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:17.350 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396464+0000", 2024-09-11T06:51:17.350 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5884000000000002e-05 2024-09-11T06:51:17.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.350 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:17.350 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.399469+0000", 2024-09-11T06:51:17.351 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0030050250000000001 2024-09-11T06:51:17.351 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.351 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:17.351 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.351 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.351 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.351 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:6 3.5 3:b2cc3336:::rbd_id.testrbdnbd21607:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:17.351 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.591899+0000", 2024-09-11T06:51:17.351 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.68456235, 2024-09-11T06:51:17.351 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032382999999999999, 2024-09-11T06:51:17.351 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:17.351 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:17.351 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.352 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:17.352 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:51:17.352 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:51:17.352 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 6 2024-09-11T06:51:17.352 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.352 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:17.352 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.352 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:17.352 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591899+0000", 2024-09-11T06:51:17.352 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.352 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.352 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.352 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:17.353 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591899+0000", 2024-09-11T06:51:17.353 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.353 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.353 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:17.353 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591902+0000", 2024-09-11T06:51:17.353 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0770000000000002e-06 2024-09-11T06:51:17.353 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.353 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:17.353 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591913+0000", 2024-09-11T06:51:17.353 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1182999999999999e-05 2024-09-11T06:51:17.353 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.354 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:17.354 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591940+0000", 2024-09-11T06:51:17.354 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7233000000000002e-05 2024-09-11T06:51:17.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.354 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.354 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.354 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591954+0000", 2024-09-11T06:51:17.354 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4503e-05 2024-09-11T06:51:17.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.354 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.354 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:17.355 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.592071+0000", 2024-09-11T06:51:17.355 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116731 2024-09-11T06:51:17.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.355 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:17.355 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.592274+0000", 2024-09-11T06:51:17.355 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00020320900000000001 2024-09-11T06:51:17.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.355 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 3", 2024-09-11T06:51:17.355 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.592576+0000", 2024-09-11T06:51:17.355 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030179000000000002 2024-09-11T06:51:17.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.356 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:17.356 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.594072+0000", 2024-09-11T06:51:17.356 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001495496 2024-09-11T06:51:17.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.356 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.356 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.594945+0000", 2024-09-11T06:51:17.356 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00087379799999999996 2024-09-11T06:51:17.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.356 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:17.356 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.595023+0000", 2024-09-11T06:51:17.357 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7751999999999997e-05 2024-09-11T06:51:17.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.357 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:17.357 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.595138+0000", 2024-09-11T06:51:17.357 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000114528 2024-09-11T06:51:17.357 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.357 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:17.357 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.357 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:44 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:17.357 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.540846+0000", 2024-09-11T06:51:17.358 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 28.735616218000001, 2024-09-11T06:51:17.358 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032138639999999999, 2024-09-11T06:51:17.358 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:17.358 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:17.358 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.358 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:17.358 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:17.358 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:17.358 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 44 2024-09-11T06:51:17.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.358 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:17.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.359 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:17.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540846+0000", 2024-09-11T06:51:17.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.359 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:17.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540846+0000", 2024-09-11T06:51:17.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.359 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:17.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540848+0000", 2024-09-11T06:51:17.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5469999999999999e-06 2024-09-11T06:51:17.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.360 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:17.360 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540863+0000", 2024-09-11T06:51:17.360 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4613000000000001e-05 2024-09-11T06:51:17.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.360 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:17.360 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540895+0000", 2024-09-11T06:51:17.360 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2153000000000002e-05 2024-09-11T06:51:17.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.360 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.360 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540912+0000", 2024-09-11T06:51:17.361 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7405999999999999e-05 2024-09-11T06:51:17.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.361 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:17.361 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.541111+0000", 2024-09-11T06:51:17.361 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019849599999999999 2024-09-11T06:51:17.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.361 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:17.361 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.541269+0000", 2024-09-11T06:51:17.361 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015792399999999999 2024-09-11T06:51:17.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.362 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:17.362 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.541853+0000", 2024-09-11T06:51:17.362 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00058443799999999995 2024-09-11T06:51:17.362 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.362 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.362 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:17.362 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.543133+0000", 2024-09-11T06:51:17.362 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012797850000000001 2024-09-11T06:51:17.362 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.362 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.362 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.362 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.543905+0000", 2024-09-11T06:51:17.362 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000771655 2024-09-11T06:51:17.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.363 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.363 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:17.363 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.543993+0000", 2024-09-11T06:51:17.363 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8168e-05 2024-09-11T06:51:17.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.363 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.363 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:17.363 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.544059+0000", 2024-09-11T06:51:17.363 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6679000000000003e-05 2024-09-11T06:51:17.363 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.363 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:17.363 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.364 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5553.0:24 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 139650593197904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:17.364 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.566567+0000", 2024-09-11T06:51:17.364 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 28.709894947999999, 2024-09-11T06:51:17.364 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034350790000000002, 2024-09-11T06:51:17.364 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:17.364 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:17.364 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.364 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:17.364 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5553", 2024-09-11T06:51:17.364 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3590511604", 2024-09-11T06:51:17.364 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 24 2024-09-11T06:51:17.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.365 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:17.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.365 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:17.365 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566567+0000", 2024-09-11T06:51:17.365 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.365 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:17.365 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566567+0000", 2024-09-11T06:51:17.365 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.365 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:17.365 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566569+0000", 2024-09-11T06:51:17.366 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.973e-06 2024-09-11T06:51:17.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.366 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:17.366 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566578+0000", 2024-09-11T06:51:17.366 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8170000000000005e-06 2024-09-11T06:51:17.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.366 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:17.366 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566602+0000", 2024-09-11T06:51:17.366 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4596e-05 2024-09-11T06:51:17.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.366 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.366 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566619+0000", 2024-09-11T06:51:17.367 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7099999999999999e-05 2024-09-11T06:51:17.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.367 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.367 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:17.367 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.567321+0000", 2024-09-11T06:51:17.367 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070168200000000002 2024-09-11T06:51:17.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.367 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.367 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:17.367 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.567454+0000", 2024-09-11T06:51:17.367 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013286400000000001 2024-09-11T06:51:17.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.367 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.368 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:17.368 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.567674+0000", 2024-09-11T06:51:17.368 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000219679 2024-09-11T06:51:17.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.368 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.368 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:17.368 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.569014+0000", 2024-09-11T06:51:17.368 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013404129999999999 2024-09-11T06:51:17.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.368 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.368 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.368 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.569634+0000", 2024-09-11T06:51:17.369 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00062029200000000004 2024-09-11T06:51:17.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.369 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:17.369 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.569805+0000", 2024-09-11T06:51:17.369 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000170366 2024-09-11T06:51:17.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.369 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:17.369 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.570002+0000", 2024-09-11T06:51:17.369 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019729700000000001 2024-09-11T06:51:17.370 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.370 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:17.370 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.370 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.370 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:82 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch reconnect cookie 139888024456048 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:17.370 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.690185+0000", 2024-09-11T06:51:17.370 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 28.586276946000002, 2024-09-11T06:51:17.370 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0045884799999999998, 2024-09-11T06:51:17.370 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:17.370 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:17.371 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:17.371 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:17.371 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:17.371 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:17.371 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 82 2024-09-11T06:51:17.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.371 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:17.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.371 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:17.371 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690185+0000", 2024-09-11T06:51:17.371 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.372 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:17.372 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690185+0000", 2024-09-11T06:51:17.372 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.372 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:17.372 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690186+0000", 2024-09-11T06:51:17.372 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8400000000000002e-07 2024-09-11T06:51:17.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:17.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690190+0000", 2024-09-11T06:51:17.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5499999999999996e-06 2024-09-11T06:51:17.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:17.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690203+0000", 2024-09-11T06:51:17.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2228999999999999e-05 2024-09-11T06:51:17.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.374 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690228+0000", 2024-09-11T06:51:17.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5587000000000001e-05 2024-09-11T06:51:17.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:17.374 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.691298+0000", 2024-09-11T06:51:17.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001070218 2024-09-11T06:51:17.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:17.375 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.691511+0000", 2024-09-11T06:51:17.375 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021301299999999999 2024-09-11T06:51:17.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.375 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:17.375 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.694773+0000", 2024-09-11T06:51:17.375 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003261899 2024-09-11T06:51:17.375 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.375 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:17.375 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.376 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:83 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify cookie 139887621709056] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:17.376 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.690242+0000", 2024-09-11T06:51:17.376 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 28.586219774, 2024-09-11T06:51:17.376 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003040067, 2024-09-11T06:51:17.376 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:17.377 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:17.377 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:17.377 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:17.377 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:17.377 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:17.377 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 83 2024-09-11T06:51:17.377 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.377 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:17.377 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.377 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:17.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690242+0000", 2024-09-11T06:51:17.378 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.378 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:17.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690242+0000", 2024-09-11T06:51:17.378 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.378 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:17.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690243+0000", 2024-09-11T06:51:17.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0640000000000001e-06 2024-09-11T06:51:17.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.379 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:17.379 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690247+0000", 2024-09-11T06:51:17.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3760000000000001e-06 2024-09-11T06:51:17.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.379 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:17.379 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690258+0000", 2024-09-11T06:51:17.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0874e-05 2024-09-11T06:51:17.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.380 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690266+0000", 2024-09-11T06:51:17.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5429999999999996e-06 2024-09-11T06:51:17.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:17.380 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.692564+0000", 2024-09-11T06:51:17.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0022977010000000001 2024-09-11T06:51:17.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:17.380 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.692882+0000", 2024-09-11T06:51:17.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00031862699999999998 2024-09-11T06:51:17.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.381 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:17.381 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693282+0000", 2024-09-11T06:51:17.381 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000399882 2024-09-11T06:51:17.381 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.381 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:17.381 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.381 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:84 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify cookie 139887823033168] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:17.381 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:51:17.381 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 28.58618517, 2024-09-11T06:51:17.382 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035622620000000001, 2024-09-11T06:51:17.382 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:17.382 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:17.382 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:17.382 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:17.382 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:17.382 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:17.382 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 84 2024-09-11T06:51:17.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.382 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:17.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.382 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:17.382 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:51:17.382 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.383 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:17.383 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:51:17.383 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.383 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:17.383 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:51:17.383 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5300000000000003e-07 2024-09-11T06:51:17.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.383 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:17.383 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690281+0000", 2024-09-11T06:51:17.384 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9330000000000003e-06 2024-09-11T06:51:17.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.384 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:17.384 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690291+0000", 2024-09-11T06:51:17.384 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7359999999999996e-06 2024-09-11T06:51:17.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.384 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.384 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690298+0000", 2024-09-11T06:51:17.384 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3699999999999997e-06 2024-09-11T06:51:17.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.385 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:17.385 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693380+0000", 2024-09-11T06:51:17.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0030811060000000001 2024-09-11T06:51:17.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.385 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:17.385 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693599+0000", 2024-09-11T06:51:17.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021947199999999999 2024-09-11T06:51:17.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.385 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:17.385 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693839+0000", 2024-09-11T06:51:17.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023989200000000001 2024-09-11T06:51:17.385 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.386 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:17.386 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.386 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5098.0:25 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 140617967560800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:17.386 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.697727+0000", 2024-09-11T06:51:17.386 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 28.578734594, 2024-09-11T06:51:17.386 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035341330000000001, 2024-09-11T06:51:17.386 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:17.386 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:17.386 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.386 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:17.386 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5098", 2024-09-11T06:51:17.387 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/844541331", 2024-09-11T06:51:17.387 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 25 2024-09-11T06:51:17.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.387 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:17.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.387 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:17.387 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697727+0000", 2024-09-11T06:51:17.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.387 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:17.387 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697727+0000", 2024-09-11T06:51:17.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.388 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:17.388 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697729+0000", 2024-09-11T06:51:17.388 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.33e-06 2024-09-11T06:51:17.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.388 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:17.388 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697734+0000", 2024-09-11T06:51:17.388 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6590000000000004e-06 2024-09-11T06:51:17.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.388 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:17.388 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697749+0000", 2024-09-11T06:51:17.389 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4867e-05 2024-09-11T06:51:17.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.389 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.389 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697759+0000", 2024-09-11T06:51:17.389 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0073e-05 2024-09-11T06:51:17.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.389 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:17.389 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.698715+0000", 2024-09-11T06:51:17.389 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00095545799999999996 2024-09-11T06:51:17.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.389 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:17.390 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.698850+0000", 2024-09-11T06:51:17.390 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000135814 2024-09-11T06:51:17.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.390 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:17.390 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.699085+0000", 2024-09-11T06:51:17.390 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023507199999999999 2024-09-11T06:51:17.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.390 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:17.390 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.700258+0000", 2024-09-11T06:51:17.390 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011726600000000001 2024-09-11T06:51:17.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.391 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.391 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.701061+0000", 2024-09-11T06:51:17.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00080256100000000003 2024-09-11T06:51:17.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.391 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:17.391 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.701144+0000", 2024-09-11T06:51:17.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3188000000000004e-05 2024-09-11T06:51:17.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.391 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:17.391 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.701261+0000", 2024-09-11T06:51:17.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000117451 2024-09-11T06:51:17.392 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.392 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:17.392 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.392 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5380.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b,call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b,call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b,call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:17.392 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:12.536478+0000", 2024-09-11T06:51:17.392 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.739983413, 2024-09-11T06:51:17.392 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032220090000000001, 2024-09-11T06:51:17.392 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:17.392 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:17.392 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:17.392 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:17.392 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5380", 2024-09-11T06:51:17.393 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/290149879", 2024-09-11T06:51:17.393 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:51:17.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.393 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:17.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.393 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:17.393 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536478+0000", 2024-09-11T06:51:17.393 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.394 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:17.394 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536478+0000", 2024-09-11T06:51:17.394 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:17.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.394 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:17.394 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536481+0000", 2024-09-11T06:51:17.394 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4059999999999999e-06 2024-09-11T06:51:17.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.394 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:17.394 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536493+0000", 2024-09-11T06:51:17.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2003e-05 2024-09-11T06:51:17.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:17.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536526+0000", 2024-09-11T06:51:17.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2839999999999997e-05 2024-09-11T06:51:17.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536542+0000", 2024-09-11T06:51:17.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6120000000000002e-05 2024-09-11T06:51:17.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.396 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:17.396 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536662+0000", 2024-09-11T06:51:17.396 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000120285 2024-09-11T06:51:17.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.396 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:17.396 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536946+0000", 2024-09-11T06:51:17.396 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00028412400000000002 2024-09-11T06:51:17.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:17.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:17.396 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:17.396 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.539700+0000", 2024-09-11T06:51:17.396 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027542310000000002 2024-09-11T06:51:17.396 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.397 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:17.397 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.397 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:17.397 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:17.397 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:17.398 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:51:17.398 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^24474\b' 2024-09-11T06:51:17.399 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^24474\b' 2024-09-11T06:51:17.399 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:51:17.400 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:17.393+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:17.400 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:51:17.401 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -C rbd-nbd 2024-09-11T06:51:17.401 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^ *24474\b' 2024-09-11T06:51:17.401 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^ *24474\b' 2024-09-11T06:51:17.406 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:51:17.407 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 0.5 2024-09-11T06:51:17.499 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:17.494+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:17.533 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:17.533 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:17.533 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:51:17.533 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:17.547 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:51:17.600 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:17.597+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:17.663 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:51:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:51:17.664 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:17.678 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:51:17.702 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:17.698+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:17.802 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:17.797+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:17.844 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:17.844 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:51:17.844 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:51:17.844 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:51:17.844 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.844 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:17.845 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:51:17.845 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 131.570480788, 2024-09-11T06:51:17.845 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283336, 2024-09-11T06:51:17.845 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:17.845 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:17.845 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.845 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:17.845 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:17.845 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:17.845 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 87 2024-09-11T06:51:17.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.845 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:17.845 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.846 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:17.846 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:51:17.846 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.846 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.846 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.846 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:17.846 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:51:17.846 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.846 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.846 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.846 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:17.846 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271768+0000", 2024-09-11T06:51:17.846 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7599999999999996e-07 2024-09-11T06:51:17.846 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.847 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.847 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:17.847 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271860+0000", 2024-09-11T06:51:17.847 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2639999999999994e-05 2024-09-11T06:51:17.847 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.847 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.847 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:17.847 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271873+0000", 2024-09-11T06:51:17.847 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3139000000000001e-05 2024-09-11T06:51:17.847 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.847 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.847 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.847 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271882+0000", 2024-09-11T06:51:17.847 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2369999999999992e-06 2024-09-11T06:51:17.847 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.847 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.848 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:17.848 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277150+0000", 2024-09-11T06:51:17.848 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052684419999999999 2024-09-11T06:51:17.848 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.848 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.848 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:17.848 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277273+0000", 2024-09-11T06:51:17.848 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012278899999999999 2024-09-11T06:51:17.848 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.848 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.848 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:17.848 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277466+0000", 2024-09-11T06:51:17.848 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193578 2024-09-11T06:51:17.849 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.849 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.849 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.849 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.280808+0000", 2024-09-11T06:51:17.849 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003341971 2024-09-11T06:51:17.849 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.849 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.849 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:17.849 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284419+0000", 2024-09-11T06:51:17.849 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003610982 2024-09-11T06:51:17.849 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.849 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.849 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:17.849 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284531+0000", 2024-09-11T06:51:17.850 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011121299999999999 2024-09-11T06:51:17.850 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.850 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.850 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:17.850 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284600+0000", 2024-09-11T06:51:17.850 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9592999999999996e-05 2024-09-11T06:51:17.850 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.850 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:17.850 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.850 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.850 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.850 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:17.850 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:51:17.851 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 131.57035053600001, 2024-09-11T06:51:17.851 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283314, 2024-09-11T06:51:17.851 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:17.851 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:17.851 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.851 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:17.851 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:17.851 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:17.851 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 88 2024-09-11T06:51:17.851 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.851 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:17.851 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.851 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:17.852 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:51:17.852 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.852 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.852 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.852 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:17.852 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:51:17.852 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.852 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.852 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.852 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:17.852 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271898+0000", 2024-09-11T06:51:17.852 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2999999999999999e-07 2024-09-11T06:51:17.852 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.853 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.853 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:17.853 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271904+0000", 2024-09-11T06:51:17.853 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.767e-06 2024-09-11T06:51:17.853 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.853 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.853 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:17.853 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271913+0000", 2024-09-11T06:51:17.853 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7200000000000001e-06 2024-09-11T06:51:17.853 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.853 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.853 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.853 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271920+0000", 2024-09-11T06:51:17.853 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7100000000000001e-06 2024-09-11T06:51:17.854 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.854 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.854 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:17.854 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278684+0000", 2024-09-11T06:51:17.854 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0067642650000000002 2024-09-11T06:51:17.854 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.854 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.854 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:17.854 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278838+0000", 2024-09-11T06:51:17.854 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015371499999999999 2024-09-11T06:51:17.854 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.854 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.854 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:17.855 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.279040+0000", 2024-09-11T06:51:17.855 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020207499999999999 2024-09-11T06:51:17.855 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.855 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.855 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.855 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.281448+0000", 2024-09-11T06:51:17.855 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002407686 2024-09-11T06:51:17.855 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.855 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.855 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:17.855 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284615+0000", 2024-09-11T06:51:17.855 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031675499999999999 2024-09-11T06:51:17.855 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.855 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284682+0000", 2024-09-11T06:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7032000000000005e-05 2024-09-11T06:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284730+0000", 2024-09-11T06:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7790000000000002e-05 2024-09-11T06:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 131.401501336, 2024-09-11T06:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014944941, 2024-09-11T06:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 111 2024-09-11T06:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.858 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.858 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:17.858 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:51:17.858 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.858 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.858 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.858 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:17.858 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440748+0000", 2024-09-11T06:51:17.858 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:51:17.858 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.858 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.858 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:17.858 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440760+0000", 2024-09-11T06:51:17.858 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1897e-05 2024-09-11T06:51:17.858 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.859 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.859 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:17.859 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440782+0000", 2024-09-11T06:51:17.859 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2552999999999999e-05 2024-09-11T06:51:17.859 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.859 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.859 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.859 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440793+0000", 2024-09-11T06:51:17.859 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0643e-05 2024-09-11T06:51:17.859 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.859 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.859 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:17.859 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449755+0000", 2024-09-11T06:51:17.859 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089622620000000004 2024-09-11T06:51:17.859 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.860 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.860 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:17.860 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449918+0000", 2024-09-11T06:51:17.860 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163086 2024-09-11T06:51:17.860 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.860 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.860 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:17.860 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450150+0000", 2024-09-11T06:51:17.860 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023204500000000001 2024-09-11T06:51:17.860 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.860 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.860 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:17.860 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454533+0000", 2024-09-11T06:51:17.860 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0043831349999999998 2024-09-11T06:51:17.860 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.861 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.861 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.861 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455511+0000", 2024-09-11T06:51:17.861 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0009776050000000001 2024-09-11T06:51:17.861 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.861 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.861 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:17.861 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455597+0000", 2024-09-11T06:51:17.861 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6446000000000005e-05 2024-09-11T06:51:17.861 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.861 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.861 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:17.861 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455691+0000", 2024-09-11T06:51:17.861 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3832000000000006e-05 2024-09-11T06:51:17.861 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.861 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:17.862 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.862 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.862 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.862 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:17.862 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:51:17.862 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 131.401431573, 2024-09-11T06:51:17.862 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018885411000000001, 2024-09-11T06:51:17.862 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:17.862 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:17.862 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.862 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:17.862 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:17.862 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:17.862 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 112 2024-09-11T06:51:17.862 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.863 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:17.863 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.863 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:17.863 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:51:17.863 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.863 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.863 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.863 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:17.863 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:51:17.863 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.863 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.863 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.863 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:17.863 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440817+0000", 2024-09-11T06:51:17.863 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:51:17.864 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.864 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.864 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:17.864 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441003+0000", 2024-09-11T06:51:17.864 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018661200000000001 2024-09-11T06:51:17.864 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.864 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.864 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:17.864 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441027+0000", 2024-09-11T06:51:17.864 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3920000000000001e-05 2024-09-11T06:51:17.864 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.864 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.864 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.864 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441062+0000", 2024-09-11T06:51:17.864 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4663000000000003e-05 2024-09-11T06:51:17.864 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.865 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.865 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:17.865 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451981+0000", 2024-09-11T06:51:17.865 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010918926000000001 2024-09-11T06:51:17.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.865 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.865 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:17.865 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452111+0000", 2024-09-11T06:51:17.865 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129529 2024-09-11T06:51:17.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.865 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.865 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:17.865 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452311+0000", 2024-09-11T06:51:17.865 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020013799999999999 2024-09-11T06:51:17.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.866 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.866 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455809+0000", 2024-09-11T06:51:17.866 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003498443 2024-09-11T06:51:17.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.866 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:17.866 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459566+0000", 2024-09-11T06:51:17.866 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003757271 2024-09-11T06:51:17.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.866 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:17.866 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459640+0000", 2024-09-11T06:51:17.866 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3949000000000006e-05 2024-09-11T06:51:17.867 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.867 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.867 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:17.867 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459701+0000", 2024-09-11T06:51:17.867 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1117000000000005e-05 2024-09-11T06:51:17.867 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.867 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:17.867 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.867 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.867 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.867 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:17.867 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:51:17.867 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 131.40115272200001, 2024-09-11T06:51:17.867 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018760915, 2024-09-11T06:51:17.868 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:17.868 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:17.868 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.868 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:17.868 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:17.868 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:17.868 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 113 2024-09-11T06:51:17.868 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.868 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:17.868 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.868 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:17.868 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:51:17.869 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.869 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.869 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.869 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:17.869 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:51:17.869 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.869 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.869 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.869 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:17.869 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441096+0000", 2024-09-11T06:51:17.869 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4130000000000001e-06 2024-09-11T06:51:17.869 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.869 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.869 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:17.870 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441110+0000", 2024-09-11T06:51:17.870 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3900000000000001e-05 2024-09-11T06:51:17.870 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.870 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.870 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:17.870 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441131+0000", 2024-09-11T06:51:17.870 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0636999999999999e-05 2024-09-11T06:51:17.870 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.870 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.870 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.870 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441147+0000", 2024-09-11T06:51:17.870 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6512999999999999e-05 2024-09-11T06:51:17.870 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.870 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.871 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:17.871 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453274+0000", 2024-09-11T06:51:17.871 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012126764999999999 2024-09-11T06:51:17.871 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.871 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.871 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:17.871 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453498+0000", 2024-09-11T06:51:17.871 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000223568 2024-09-11T06:51:17.871 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.871 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.871 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:17.871 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453872+0000", 2024-09-11T06:51:17.871 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037467099999999998 2024-09-11T06:51:17.872 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.872 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.872 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.872 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458709+0000", 2024-09-11T06:51:17.872 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0048365680000000003 2024-09-11T06:51:17.872 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.872 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.872 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:17.872 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459716+0000", 2024-09-11T06:51:17.872 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010075240000000001 2024-09-11T06:51:17.872 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.872 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.872 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:17.872 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459779+0000", 2024-09-11T06:51:17.872 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2336999999999996e-05 2024-09-11T06:51:17.873 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.873 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.873 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:17.873 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459856+0000", 2024-09-11T06:51:17.873 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7019000000000002e-05 2024-09-11T06:51:17.873 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.873 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:17.873 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.873 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.873 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.873 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:17.873 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:51:17.873 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 131.401075983, 2024-09-11T06:51:17.873 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018963114, 2024-09-11T06:51:17.873 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:17.874 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:17.874 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.874 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:17.874 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:17.874 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:17.874 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 114 2024-09-11T06:51:17.874 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.874 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:17.874 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.874 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:17.874 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:51:17.874 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.874 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.874 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.875 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:17.875 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:51:17.875 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.875 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.875 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.875 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:17.875 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:51:17.875 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9299999999999996e-07 2024-09-11T06:51:17.875 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.875 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.875 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:17.876 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441180+0000", 2024-09-11T06:51:17.876 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7200000000000006e-06 2024-09-11T06:51:17.876 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.876 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.876 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:17.876 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441265+0000", 2024-09-11T06:51:17.876 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4753000000000007e-05 2024-09-11T06:51:17.876 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.876 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.876 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.876 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441288+0000", 2024-09-11T06:51:17.876 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2860000000000001e-05 2024-09-11T06:51:17.877 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.877 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.877 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:17.877 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454585+0000", 2024-09-11T06:51:17.877 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013297286 2024-09-11T06:51:17.877 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.877 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.877 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:17.877 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454714+0000", 2024-09-11T06:51:17.877 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000128465 2024-09-11T06:51:17.877 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.877 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.877 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:17.877 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454909+0000", 2024-09-11T06:51:17.878 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019522499999999999 2024-09-11T06:51:17.878 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.878 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.878 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:17.878 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459868+0000", 2024-09-11T06:51:17.878 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0049591239999999997 2024-09-11T06:51:17.879 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.879 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.879 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.879 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459988+0000", 2024-09-11T06:51:17.879 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000120532 2024-09-11T06:51:17.879 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.879 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.879 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:17.879 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460071+0000", 2024-09-11T06:51:17.879 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2256000000000007e-05 2024-09-11T06:51:17.879 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.879 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.880 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:17.880 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460135+0000", 2024-09-11T06:51:17.880 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3999999999999997e-05 2024-09-11T06:51:17.880 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.880 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:17.880 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.880 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.880 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.880 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:17.880 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:51:17.880 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 131.400918354, 2024-09-11T06:51:17.880 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.019036849000000002, 2024-09-11T06:51:17.880 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:17.881 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:17.881 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.881 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:17.881 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:17.881 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:17.881 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 115 2024-09-11T06:51:17.881 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.881 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:17.881 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.881 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:17.881 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:51:17.881 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.881 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.881 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.882 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:17.882 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:51:17.882 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.882 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.882 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.882 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:17.882 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441330+0000", 2024-09-11T06:51:17.882 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1930000000000001e-06 2024-09-11T06:51:17.882 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.882 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.882 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:17.882 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441544+0000", 2024-09-11T06:51:17.882 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021405799999999999 2024-09-11T06:51:17.882 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.883 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.883 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:17.883 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441572+0000", 2024-09-11T06:51:17.883 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7317e-05 2024-09-11T06:51:17.883 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.883 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.883 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.883 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441593+0000", 2024-09-11T06:51:17.883 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1702999999999999e-05 2024-09-11T06:51:17.883 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.883 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.883 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:17.883 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455891+0000", 2024-09-11T06:51:17.883 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01429708 2024-09-11T06:51:17.884 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.884 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.884 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:17.884 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456046+0000", 2024-09-11T06:51:17.884 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000155682 2024-09-11T06:51:17.884 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.884 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.884 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:17.884 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456242+0000", 2024-09-11T06:51:17.884 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000195692 2024-09-11T06:51:17.884 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.884 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.884 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:17.884 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459876+0000", 2024-09-11T06:51:17.885 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036336480000000002 2024-09-11T06:51:17.885 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.885 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.885 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.885 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460245+0000", 2024-09-11T06:51:17.885 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036901700000000002 2024-09-11T06:51:17.885 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.885 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.885 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:17.885 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460316+0000", 2024-09-11T06:51:17.885 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1846000000000003e-05 2024-09-11T06:51:17.885 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.885 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.886 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:17.886 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460366+0000", 2024-09-11T06:51:17.886 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9613000000000001e-05 2024-09-11T06:51:17.886 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.886 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:17.886 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.886 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.886 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.886 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:17.886 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:51:17.886 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 131.40059627400001, 2024-09-11T06:51:17.886 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01904432, 2024-09-11T06:51:17.886 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:17.886 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:17.887 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.887 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:17.887 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:17.887 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:17.887 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 116 2024-09-11T06:51:17.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.887 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:17.887 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.887 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:17.887 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:51:17.887 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.887 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.887 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:17.888 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:51:17.888 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.888 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.888 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.888 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:17.888 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441653+0000", 2024-09-11T06:51:17.888 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:51:17.888 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.888 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.888 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:17.888 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441665+0000", 2024-09-11T06:51:17.888 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1653e-05 2024-09-11T06:51:17.888 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.888 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.889 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:17.889 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441684+0000", 2024-09-11T06:51:17.889 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9490000000000001e-05 2024-09-11T06:51:17.889 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.889 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.889 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.889 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441699+0000", 2024-09-11T06:51:17.889 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5044e-05 2024-09-11T06:51:17.889 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.889 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.889 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:17.889 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457297+0000", 2024-09-11T06:51:17.890 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015598241000000001 2024-09-11T06:51:17.890 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.890 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.890 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:17.890 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457577+0000", 2024-09-11T06:51:17.890 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027930799999999998 2024-09-11T06:51:17.890 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.890 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.890 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:17.890 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457957+0000", 2024-09-11T06:51:17.890 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00038088 2024-09-11T06:51:17.890 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.890 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.890 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.891 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460432+0000", 2024-09-11T06:51:17.891 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024743349999999998 2024-09-11T06:51:17.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.891 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.891 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:17.891 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460584+0000", 2024-09-11T06:51:17.891 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015232900000000001 2024-09-11T06:51:17.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.891 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.891 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:17.891 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460646+0000", 2024-09-11T06:51:17.891 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1883000000000004e-05 2024-09-11T06:51:17.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.891 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.892 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:17.892 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460696+0000", 2024-09-11T06:51:17.892 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9589999999999998e-05 2024-09-11T06:51:17.892 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.892 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:17.892 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.892 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.892 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.892 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:17.892 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:51:17.892 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 131.40052420800001, 2024-09-11T06:51:17.892 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.021834473, 2024-09-11T06:51:17.892 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:17.893 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:17.893 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.893 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:17.893 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:17.893 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:17.893 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 117 2024-09-11T06:51:17.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.893 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:17.893 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.893 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:17.893 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:51:17.893 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.894 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.894 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:17.894 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:51:17.894 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.894 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.894 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:17.894 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441725+0000", 2024-09-11T06:51:17.894 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5069999999999999e-06 2024-09-11T06:51:17.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.894 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.894 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:17.894 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441738+0000", 2024-09-11T06:51:17.895 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2813000000000001e-05 2024-09-11T06:51:17.895 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.895 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.895 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:17.895 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441759+0000", 2024-09-11T06:51:17.895 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1246999999999998e-05 2024-09-11T06:51:17.895 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.895 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.895 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.895 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441779+0000", 2024-09-11T06:51:17.895 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0392999999999999e-05 2024-09-11T06:51:17.895 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.895 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.896 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:17.896 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458804+0000", 2024-09-11T06:51:17.896 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.017024182999999998 2024-09-11T06:51:17.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.896 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.896 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:17.896 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458977+0000", 2024-09-11T06:51:17.896 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000173926 2024-09-11T06:51:17.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.896 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.896 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:17.896 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459170+0000", 2024-09-11T06:51:17.896 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000192184 2024-09-11T06:51:17.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.897 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.897 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.897 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.462300+0000", 2024-09-11T06:51:17.897 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031304969999999999 2024-09-11T06:51:17.897 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.897 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.897 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:17.897 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463408+0000", 2024-09-11T06:51:17.897 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011077839999999999 2024-09-11T06:51:17.897 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.897 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.897 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:17.897 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463497+0000", 2024-09-11T06:51:17.897 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8828999999999999e-05 2024-09-11T06:51:17.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.898 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.898 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:17.898 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463558+0000", 2024-09-11T06:51:17.898 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1110000000000003e-05 2024-09-11T06:51:17.898 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.898 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:17.898 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.898 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.898 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:17.898 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:51:17.898 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 131.02388270599999, 2024-09-11T06:51:17.898 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013136297, 2024-09-11T06:51:17.898 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:17.899 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:17.899 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.899 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:17.899 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:17.899 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:17.899 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 138 2024-09-11T06:51:17.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.899 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:17.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.899 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:17.899 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:51:17.899 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.899 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:17.899 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:51:17.900 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.900 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.900 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.900 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:17.900 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:51:17.900 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.89e-07 2024-09-11T06:51:17.900 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.900 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.900 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:17.900 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818369+0000", 2024-09-11T06:51:17.900 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4170000000000001e-06 2024-09-11T06:51:17.900 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.900 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.900 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:17.901 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818376+0000", 2024-09-11T06:51:17.901 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4270000000000002e-06 2024-09-11T06:51:17.901 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.901 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.901 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.901 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818383+0000", 2024-09-11T06:51:17.901 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3060000000000003e-06 2024-09-11T06:51:17.901 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.901 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.901 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:17.901 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824783+0000", 2024-09-11T06:51:17.901 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064003020000000001 2024-09-11T06:51:17.901 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.901 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.902 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:17.902 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825013+0000", 2024-09-11T06:51:17.902 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023039499999999999 2024-09-11T06:51:17.902 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.902 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.902 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:17.902 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825484+0000", 2024-09-11T06:51:17.902 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000470562 2024-09-11T06:51:17.902 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.902 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.902 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.902 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828633+0000", 2024-09-11T06:51:17.902 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031496689999999999 2024-09-11T06:51:17.902 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.903 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.903 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:17.903 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831413+0000", 2024-09-11T06:51:17.903 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027793219999999999 2024-09-11T06:51:17.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.903 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.903 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:17.903 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831450+0000", 2024-09-11T06:51:17.903 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7370999999999998e-05 2024-09-11T06:51:17.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.903 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.903 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:17.903 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831501+0000", 2024-09-11T06:51:17.903 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1036999999999999e-05 2024-09-11T06:51:17.903 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.903 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:17.904 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.904 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.904 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.904 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:17.904 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:51:17.904 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 131.02385822599999, 2024-09-11T06:51:17.904 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013223011, 2024-09-11T06:51:17.904 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:17.904 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:17.904 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.904 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:17.904 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:17.904 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:17.904 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 139 2024-09-11T06:51:17.904 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.905 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:17.905 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.905 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:17.905 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:51:17.905 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.905 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.905 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:17.905 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:51:17.905 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.905 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.905 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:17.905 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818390+0000", 2024-09-11T06:51:17.905 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5600000000000001e-07 2024-09-11T06:51:17.906 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.906 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.906 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:17.906 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818394+0000", 2024-09-11T06:51:17.906 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5929999999999999e-06 2024-09-11T06:51:17.906 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.906 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.906 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:17.906 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818399+0000", 2024-09-11T06:51:17.906 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7740000000000004e-06 2024-09-11T06:51:17.906 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.906 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.906 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.907 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818412+0000", 2024-09-11T06:51:17.907 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2625e-05 2024-09-11T06:51:17.907 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.907 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.907 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:17.907 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826205+0000", 2024-09-11T06:51:17.907 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0077930730000000002 2024-09-11T06:51:17.907 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.907 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.907 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:17.907 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826281+0000", 2024-09-11T06:51:17.907 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5876000000000003e-05 2024-09-11T06:51:17.907 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.907 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.907 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:17.908 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826384+0000", 2024-09-11T06:51:17.908 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103838 2024-09-11T06:51:17.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.908 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.908 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.908 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830642+0000", 2024-09-11T06:51:17.908 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004257083 2024-09-11T06:51:17.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.908 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.908 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:17.908 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831509+0000", 2024-09-11T06:51:17.908 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086730900000000003 2024-09-11T06:51:17.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.908 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.909 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:17.909 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831547+0000", 2024-09-11T06:51:17.909 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7793000000000002e-05 2024-09-11T06:51:17.909 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.909 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.909 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:17.909 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831612+0000", 2024-09-11T06:51:17.909 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5691000000000005e-05 2024-09-11T06:51:17.909 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.909 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:17.909 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.909 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.909 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.909 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:17.910 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:51:17.910 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 131.02382953200001, 2024-09-11T06:51:17.910 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016098024999999998, 2024-09-11T06:51:17.910 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:17.910 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:17.910 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.910 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:17.910 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:17.910 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:17.910 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 140 2024-09-11T06:51:17.910 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.910 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:17.910 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.910 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:17.911 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:51:17.911 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.911 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.911 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.911 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:17.911 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:51:17.911 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.911 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.911 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.911 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:17.911 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:51:17.911 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5199999999999998e-07 2024-09-11T06:51:17.911 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.911 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.912 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:17.912 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818479+0000", 2024-09-11T06:51:17.912 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0368000000000002e-05 2024-09-11T06:51:17.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.912 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.912 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:17.912 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818487+0000", 2024-09-11T06:51:17.912 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2339999999999994e-06 2024-09-11T06:51:17.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.912 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.912 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.912 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818491+0000", 2024-09-11T06:51:17.912 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4329999999999997e-06 2024-09-11T06:51:17.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.913 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.913 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:17.913 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828080+0000", 2024-09-11T06:51:17.913 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0095890449999999992 2024-09-11T06:51:17.913 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.913 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.913 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:17.913 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828130+0000", 2024-09-11T06:51:17.913 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9239000000000001e-05 2024-09-11T06:51:17.913 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.913 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.913 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:17.913 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828208+0000", 2024-09-11T06:51:17.914 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.852e-05 2024-09-11T06:51:17.914 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.914 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.914 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:17.914 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831625+0000", 2024-09-11T06:51:17.914 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034169410000000002 2024-09-11T06:51:17.914 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.914 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.914 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.914 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834411+0000", 2024-09-11T06:51:17.914 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002786111 2024-09-11T06:51:17.914 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.914 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.914 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:17.915 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834457+0000", 2024-09-11T06:51:17.915 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6041000000000002e-05 2024-09-11T06:51:17.915 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.915 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.915 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:17.915 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834516+0000", 2024-09-11T06:51:17.915 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8740999999999999e-05 2024-09-11T06:51:17.915 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.915 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:17.915 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.915 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.915 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.915 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:17.915 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:51:17.915 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 131.02374846500001, 2024-09-11T06:51:17.915 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016588772000000002, 2024-09-11T06:51:17.916 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:17.916 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:17.916 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.916 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:17.916 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:17.916 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:17.916 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 141 2024-09-11T06:51:17.916 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.916 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:17.916 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.916 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:17.916 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:51:17.916 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.917 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.917 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.917 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:17.917 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:51:17.917 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.917 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.917 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.917 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:17.917 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818500+0000", 2024-09-11T06:51:17.917 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0800000000000005e-07 2024-09-11T06:51:17.917 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.917 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.917 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:17.917 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818505+0000", 2024-09-11T06:51:17.918 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2429999999999996e-06 2024-09-11T06:51:17.918 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.918 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.918 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:17.918 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818510+0000", 2024-09-11T06:51:17.918 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9339999999999997e-06 2024-09-11T06:51:17.918 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.918 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.918 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.918 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818514+0000", 2024-09-11T06:51:17.918 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7270000000000001e-06 2024-09-11T06:51:17.918 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.918 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.918 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:17.919 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828711+0000", 2024-09-11T06:51:17.919 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010197031 2024-09-11T06:51:17.919 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.919 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.919 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:17.919 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828853+0000", 2024-09-11T06:51:17.919 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014201200000000001 2024-09-11T06:51:17.919 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.919 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.919 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:17.919 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.829115+0000", 2024-09-11T06:51:17.919 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026257500000000003 2024-09-11T06:51:17.919 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.919 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.920 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:17.920 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831629+0000", 2024-09-11T06:51:17.920 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025138169999999998 2024-09-11T06:51:17.920 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.920 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.920 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.920 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834991+0000", 2024-09-11T06:51:17.920 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033624459999999998 2024-09-11T06:51:17.920 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.920 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.920 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:17.920 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835032+0000", 2024-09-11T06:51:17.920 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1001000000000003e-05 2024-09-11T06:51:17.921 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.921 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.921 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:17.921 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835088+0000", 2024-09-11T06:51:17.921 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5478e-05 2024-09-11T06:51:17.921 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.921 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:17.921 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.921 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.921 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.921 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5235.0:1 2.5 2:b2cc3336:::rbd_id.testrbdnbd21607:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-11T06:51:17.921 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:51:17.921 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 43.282946443, 2024-09-11T06:51:17.922 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.68353462499999995, 2024-09-11T06:51:17.922 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:17.922 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:17.922 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:17.922 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:17.922 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5235", 2024-09-11T06:51:17.922 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2008977164", 2024-09-11T06:51:17.922 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:51:17.922 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.922 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:17.922 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.922 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:17.922 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:51:17.922 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.922 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.922 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.923 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:17.923 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:51:17.923 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.923 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.923 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.923 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:17.923 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559303+0000", 2024-09-11T06:51:17.923 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1830000000000001e-06 2024-09-11T06:51:17.923 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.923 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.923 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:17.923 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559314+0000", 2024-09-11T06:51:17.923 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0396e-05 2024-09-11T06:51:17.923 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.924 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.924 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:17.924 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559335+0000", 2024-09-11T06:51:17.924 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1183e-05 2024-09-11T06:51:17.924 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.924 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.924 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.924 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559354+0000", 2024-09-11T06:51:17.924 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9225999999999999e-05 2024-09-11T06:51:17.924 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.924 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.924 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:17.924 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559522+0000", 2024-09-11T06:51:17.924 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000168021 2024-09-11T06:51:17.924 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.925 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.925 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:51:17.925 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559543+0000", 2024-09-11T06:51:17.925 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0616999999999999e-05 2024-09-11T06:51:17.925 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.925 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.925 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:17.925 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.238584+0000", 2024-09-11T06:51:17.925 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.67904107499999999 2024-09-11T06:51:17.925 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.925 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.925 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for missing object", 2024-09-11T06:51:17.925 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.238933+0000", 2024-09-11T06:51:17.925 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034879600000000001 2024-09-11T06:51:17.926 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.926 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.926 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:17.926 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.241968+0000", 2024-09-11T06:51:17.926 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030350830000000001 2024-09-11T06:51:17.926 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.926 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.926 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:17.926 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.242281+0000", 2024-09-11T06:51:17.926 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00031299499999999999 2024-09-11T06:51:17.926 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.926 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.926 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:17.926 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.242836+0000", 2024-09-11T06:51:17.927 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00055504999999999997 2024-09-11T06:51:17.927 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.927 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:17.927 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.927 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.927 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.927 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5235.0:18 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 0~1024 [fadvise_sequential]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T06:51:17.927 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:51:17.927 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 42.580454004000003, 2024-09-11T06:51:17.927 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.048993760999999997, 2024-09-11T06:51:17.927 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:17.927 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:17.927 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.927 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:17.928 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5235", 2024-09-11T06:51:17.928 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2008977164", 2024-09-11T06:51:17.928 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 18 2024-09-11T06:51:17.928 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.928 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:17.928 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.928 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:17.928 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:51:17.928 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.928 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.928 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.928 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:17.928 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:51:17.928 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.929 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.929 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.929 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:17.929 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261797+0000", 2024-09-11T06:51:17.929 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2059999999999999e-06 2024-09-11T06:51:17.929 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.929 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.929 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:17.929 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261812+0000", 2024-09-11T06:51:17.929 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5159e-05 2024-09-11T06:51:17.929 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.929 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.929 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:17.929 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261852+0000", 2024-09-11T06:51:17.930 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.9906e-05 2024-09-11T06:51:17.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.930 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.930 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.930 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261865+0000", 2024-09-11T06:51:17.930 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.312e-05 2024-09-11T06:51:17.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.930 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.930 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:17.930 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.263166+0000", 2024-09-11T06:51:17.930 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001301134 2024-09-11T06:51:17.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.930 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.931 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for missing object", 2024-09-11T06:51:17.931 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.263529+0000", 2024-09-11T06:51:17.931 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036279900000000002 2024-09-11T06:51:17.931 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.931 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.931 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:17.931 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.300567+0000", 2024-09-11T06:51:17.931 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.037038326000000003 2024-09-11T06:51:17.931 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.931 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.931 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:17.931 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.300800+0000", 2024-09-11T06:51:17.931 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023323300000000001 2024-09-11T06:51:17.931 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.931 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.932 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:17.932 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.306709+0000", 2024-09-11T06:51:17.932 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0059087690000000003 2024-09-11T06:51:17.932 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.932 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.932 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:17.932 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.306850+0000", 2024-09-11T06:51:17.932 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014106200000000001 2024-09-11T06:51:17.932 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.932 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.932 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:17.932 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.307121+0000", 2024-09-11T06:51:17.932 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027083599999999999 2024-09-11T06:51:17.932 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.932 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.933 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.933 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.308809+0000", 2024-09-11T06:51:17.933 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0016880759999999999 2024-09-11T06:51:17.933 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.933 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.933 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:17.933 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310382+0000", 2024-09-11T06:51:17.933 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0015732840000000001 2024-09-11T06:51:17.933 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.933 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.933 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:17.933 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310501+0000", 2024-09-11T06:51:17.933 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000118276 2024-09-11T06:51:17.933 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.933 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.934 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:17.934 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310787+0000", 2024-09-11T06:51:17.934 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00028657500000000001 2024-09-11T06:51:17.934 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.934 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:17.934 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.934 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.934 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.934 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:70 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 393216~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:17.934 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596118+0000", 2024-09-11T06:51:17.934 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 29.246129602, 2024-09-11T06:51:17.934 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012279145, 2024-09-11T06:51:17.934 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:17.934 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:17.935 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.935 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:17.935 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:17.935 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:17.935 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 70 2024-09-11T06:51:17.935 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.935 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:17.935 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.935 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:17.935 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596118+0000", 2024-09-11T06:51:17.935 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.935 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.935 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.935 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:17.935 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596118+0000", 2024-09-11T06:51:17.935 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.936 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.936 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.936 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:17.936 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596119+0000", 2024-09-11T06:51:17.936 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0299999999999997e-07 2024-09-11T06:51:17.936 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.936 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.936 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:17.936 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596238+0000", 2024-09-11T06:51:17.936 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000118798 2024-09-11T06:51:17.936 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.936 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.936 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:17.936 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596250+0000", 2024-09-11T06:51:17.937 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2775999999999999e-05 2024-09-11T06:51:17.937 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.937 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.937 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.937 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596259+0000", 2024-09-11T06:51:17.937 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3100000000000001e-06 2024-09-11T06:51:17.937 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.937 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.937 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:17.937 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.603389+0000", 2024-09-11T06:51:17.937 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0071300759999999999 2024-09-11T06:51:17.937 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.937 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.937 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:17.938 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.603540+0000", 2024-09-11T06:51:17.938 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015111700000000001 2024-09-11T06:51:17.938 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.938 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.938 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:17.938 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.603766+0000", 2024-09-11T06:51:17.938 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022570899999999999 2024-09-11T06:51:17.938 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.938 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.938 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.938 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.606676+0000", 2024-09-11T06:51:17.938 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0029106380000000001 2024-09-11T06:51:17.938 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.938 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.939 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:17.939 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608173+0000", 2024-09-11T06:51:17.939 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014967629999999999 2024-09-11T06:51:17.939 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.939 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.939 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:17.939 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608277+0000", 2024-09-11T06:51:17.939 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000104398 2024-09-11T06:51:17.939 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.939 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.939 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:17.939 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608397+0000", 2024-09-11T06:51:17.939 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000119657 2024-09-11T06:51:17.939 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.940 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:17.940 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.940 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.940 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.940 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:71 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 524288~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:17.940 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596272+0000", 2024-09-11T06:51:17.940 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 29.245975348999998, 2024-09-11T06:51:17.940 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013390681999999999, 2024-09-11T06:51:17.940 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:17.940 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:17.940 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.940 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:17.940 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:17.941 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:17.941 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 71 2024-09-11T06:51:17.941 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.941 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:17.941 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.941 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:17.941 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596272+0000", 2024-09-11T06:51:17.941 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.941 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.941 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.941 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:17.941 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596272+0000", 2024-09-11T06:51:17.941 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.941 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.942 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.942 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:17.942 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596273+0000", 2024-09-11T06:51:17.942 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.9999999999999996e-07 2024-09-11T06:51:17.942 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.942 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.942 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:17.942 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596395+0000", 2024-09-11T06:51:17.942 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012204799999999999 2024-09-11T06:51:17.942 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.942 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.942 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:17.942 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596411+0000", 2024-09-11T06:51:17.942 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5746000000000001e-05 2024-09-11T06:51:17.943 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.943 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.943 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.943 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596419+0000", 2024-09-11T06:51:17.943 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2600000000000005e-06 2024-09-11T06:51:17.943 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.943 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.943 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:17.943 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.605311+0000", 2024-09-11T06:51:17.943 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0088915179999999993 2024-09-11T06:51:17.943 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.943 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.943 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:17.943 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.605436+0000", 2024-09-11T06:51:17.943 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000125203 2024-09-11T06:51:17.944 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.944 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.944 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:17.944 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.605625+0000", 2024-09-11T06:51:17.944 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018961299999999999 2024-09-11T06:51:17.944 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.944 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.944 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.944 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609477+0000", 2024-09-11T06:51:17.944 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0038513549999999999 2024-09-11T06:51:17.944 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.944 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.944 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:17.944 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609499+0000", 2024-09-11T06:51:17.944 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2215e-05 2024-09-11T06:51:17.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.945 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.945 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:17.945 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609589+0000", 2024-09-11T06:51:17.945 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0029000000000001e-05 2024-09-11T06:51:17.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.945 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.945 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:17.945 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609663+0000", 2024-09-11T06:51:17.945 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3894999999999996e-05 2024-09-11T06:51:17.945 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.945 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:17.945 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.946 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.946 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:72 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 655360~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:17.946 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596432+0000", 2024-09-11T06:51:17.946 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 29.245815862000001, 2024-09-11T06:51:17.946 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01414727, 2024-09-11T06:51:17.946 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:17.946 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:17.946 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.946 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:17.946 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:17.946 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:17.946 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 72 2024-09-11T06:51:17.946 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.946 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:17.947 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.947 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:17.947 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596432+0000", 2024-09-11T06:51:17.947 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.947 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.947 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.947 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:17.947 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596432+0000", 2024-09-11T06:51:17.947 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.947 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.947 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.947 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:17.947 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596433+0000", 2024-09-11T06:51:17.947 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1999999999999998e-07 2024-09-11T06:51:17.947 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.948 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.948 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:17.948 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596556+0000", 2024-09-11T06:51:17.948 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012379000000000001 2024-09-11T06:51:17.948 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.948 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.948 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:17.948 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596571+0000", 2024-09-11T06:51:17.948 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4749999999999999e-05 2024-09-11T06:51:17.948 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.948 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.948 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.948 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596579+0000", 2024-09-11T06:51:17.948 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2500000000000006e-06 2024-09-11T06:51:17.948 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.949 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.949 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:17.949 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.606784+0000", 2024-09-11T06:51:17.949 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010204757 2024-09-11T06:51:17.949 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.949 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.949 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:17.949 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.606950+0000", 2024-09-11T06:51:17.949 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000166229 2024-09-11T06:51:17.949 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.949 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.949 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:17.949 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.607176+0000", 2024-09-11T06:51:17.949 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022549899999999999 2024-09-11T06:51:17.949 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.950 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.950 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:17.950 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610363+0000", 2024-09-11T06:51:17.950 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031874690000000001 2024-09-11T06:51:17.950 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.950 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.950 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.950 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610490+0000", 2024-09-11T06:51:17.950 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012639899999999999 2024-09-11T06:51:17.950 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.950 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.950 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:17.950 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610529+0000", 2024-09-11T06:51:17.950 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.8936000000000001e-05 2024-09-11T06:51:17.950 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.951 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.951 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:17.951 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610579+0000", 2024-09-11T06:51:17.951 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0371000000000003e-05 2024-09-11T06:51:17.951 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.951 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:17.951 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.951 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.951 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.951 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:73 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 786432~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:17.951 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596592+0000", 2024-09-11T06:51:17.951 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 29.245655609, 2024-09-11T06:51:17.951 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014914779, 2024-09-11T06:51:17.951 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:17.951 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:17.952 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.952 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:17.952 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:17.952 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:17.952 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 73 2024-09-11T06:51:17.952 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.952 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:17.952 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.952 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:17.952 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596592+0000", 2024-09-11T06:51:17.952 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.952 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.952 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.952 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:17.952 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596592+0000", 2024-09-11T06:51:17.953 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.953 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.953 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.953 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:17.953 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596593+0000", 2024-09-11T06:51:17.953 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.0699999999999996e-07 2024-09-11T06:51:17.953 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.953 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.953 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:17.953 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596786+0000", 2024-09-11T06:51:17.953 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019278200000000001 2024-09-11T06:51:17.953 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.953 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.953 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:17.953 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596801+0000", 2024-09-11T06:51:17.954 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5690000000000001e-05 2024-09-11T06:51:17.954 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.956 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.956 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.956 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596815+0000", 2024-09-11T06:51:17.956 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3753e-05 2024-09-11T06:51:17.956 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.956 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.956 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:17.957 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608442+0000", 2024-09-11T06:51:17.957 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011627385 2024-09-11T06:51:17.957 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.957 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.957 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:17.957 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608570+0000", 2024-09-11T06:51:17.957 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012740399999999999 2024-09-11T06:51:17.957 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.957 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.957 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:17.957 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608754+0000", 2024-09-11T06:51:17.957 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018392899999999999 2024-09-11T06:51:17.957 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.957 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.957 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:17.958 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610370+0000", 2024-09-11T06:51:17.958 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001615998 2024-09-11T06:51:17.958 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.958 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.958 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.958 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611420+0000", 2024-09-11T06:51:17.958 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010502790000000001 2024-09-11T06:51:17.958 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.958 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.958 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:17.958 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611458+0000", 2024-09-11T06:51:17.958 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7749000000000001e-05 2024-09-11T06:51:17.958 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.959 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.959 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:17.959 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611507+0000", 2024-09-11T06:51:17.959 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9002999999999998e-05 2024-09-11T06:51:17.959 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.959 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:17.959 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.959 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.959 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.959 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:74 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 917504~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:17.959 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596849+0000", 2024-09-11T06:51:17.959 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 29.245398971, 2024-09-11T06:51:17.959 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015202963999999999, 2024-09-11T06:51:17.959 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:17.960 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:17.960 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:17.960 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:17.960 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:17.960 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:17.960 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 74 2024-09-11T06:51:17.960 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.960 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:17.960 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.960 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:17.960 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596849+0000", 2024-09-11T06:51:17.960 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.960 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.960 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.960 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:17.961 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596849+0000", 2024-09-11T06:51:17.961 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:17.961 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.961 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.961 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:17.961 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596850+0000", 2024-09-11T06:51:17.961 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3170000000000001e-06 2024-09-11T06:51:17.961 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.961 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.961 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:17.961 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.597118+0000", 2024-09-11T06:51:17.961 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026772999999999999 2024-09-11T06:51:17.961 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.961 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.962 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:17.962 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.597136+0000", 2024-09-11T06:51:17.962 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8467000000000001e-05 2024-09-11T06:51:17.962 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.962 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.962 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:17.962 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.597148+0000", 2024-09-11T06:51:17.962 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1486e-05 2024-09-11T06:51:17.962 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.962 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.962 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:17.962 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609714+0000", 2024-09-11T06:51:17.962 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012566124 2024-09-11T06:51:17.962 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.962 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.963 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:17.963 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609792+0000", 2024-09-11T06:51:17.963 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.8579999999999996e-05 2024-09-11T06:51:17.963 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.963 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.963 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:17.963 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609899+0000", 2024-09-11T06:51:17.963 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00010716499999999999 2024-09-11T06:51:17.963 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.963 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.963 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:17.963 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610865+0000", 2024-09-11T06:51:17.963 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00096540299999999999 2024-09-11T06:51:17.963 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.963 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.964 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:17.964 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611970+0000", 2024-09-11T06:51:17.964 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001105118 2024-09-11T06:51:17.964 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.964 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.964 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:17.964 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.612005+0000", 2024-09-11T06:51:17.964 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4789000000000002e-05 2024-09-11T06:51:17.964 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:17.964 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:17.964 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:17.964 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.612052+0000", 2024-09-11T06:51:17.964 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6785e-05 2024-09-11T06:51:17.964 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.965 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:17.965 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.965 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:17.965 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:17.965 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:17.965 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:51:17.967 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV= 2024-09-11T06:51:17.967 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd feature disable rbd/testrbdnbd21607 journaling 2024-09-11T06:51:17.967 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:51:17.968 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:17.961+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:17.972 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is unset 2024-09-11T06:51:17.977 INFO:teuthology.orchestra.run.smithi071.stderr:marked out osd.1. 2024-09-11T06:51:18.067 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:18.062+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:18.115 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:51:18.115 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:51:18.132 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:51:18.132 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:51:18.138 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:18.138 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:18.138 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:51:18.138 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:18.148 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:51:18.148 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:51:18.151 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:51:18.168 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:51:18.169 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:51:18.186 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:51:18.186 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:51:18.201 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:18.197+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:18.204 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:51:18.204 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:51:18.269 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:18.266+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:18.275 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:18.275 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:18.289 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:51:18.381 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:18.377+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:18.461 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:18.461 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:18.474 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:18.469+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:18.475 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:51:18.577 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:18.573+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:18.642 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:18.642 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:18.656 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:51:18.678 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:18.674+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:18.781 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:18.778+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:18.824 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:18.824 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:18.838 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:51:18.889 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:18.885+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:18.961 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:18.961 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:18.977 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:51:19.029 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:19.026+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:19.084 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:19.077+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:19.123 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:19.123 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:19.139 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:51:19.185 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:19.181+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:19.286 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:19.282+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:19.305 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:19.305 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:19.319 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:51:19.405 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:19.402+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:19.486 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:19.486 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:19.487 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:19.485+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:19.500 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:51:19.588 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:19.582+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:19.664 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:19.664 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:19.679 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:51:19.721 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:19.718+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:19.790 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:19.786+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:19.845 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:19.845 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:19.861 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:51:19.890 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:19.885+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:19.991 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:19.985+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:20.003 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T06:51:20.040 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:20.040 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:20.058 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:51:20.101 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:20.098+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:20.193 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:20.190+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:20.217 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:20.218 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:20.222 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:51:20.232 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:51:20.294 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:20.289+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:20.362 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:51:20.362 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:51:20.378 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:51:20.379 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:51:20.395 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:51:20.395 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:51:20.395 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:20.390+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:20.397 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:20.397 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:20.397 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:51:20.397 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:51:20.398 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:20.412 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:51:20.418 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:51:20.418 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:51:20.436 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:51:20.437 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:51:20.457 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:51:20.457 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:51:20.496 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:20.489+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:20.536 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:20.537 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:51:20.537 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:51:20.537 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:51:20.537 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.537 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:26 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 140250278007648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:20.537 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:51:20.537 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 49.805136165999997, 2024-09-11T06:51:20.537 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034645140000000001, 2024-09-11T06:51:20.537 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:20.537 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:20.537 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:20.537 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:20.537 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:20.538 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:20.538 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:51:20.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.538 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:20.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.538 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:20.538 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:51:20.538 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.538 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:20.538 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:51:20.538 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.539 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:20.539 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729634+0000", 2024-09-11T06:51:20.539 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0470000000000001e-06 2024-09-11T06:51:20.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.539 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:20.539 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729640+0000", 2024-09-11T06:51:20.539 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8860000000000001e-06 2024-09-11T06:51:20.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.539 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:20.539 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729654+0000", 2024-09-11T06:51:20.539 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:51:20.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.540 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:20.540 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729664+0000", 2024-09-11T06:51:20.540 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.594e-06 2024-09-11T06:51:20.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.540 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:20.540 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730422+0000", 2024-09-11T06:51:20.540 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000758648 2024-09-11T06:51:20.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.540 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:20.540 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730571+0000", 2024-09-11T06:51:20.540 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000148932 2024-09-11T06:51:20.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.541 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.541 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:20.541 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730936+0000", 2024-09-11T06:51:20.541 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036510900000000002 2024-09-11T06:51:20.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.541 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.541 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:20.541 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732159+0000", 2024-09-11T06:51:20.541 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001222302 2024-09-11T06:51:20.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.541 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.541 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:20.542 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732763+0000", 2024-09-11T06:51:20.542 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00060421599999999998 2024-09-11T06:51:20.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.542 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:20.542 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732916+0000", 2024-09-11T06:51:20.542 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015264 2024-09-11T06:51:20.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.542 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:20.542 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733097+0000", 2024-09-11T06:51:20.542 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000181504 2024-09-11T06:51:20.542 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.542 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:20.543 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.543 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.543 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.543 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:27 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:20.543 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:51:20.543 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 49.804162263999999, 2024-09-11T06:51:20.543 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003072852, 2024-09-11T06:51:20.543 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:20.543 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:20.543 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:20.543 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:20.543 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:20.543 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:20.543 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27 2024-09-11T06:51:20.543 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.544 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:20.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.544 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:20.544 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:51:20.544 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.544 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:20.544 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730606+0000", 2024-09-11T06:51:20.544 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.544 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:20.544 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730608+0000", 2024-09-11T06:51:20.544 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8369999999999999e-06 2024-09-11T06:51:20.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.545 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:20.545 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730617+0000", 2024-09-11T06:51:20.545 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1859999999999992e-06 2024-09-11T06:51:20.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.545 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:20.545 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730636+0000", 2024-09-11T06:51:20.545 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9296e-05 2024-09-11T06:51:20.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.545 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:20.545 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730674+0000", 2024-09-11T06:51:20.545 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8609000000000003e-05 2024-09-11T06:51:20.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.546 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:20.546 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731496+0000", 2024-09-11T06:51:20.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00082112299999999999 2024-09-11T06:51:20.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.546 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:20.546 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731722+0000", 2024-09-11T06:51:20.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022651400000000001 2024-09-11T06:51:20.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.546 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:20.546 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733215+0000", 2024-09-11T06:51:20.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001493257 2024-09-11T06:51:20.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.547 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:20.547 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733452+0000", 2024-09-11T06:51:20.547 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023631600000000001 2024-09-11T06:51:20.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.547 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:20.547 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733679+0000", 2024-09-11T06:51:20.547 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022771400000000001 2024-09-11T06:51:20.547 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.547 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:20.547 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.548 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:73 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:20.548 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:51:20.548 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 49.804077980000002, 2024-09-11T06:51:20.548 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033454439999999999, 2024-09-11T06:51:20.548 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:20.548 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:20.548 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:20.548 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:20.548 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:51:20.548 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:51:20.548 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73 2024-09-11T06:51:20.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.548 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:20.548 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:20.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:51:20.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:20.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:51:20.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:20.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730694+0000", 2024-09-11T06:51:20.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5099999999999999e-06 2024-09-11T06:51:20.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.550 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:20.550 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730708+0000", 2024-09-11T06:51:20.550 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3237e-05 2024-09-11T06:51:20.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.550 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:20.550 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730738+0000", 2024-09-11T06:51:20.550 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0688999999999997e-05 2024-09-11T06:51:20.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.550 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:20.550 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730759+0000", 2024-09-11T06:51:20.550 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0818999999999998e-05 2024-09-11T06:51:20.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.551 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:20.551 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731841+0000", 2024-09-11T06:51:20.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010821819999999999 2024-09-11T06:51:20.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.551 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:20.551 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732092+0000", 2024-09-11T06:51:20.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025086299999999999 2024-09-11T06:51:20.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.551 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:20.551 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733730+0000", 2024-09-11T06:51:20.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0016374600000000001 2024-09-11T06:51:20.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.551 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:20.552 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733866+0000", 2024-09-11T06:51:20.552 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000136666 2024-09-11T06:51:20.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.552 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.552 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:20.552 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.734036+0000", 2024-09-11T06:51:20.552 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000170018 2024-09-11T06:51:20.552 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.552 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:20.552 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.552 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.552 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:18 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:20.552 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:51:20.552 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 49.660660790999998, 2024-09-11T06:51:20.553 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0031646579999999999, 2024-09-11T06:51:20.553 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:20.553 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:20.553 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:20.553 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:20.553 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:20.553 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:20.553 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18 2024-09-11T06:51:20.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.553 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:20.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.553 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:20.553 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:51:20.553 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.554 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:20.554 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874108+0000", 2024-09-11T06:51:20.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.554 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:20.554 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874109+0000", 2024-09-11T06:51:20.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.426e-06 2024-09-11T06:51:20.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.554 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:20.554 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874118+0000", 2024-09-11T06:51:20.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6929999999999996e-06 2024-09-11T06:51:20.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.555 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.555 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:20.555 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874134+0000", 2024-09-11T06:51:20.555 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5503e-05 2024-09-11T06:51:20.555 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.555 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.555 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:20.555 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874144+0000", 2024-09-11T06:51:20.555 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0207000000000001e-05 2024-09-11T06:51:20.555 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.555 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.555 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:20.555 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874545+0000", 2024-09-11T06:51:20.555 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040120800000000001 2024-09-11T06:51:20.555 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.556 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.556 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:20.556 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.874686+0000", 2024-09-11T06:51:20.556 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014128599999999999 2024-09-11T06:51:20.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.556 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.556 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:20.556 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.875023+0000", 2024-09-11T06:51:20.556 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033706600000000001 2024-09-11T06:51:20.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.556 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.556 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:20.556 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.876217+0000", 2024-09-11T06:51:20.556 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011938999999999999 2024-09-11T06:51:20.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.557 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.557 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:20.557 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877042+0000", 2024-09-11T06:51:20.557 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00082499899999999998 2024-09-11T06:51:20.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.557 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.557 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:20.557 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877128+0000", 2024-09-11T06:51:20.557 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5903999999999996e-05 2024-09-11T06:51:20.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.557 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.557 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:20.557 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877273+0000", 2024-09-11T06:51:20.557 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144466 2024-09-11T06:51:20.558 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.558 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:20.558 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.558 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.558 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:24 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:20.558 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:51:20.558 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 49.646155084999997, 2024-09-11T06:51:20.558 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002980271, 2024-09-11T06:51:20.558 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:20.558 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:20.558 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:20.558 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:20.558 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:20.558 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:20.559 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 24 2024-09-11T06:51:20.559 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.559 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:20.559 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.559 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:20.559 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:51:20.559 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.559 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.559 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.559 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:20.559 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888614+0000", 2024-09-11T06:51:20.559 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.559 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.559 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.559 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:20.560 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888616+0000", 2024-09-11T06:51:20.560 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0030000000000001e-06 2024-09-11T06:51:20.560 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.560 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.560 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:20.560 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888626+0000", 2024-09-11T06:51:20.560 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0502999999999999e-05 2024-09-11T06:51:20.560 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.560 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.560 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:20.560 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888655+0000", 2024-09-11T06:51:20.560 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8419999999999999e-05 2024-09-11T06:51:20.560 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.560 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.560 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:20.561 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888688+0000", 2024-09-11T06:51:20.561 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3212e-05 2024-09-11T06:51:20.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.561 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.561 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:20.561 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.888829+0000", 2024-09-11T06:51:20.561 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014114299999999999 2024-09-11T06:51:20.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.561 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.561 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:20.561 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.889082+0000", 2024-09-11T06:51:20.561 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025279499999999999 2024-09-11T06:51:20.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.561 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.561 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:20.562 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.889577+0000", 2024-09-11T06:51:20.562 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00049536299999999997 2024-09-11T06:51:20.562 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.562 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.562 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:20.562 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.890811+0000", 2024-09-11T06:51:20.562 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001233858 2024-09-11T06:51:20.562 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.562 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.562 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:20.562 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891368+0000", 2024-09-11T06:51:20.562 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00055677400000000001 2024-09-11T06:51:20.562 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.562 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.563 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:20.563 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891456+0000", 2024-09-11T06:51:20.563 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8246999999999997e-05 2024-09-11T06:51:20.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.563 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:20.563 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.891594+0000", 2024-09-11T06:51:20.563 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013795300000000001 2024-09-11T06:51:20.563 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.563 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:20.563 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.563 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:26 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 139740787512064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:20.564 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:51:20.564 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 49.642452441000003, 2024-09-11T06:51:20.564 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003613733, 2024-09-11T06:51:20.564 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:20.564 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:20.564 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:20.564 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:20.564 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:20.564 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:20.564 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:51:20.564 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.564 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:20.564 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.564 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:20.565 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:51:20.565 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.565 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.565 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.565 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:20.565 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:51:20.565 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.565 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.565 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.565 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:20.565 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892318+0000", 2024-09-11T06:51:20.565 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.623e-06 2024-09-11T06:51:20.565 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.565 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.565 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:20.566 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892325+0000", 2024-09-11T06:51:20.566 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:51:20.566 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.566 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:20.566 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892346+0000", 2024-09-11T06:51:20.566 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1246e-05 2024-09-11T06:51:20.566 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.566 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:20.566 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892362+0000", 2024-09-11T06:51:20.566 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5409000000000001e-05 2024-09-11T06:51:20.566 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.567 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:20.567 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892883+0000", 2024-09-11T06:51:20.567 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00052087599999999998 2024-09-11T06:51:20.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.567 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:20.567 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893123+0000", 2024-09-11T06:51:20.567 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000240769 2024-09-11T06:51:20.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.567 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:20.567 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893526+0000", 2024-09-11T06:51:20.567 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040300200000000002 2024-09-11T06:51:20.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:20.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894820+0000", 2024-09-11T06:51:20.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012934159999999999 2024-09-11T06:51:20.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:20.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895608+0000", 2024-09-11T06:51:20.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00078788100000000004 2024-09-11T06:51:20.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:20.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895724+0000", 2024-09-11T06:51:20.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011595999999999999 2024-09-11T06:51:20.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.569 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:20.569 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895930+0000", 2024-09-11T06:51:20.569 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000206391 2024-09-11T06:51:20.569 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.569 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:20.569 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.569 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:133 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:20.569 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:51:20.569 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 49.641654424000002, 2024-09-11T06:51:20.570 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035017360000000001, 2024-09-11T06:51:20.570 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:20.570 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:20.570 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:20.570 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:20.570 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:51:20.570 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:51:20.570 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 133 2024-09-11T06:51:20.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.570 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:20.570 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.570 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:20.570 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:51:20.570 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.571 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.571 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:20.571 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:51:20.571 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.571 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.571 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:20.571 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893116+0000", 2024-09-11T06:51:20.571 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1629999999999999e-06 2024-09-11T06:51:20.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.571 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.571 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:20.571 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893128+0000", 2024-09-11T06:51:20.571 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.11e-05 2024-09-11T06:51:20.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.572 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:20.572 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893156+0000", 2024-09-11T06:51:20.572 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8266e-05 2024-09-11T06:51:20.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.572 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:20.572 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893171+0000", 2024-09-11T06:51:20.572 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5426000000000001e-05 2024-09-11T06:51:20.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.572 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:20.572 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894561+0000", 2024-09-11T06:51:20.573 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013899959999999999 2024-09-11T06:51:20.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.573 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.573 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:20.573 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894781+0000", 2024-09-11T06:51:20.573 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021987100000000001 2024-09-11T06:51:20.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.573 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.573 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:20.573 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896349+0000", 2024-09-11T06:51:20.573 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015681779999999999 2024-09-11T06:51:20.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.573 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.573 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:20.574 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896458+0000", 2024-09-11T06:51:20.574 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000108187 2024-09-11T06:51:20.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.574 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.574 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:20.574 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896616+0000", 2024-09-11T06:51:20.574 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015854899999999999 2024-09-11T06:51:20.574 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.574 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:20.574 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.574 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.574 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:27 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:20.574 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:51:20.574 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 49.641643043999998, 2024-09-11T06:51:20.575 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003151989, 2024-09-11T06:51:20.575 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:20.575 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:20.575 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:20.575 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:20.575 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:20.575 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:20.575 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27 2024-09-11T06:51:20.575 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.575 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:20.575 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.575 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:20.575 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:51:20.575 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.575 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.576 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.576 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:20.576 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893126+0000", 2024-09-11T06:51:20.576 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.576 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.576 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:20.576 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893128+0000", 2024-09-11T06:51:20.576 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.013e-06 2024-09-11T06:51:20.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.576 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.576 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:20.576 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893136+0000", 2024-09-11T06:51:20.576 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4190000000000002e-06 2024-09-11T06:51:20.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.577 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.577 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:20.577 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893158+0000", 2024-09-11T06:51:20.577 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1506999999999999e-05 2024-09-11T06:51:20.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.577 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.577 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:20.577 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893170+0000", 2024-09-11T06:51:20.577 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2741999999999999e-05 2024-09-11T06:51:20.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.577 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.577 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:20.577 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894255+0000", 2024-09-11T06:51:20.577 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001084453 2024-09-11T06:51:20.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.578 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.578 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:20.578 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894461+0000", 2024-09-11T06:51:20.578 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00020586700000000001 2024-09-11T06:51:20.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.578 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.578 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:20.578 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896033+0000", 2024-09-11T06:51:20.578 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001572092 2024-09-11T06:51:20.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.578 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.578 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:20.578 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896149+0000", 2024-09-11T06:51:20.578 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116636 2024-09-11T06:51:20.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.578 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.579 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:20.579 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896278+0000", 2024-09-11T06:51:20.579 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012826 2024-09-11T06:51:20.579 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.579 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:20.579 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.579 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.579 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:176 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch reconnect cookie 139735284585312 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:20.579 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:51:20.579 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 49.610276225, 2024-09-11T06:51:20.579 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033620960000000002, 2024-09-11T06:51:20.579 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:20.579 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:20.579 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:20.580 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:20.580 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:51:20.580 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:51:20.580 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 176 2024-09-11T06:51:20.580 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.580 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:20.580 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.580 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:20.580 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:51:20.580 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.580 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.580 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.580 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:20.581 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:51:20.581 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.581 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.581 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.581 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:20.581 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924494+0000", 2024-09-11T06:51:20.581 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.967e-06 2024-09-11T06:51:20.581 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.581 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.581 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:20.581 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924501+0000", 2024-09-11T06:51:20.581 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2659999999999998e-06 2024-09-11T06:51:20.581 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.581 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.581 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:20.582 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924520+0000", 2024-09-11T06:51:20.582 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9038999999999998e-05 2024-09-11T06:51:20.582 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.582 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.582 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:20.582 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924535+0000", 2024-09-11T06:51:20.582 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5359999999999999e-05 2024-09-11T06:51:20.582 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.582 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.582 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:20.582 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.925544+0000", 2024-09-11T06:51:20.582 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010085109999999999 2024-09-11T06:51:20.582 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.582 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.582 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:20.582 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.925671+0000", 2024-09-11T06:51:20.583 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012756599999999999 2024-09-11T06:51:20.583 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.583 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.583 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:20.583 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.927855+0000", 2024-09-11T06:51:20.583 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0021833870000000002 2024-09-11T06:51:20.583 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.583 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:20.583 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.583 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.583 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.583 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4828.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch watch cookie 140587969811264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-11T06:51:20.583 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:51:20.583 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 44.288573266999997, 2024-09-11T06:51:20.584 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032778859999999998, 2024-09-11T06:51:20.584 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:20.584 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:20.584 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:20.584 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:20.584 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4828", 2024-09-11T06:51:20.584 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2410288650", 2024-09-11T06:51:20.584 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:51:20.584 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.585 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:20.585 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.585 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:20.585 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:51:20.585 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.585 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.585 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:20.585 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:51:20.585 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.585 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.585 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:20.586 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246198+0000", 2024-09-11T06:51:20.586 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.667e-06 2024-09-11T06:51:20.586 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.586 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.586 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:20.586 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246209+0000", 2024-09-11T06:51:20.586 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0409e-05 2024-09-11T06:51:20.586 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.586 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.586 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:20.586 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246241+0000", 2024-09-11T06:51:20.586 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2592999999999998e-05 2024-09-11T06:51:20.586 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.586 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.587 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:20.587 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246260+0000", 2024-09-11T06:51:20.587 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9343000000000001e-05 2024-09-11T06:51:20.587 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.587 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.587 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:20.587 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246466+0000", 2024-09-11T06:51:20.587 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000205701 2024-09-11T06:51:20.587 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.587 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.587 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:20.587 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246713+0000", 2024-09-11T06:51:20.587 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024646999999999999 2024-09-11T06:51:20.588 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.588 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.588 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:20.588 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.247133+0000", 2024-09-11T06:51:20.588 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000420536 2024-09-11T06:51:20.588 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.588 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.588 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:20.588 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.248541+0000", 2024-09-11T06:51:20.588 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001407801 2024-09-11T06:51:20.588 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.588 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.589 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:20.589 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249241+0000", 2024-09-11T06:51:20.589 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070043500000000001 2024-09-11T06:51:20.589 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.589 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.589 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:20.589 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249326+0000", 2024-09-11T06:51:20.589 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5037999999999995e-05 2024-09-11T06:51:20.589 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.589 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.590 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:20.590 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249473+0000", 2024-09-11T06:51:20.590 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014689299999999999 2024-09-11T06:51:20.590 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.590 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:20.590 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.590 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.590 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.590 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5313.0:25 2.1 2:967e1786:ns::rbd_object_map.11adc335555f.0000000000000004:head [writefull 0~38] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:51:20.590 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.256788+0000", 2024-09-11T06:51:20.590 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.277980268, 2024-09-11T06:51:20.590 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0030446449999999999, 2024-09-11T06:51:20.591 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:20.591 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:20.591 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:20.591 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:20.591 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5313", 2024-09-11T06:51:20.591 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/624314229", 2024-09-11T06:51:20.591 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 25 2024-09-11T06:51:20.591 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.591 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:20.591 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.591 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:20.591 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256788+0000", 2024-09-11T06:51:20.592 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.592 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.592 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.592 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:20.592 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256788+0000", 2024-09-11T06:51:20.592 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.592 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.592 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.592 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:20.592 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256791+0000", 2024-09-11T06:51:20.593 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5770000000000001e-06 2024-09-11T06:51:20.593 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.593 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.593 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:20.593 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256802+0000", 2024-09-11T06:51:20.593 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0742e-05 2024-09-11T06:51:20.593 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.593 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.593 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:20.593 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256823+0000", 2024-09-11T06:51:20.593 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0999999999999999e-05 2024-09-11T06:51:20.594 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.594 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.594 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:20.594 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256841+0000", 2024-09-11T06:51:20.594 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8099000000000001e-05 2024-09-11T06:51:20.594 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.594 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.594 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:20.594 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.256988+0000", 2024-09-11T06:51:20.594 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014756400000000001 2024-09-11T06:51:20.595 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.595 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.595 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:20.595 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.257234+0000", 2024-09-11T06:51:20.595 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024524399999999999 2024-09-11T06:51:20.595 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.595 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.595 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:20.596 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.257446+0000", 2024-09-11T06:51:20.596 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000212241 2024-09-11T06:51:20.596 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.596 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.596 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:20.596 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259292+0000", 2024-09-11T06:51:20.596 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001845937 2024-09-11T06:51:20.596 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.596 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.596 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:20.596 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259681+0000", 2024-09-11T06:51:20.596 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00038940699999999998 2024-09-11T06:51:20.597 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.597 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.597 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:20.597 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259770+0000", 2024-09-11T06:51:20.597 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8385000000000002e-05 2024-09-11T06:51:20.597 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.597 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.597 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:20.597 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.259833+0000", 2024-09-11T06:51:20.597 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3448999999999995e-05 2024-09-11T06:51:20.597 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.598 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:20.598 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.598 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.598 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.598 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5328.0:14 2.1 2:967e1786:ns::rbd_object_map.11adc335555f.0000000000000004:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:51:20.598 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.396257+0000", 2024-09-11T06:51:20.598 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.138511819000001, 2024-09-11T06:51:20.598 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032125370000000001, 2024-09-11T06:51:20.598 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:20.598 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:20.598 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:20.599 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:20.599 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5328", 2024-09-11T06:51:20.599 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2386493698", 2024-09-11T06:51:20.599 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 14 2024-09-11T06:51:20.599 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.599 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:20.599 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.599 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:20.599 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396257+0000", 2024-09-11T06:51:20.599 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.599 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.600 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.600 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:20.600 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396257+0000", 2024-09-11T06:51:20.600 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.600 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.600 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.600 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:20.600 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396259+0000", 2024-09-11T06:51:20.600 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4830000000000002e-06 2024-09-11T06:51:20.600 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.600 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.601 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:20.601 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396271+0000", 2024-09-11T06:51:20.601 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.115e-05 2024-09-11T06:51:20.601 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.601 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.601 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:20.601 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396292+0000", 2024-09-11T06:51:20.601 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1761999999999999e-05 2024-09-11T06:51:20.601 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.601 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.602 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:20.602 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396314+0000", 2024-09-11T06:51:20.602 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1982999999999999e-05 2024-09-11T06:51:20.602 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.602 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.602 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:20.602 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396409+0000", 2024-09-11T06:51:20.602 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4250000000000001e-05 2024-09-11T06:51:20.602 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.602 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.602 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:20.602 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396464+0000", 2024-09-11T06:51:20.603 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5884000000000002e-05 2024-09-11T06:51:20.603 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.603 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.603 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:20.603 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.399469+0000", 2024-09-11T06:51:20.603 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0030050250000000001 2024-09-11T06:51:20.603 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.603 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:20.603 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.603 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.603 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.604 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:6 3.5 3:b2cc3336:::rbd_id.testrbdnbd21607:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:20.604 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.591899+0000", 2024-09-11T06:51:20.604 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 34.942869318, 2024-09-11T06:51:20.604 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032382999999999999, 2024-09-11T06:51:20.604 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:20.604 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:20.604 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:20.604 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:20.604 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:51:20.604 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:51:20.604 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 6 2024-09-11T06:51:20.604 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.605 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:20.605 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.605 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:20.605 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591899+0000", 2024-09-11T06:51:20.605 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.605 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.605 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.605 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:20.605 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591899+0000", 2024-09-11T06:51:20.605 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.605 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.605 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.606 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:20.606 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591902+0000", 2024-09-11T06:51:20.606 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0770000000000002e-06 2024-09-11T06:51:20.606 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.606 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.606 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:20.606 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591913+0000", 2024-09-11T06:51:20.606 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1182999999999999e-05 2024-09-11T06:51:20.606 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.606 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.606 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:20.606 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591940+0000", 2024-09-11T06:51:20.607 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7233000000000002e-05 2024-09-11T06:51:20.607 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.607 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.607 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:20.607 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591954+0000", 2024-09-11T06:51:20.607 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4503e-05 2024-09-11T06:51:20.607 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.607 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.607 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:20.607 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.592071+0000", 2024-09-11T06:51:20.607 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116731 2024-09-11T06:51:20.607 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.607 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.607 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:20.608 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.592274+0000", 2024-09-11T06:51:20.608 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00020320900000000001 2024-09-11T06:51:20.608 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.608 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.608 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 3", 2024-09-11T06:51:20.608 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.592576+0000", 2024-09-11T06:51:20.608 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030179000000000002 2024-09-11T06:51:20.608 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.608 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.608 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:20.608 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.594072+0000", 2024-09-11T06:51:20.608 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001495496 2024-09-11T06:51:20.608 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.609 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.609 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:20.609 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.594945+0000", 2024-09-11T06:51:20.609 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00087379799999999996 2024-09-11T06:51:20.609 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.609 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.609 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:20.609 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.595023+0000", 2024-09-11T06:51:20.609 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7751999999999997e-05 2024-09-11T06:51:20.609 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.609 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.609 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:20.609 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.595138+0000", 2024-09-11T06:51:20.609 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000114528 2024-09-11T06:51:20.610 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.610 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:20.610 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.610 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.610 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.610 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:44 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:20.610 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.540846+0000", 2024-09-11T06:51:20.610 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.993923186, 2024-09-11T06:51:20.610 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032138639999999999, 2024-09-11T06:51:20.610 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:20.610 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:20.610 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:20.610 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:20.611 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:20.611 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:20.611 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 44 2024-09-11T06:51:20.611 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.611 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:20.611 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.611 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:20.611 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540846+0000", 2024-09-11T06:51:20.611 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.611 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.611 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.611 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:20.611 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540846+0000", 2024-09-11T06:51:20.611 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.612 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.612 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.612 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:20.612 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540848+0000", 2024-09-11T06:51:20.612 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5469999999999999e-06 2024-09-11T06:51:20.612 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.612 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.612 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:20.612 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540863+0000", 2024-09-11T06:51:20.612 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4613000000000001e-05 2024-09-11T06:51:20.612 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.612 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.612 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:20.612 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540895+0000", 2024-09-11T06:51:20.613 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2153000000000002e-05 2024-09-11T06:51:20.613 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.613 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.613 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:20.613 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540912+0000", 2024-09-11T06:51:20.613 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7405999999999999e-05 2024-09-11T06:51:20.613 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.613 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.613 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:20.613 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.541111+0000", 2024-09-11T06:51:20.613 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019849599999999999 2024-09-11T06:51:20.613 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.613 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.613 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:20.614 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.541269+0000", 2024-09-11T06:51:20.614 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015792399999999999 2024-09-11T06:51:20.614 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.614 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.614 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:20.614 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.541853+0000", 2024-09-11T06:51:20.614 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00058443799999999995 2024-09-11T06:51:20.614 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.614 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.614 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:20.614 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.543133+0000", 2024-09-11T06:51:20.614 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012797850000000001 2024-09-11T06:51:20.614 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.615 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.615 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:20.615 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.543905+0000", 2024-09-11T06:51:20.615 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000771655 2024-09-11T06:51:20.615 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.615 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.615 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:20.615 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.543993+0000", 2024-09-11T06:51:20.615 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8168e-05 2024-09-11T06:51:20.615 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.615 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.615 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:20.615 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.544059+0000", 2024-09-11T06:51:20.615 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6679000000000003e-05 2024-09-11T06:51:20.616 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.616 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:20.616 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.616 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.616 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.616 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5553.0:24 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 139650593197904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:20.616 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.566567+0000", 2024-09-11T06:51:20.616 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.968201916000002, 2024-09-11T06:51:20.616 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034350790000000002, 2024-09-11T06:51:20.616 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:20.616 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:20.617 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:20.617 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:20.617 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5553", 2024-09-11T06:51:20.617 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3590511604", 2024-09-11T06:51:20.617 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 24 2024-09-11T06:51:20.617 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.617 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:20.617 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.617 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:20.617 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566567+0000", 2024-09-11T06:51:20.617 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.617 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.618 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.618 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:20.618 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566567+0000", 2024-09-11T06:51:20.618 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.618 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.618 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.618 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:20.618 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566569+0000", 2024-09-11T06:51:20.618 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.973e-06 2024-09-11T06:51:20.618 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.618 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.618 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:20.619 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566578+0000", 2024-09-11T06:51:20.619 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8170000000000005e-06 2024-09-11T06:51:20.619 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.619 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.620 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:20.620 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566602+0000", 2024-09-11T06:51:20.620 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4596e-05 2024-09-11T06:51:20.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.621 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.621 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:20.621 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566619+0000", 2024-09-11T06:51:20.621 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7099999999999999e-05 2024-09-11T06:51:20.621 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.621 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.621 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:20.621 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.567321+0000", 2024-09-11T06:51:20.621 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070168200000000002 2024-09-11T06:51:20.621 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.621 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.622 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:20.622 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.567454+0000", 2024-09-11T06:51:20.622 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013286400000000001 2024-09-11T06:51:20.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.622 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.622 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:20.622 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.567674+0000", 2024-09-11T06:51:20.622 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000219679 2024-09-11T06:51:20.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.622 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.622 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:20.623 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.569014+0000", 2024-09-11T06:51:20.623 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013404129999999999 2024-09-11T06:51:20.623 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.623 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.623 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:20.623 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.569634+0000", 2024-09-11T06:51:20.623 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00062029200000000004 2024-09-11T06:51:20.623 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.623 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.623 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:20.623 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.569805+0000", 2024-09-11T06:51:20.624 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000170366 2024-09-11T06:51:20.624 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.624 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.624 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:20.624 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.570002+0000", 2024-09-11T06:51:20.624 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019729700000000001 2024-09-11T06:51:20.624 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.624 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:20.624 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.624 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.624 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.625 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:82 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch reconnect cookie 139888024456048 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:20.625 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.690185+0000", 2024-09-11T06:51:20.625 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.844583914000001, 2024-09-11T06:51:20.625 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0045884799999999998, 2024-09-11T06:51:20.625 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:20.625 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:20.625 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:20.625 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:20.625 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:20.625 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:20.625 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 82 2024-09-11T06:51:20.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.626 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:20.626 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.626 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:20.626 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690185+0000", 2024-09-11T06:51:20.626 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.626 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.626 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:20.626 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690185+0000", 2024-09-11T06:51:20.626 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.627 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.627 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.627 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:20.627 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690186+0000", 2024-09-11T06:51:20.627 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8400000000000002e-07 2024-09-11T06:51:20.627 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.627 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.627 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:20.627 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690190+0000", 2024-09-11T06:51:20.627 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5499999999999996e-06 2024-09-11T06:51:20.628 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.628 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.628 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:20.628 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690203+0000", 2024-09-11T06:51:20.628 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2228999999999999e-05 2024-09-11T06:51:20.628 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.628 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.628 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:20.628 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690228+0000", 2024-09-11T06:51:20.628 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5587000000000001e-05 2024-09-11T06:51:20.628 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.629 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.629 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:20.629 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.691298+0000", 2024-09-11T06:51:20.629 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001070218 2024-09-11T06:51:20.629 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.629 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.629 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:20.629 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.691511+0000", 2024-09-11T06:51:20.629 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021301299999999999 2024-09-11T06:51:20.629 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.630 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.630 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:20.630 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.694773+0000", 2024-09-11T06:51:20.630 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003261899 2024-09-11T06:51:20.630 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.630 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:20.630 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.630 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.630 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.630 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:83 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify cookie 139887621709056] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:20.630 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.690242+0000", 2024-09-11T06:51:20.630 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.844526741999999, 2024-09-11T06:51:20.631 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003040067, 2024-09-11T06:51:20.631 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:20.631 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:20.631 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:20.631 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:20.631 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:20.631 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:20.631 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 83 2024-09-11T06:51:20.631 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.631 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:20.631 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.632 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:20.632 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690242+0000", 2024-09-11T06:51:20.632 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.632 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.632 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.632 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:20.632 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690242+0000", 2024-09-11T06:51:20.632 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.632 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.632 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.632 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:20.633 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690243+0000", 2024-09-11T06:51:20.633 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0640000000000001e-06 2024-09-11T06:51:20.633 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.633 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.633 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:20.633 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690247+0000", 2024-09-11T06:51:20.633 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3760000000000001e-06 2024-09-11T06:51:20.633 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.633 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.633 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:20.633 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690258+0000", 2024-09-11T06:51:20.634 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0874e-05 2024-09-11T06:51:20.634 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.634 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.634 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:20.634 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690266+0000", 2024-09-11T06:51:20.634 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5429999999999996e-06 2024-09-11T06:51:20.634 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.634 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.634 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:20.634 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.692564+0000", 2024-09-11T06:51:20.634 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0022977010000000001 2024-09-11T06:51:20.634 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.634 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.635 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:20.635 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.692882+0000", 2024-09-11T06:51:20.635 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00031862699999999998 2024-09-11T06:51:20.635 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.635 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.635 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:20.635 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693282+0000", 2024-09-11T06:51:20.635 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000399882 2024-09-11T06:51:20.635 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.635 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:20.635 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.635 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.635 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.635 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:84 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify cookie 139887823033168] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:20.635 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:51:20.636 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.844492138, 2024-09-11T06:51:20.636 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035622620000000001, 2024-09-11T06:51:20.636 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:20.636 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:20.636 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:20.636 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:20.636 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:20.636 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:20.636 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 84 2024-09-11T06:51:20.636 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.636 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:20.636 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.636 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:20.636 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:51:20.637 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.637 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.637 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.637 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:20.637 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:51:20.637 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.637 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.637 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.637 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:20.637 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:51:20.637 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5300000000000003e-07 2024-09-11T06:51:20.637 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.637 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.637 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:20.638 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690281+0000", 2024-09-11T06:51:20.638 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9330000000000003e-06 2024-09-11T06:51:20.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.638 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.638 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:20.638 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690291+0000", 2024-09-11T06:51:20.638 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7359999999999996e-06 2024-09-11T06:51:20.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.638 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.638 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:20.638 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690298+0000", 2024-09-11T06:51:20.638 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3699999999999997e-06 2024-09-11T06:51:20.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.638 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.638 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:20.639 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693380+0000", 2024-09-11T06:51:20.639 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0030811060000000001 2024-09-11T06:51:20.639 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.639 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.639 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:20.639 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693599+0000", 2024-09-11T06:51:20.639 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021947199999999999 2024-09-11T06:51:20.639 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.639 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.639 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:20.639 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693839+0000", 2024-09-11T06:51:20.639 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023989200000000001 2024-09-11T06:51:20.639 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.639 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:20.639 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.640 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.640 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5098.0:25 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 140617967560800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:20.640 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.697727+0000", 2024-09-11T06:51:20.640 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.837041562, 2024-09-11T06:51:20.640 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035341330000000001, 2024-09-11T06:51:20.640 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:20.640 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:20.640 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:20.640 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:20.640 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5098", 2024-09-11T06:51:20.640 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/844541331", 2024-09-11T06:51:20.640 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 25 2024-09-11T06:51:20.641 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.641 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:20.641 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.641 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:20.641 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697727+0000", 2024-09-11T06:51:20.641 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.641 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.641 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.641 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:20.641 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697727+0000", 2024-09-11T06:51:20.641 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.641 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.641 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.642 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:20.642 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697729+0000", 2024-09-11T06:51:20.642 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.33e-06 2024-09-11T06:51:20.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.642 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.642 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:20.642 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697734+0000", 2024-09-11T06:51:20.642 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6590000000000004e-06 2024-09-11T06:51:20.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.642 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.642 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:20.642 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697749+0000", 2024-09-11T06:51:20.642 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4867e-05 2024-09-11T06:51:20.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.643 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.643 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:20.643 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697759+0000", 2024-09-11T06:51:20.643 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0073e-05 2024-09-11T06:51:20.643 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.643 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.643 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:20.643 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.698715+0000", 2024-09-11T06:51:20.643 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00095545799999999996 2024-09-11T06:51:20.643 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.643 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.643 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:20.643 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.698850+0000", 2024-09-11T06:51:20.644 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000135814 2024-09-11T06:51:20.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.644 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.644 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:20.644 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.699085+0000", 2024-09-11T06:51:20.644 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023507199999999999 2024-09-11T06:51:20.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.644 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.644 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:20.644 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.700258+0000", 2024-09-11T06:51:20.644 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011726600000000001 2024-09-11T06:51:20.645 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.645 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.645 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:20.645 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.701061+0000", 2024-09-11T06:51:20.645 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00080256100000000003 2024-09-11T06:51:20.645 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.645 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.645 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:20.645 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.701144+0000", 2024-09-11T06:51:20.645 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3188000000000004e-05 2024-09-11T06:51:20.645 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.645 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.646 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:20.646 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.701261+0000", 2024-09-11T06:51:20.646 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000117451 2024-09-11T06:51:20.646 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.646 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:20.646 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.646 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.646 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.646 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5380.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b,call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b,call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b,call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:20.646 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:12.536478+0000", 2024-09-11T06:51:20.646 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 7.9982903810000003, 2024-09-11T06:51:20.646 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032220090000000001, 2024-09-11T06:51:20.647 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:20.647 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:20.647 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:20.647 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:20.647 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5380", 2024-09-11T06:51:20.647 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/290149879", 2024-09-11T06:51:20.647 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:51:20.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.647 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:20.647 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.647 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:20.648 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536478+0000", 2024-09-11T06:51:20.648 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.648 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.648 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.648 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:20.648 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536478+0000", 2024-09-11T06:51:20.648 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:20.648 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.648 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.648 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:20.648 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536481+0000", 2024-09-11T06:51:20.648 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4059999999999999e-06 2024-09-11T06:51:20.648 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.649 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.649 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:20.649 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536493+0000", 2024-09-11T06:51:20.649 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2003e-05 2024-09-11T06:51:20.649 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.649 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.649 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:20.649 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536526+0000", 2024-09-11T06:51:20.649 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2839999999999997e-05 2024-09-11T06:51:20.649 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.649 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.649 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:20.649 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536542+0000", 2024-09-11T06:51:20.649 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6120000000000002e-05 2024-09-11T06:51:20.650 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.650 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.650 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:20.650 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536662+0000", 2024-09-11T06:51:20.650 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000120285 2024-09-11T06:51:20.650 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.650 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.650 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:20.650 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536946+0000", 2024-09-11T06:51:20.650 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00028412400000000002 2024-09-11T06:51:20.650 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:20.650 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:20.650 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:20.650 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.539700+0000", 2024-09-11T06:51:20.651 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027542310000000002 2024-09-11T06:51:20.651 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.651 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:20.651 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.651 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:20.651 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:20.651 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:20.652 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd config image rm rbd/testrbdnbd21607 rbd_discard_granularity_bytes 2024-09-11T06:51:20.653 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:51:20.654 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:20.646+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:20.683 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo rbd device --device-type nbd map rbd/testrbdnbd21607 2024-09-11T06:51:20.683 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:51:20.683 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:51:20.684 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:51:20.685 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which rbd 2024-09-11T06:51:20.687 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/rbd 2024-09-11T06:51:20.687 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:51:20.687 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map rbd/testrbdnbd21607 2024-09-11T06:51:20.754 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:20.749+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:20.788 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:20.788 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:20.788 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:51:20.788 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:20.802 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:51:20.833 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV=/dev/nbd2 2024-09-11T06:51:20.833 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd 2024-09-11T06:51:20.833 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:51:20.833 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:51:20.833 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:51:20.833 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd2'\'']/id' 2024-09-11T06:51:20.854 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:20.850+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:20.936 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=24979 2024-09-11T06:51:20.936 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 24979 2024-09-11T06:51:20.936 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 24979 -C rbd-nbd 2024-09-11T06:51:20.943 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:51:20.943 INFO:tasks.workunit.client.0.smithi176.stdout: 24979 ? 00:00:00 rbd-nbd 2024-09-11T06:51:20.944 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd feature enable rbd/testrbdnbd21607 journaling 2024-09-11T06:51:20.955 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:20.949+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:20.969 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:20.969 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:20.969 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:51:20.969 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:51:20.969 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:20.977 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is set 2024-09-11T06:51:20.984 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:51:21.069 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:21.065+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:21.147 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:21.147 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:51:21.147 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:51:21.147 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:51:21.147 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.147 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:21.147 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:51:21.147 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 134.87072823599999, 2024-09-11T06:51:21.148 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283336, 2024-09-11T06:51:21.148 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.148 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.148 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:21.148 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:21.148 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:21.148 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:21.148 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 87 2024-09-11T06:51:21.148 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.148 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.148 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.148 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.149 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:51:21.149 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.149 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.149 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.149 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.149 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:51:21.149 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.149 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.149 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.149 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.150 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271768+0000", 2024-09-11T06:51:21.150 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7599999999999996e-07 2024-09-11T06:51:21.150 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.150 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.150 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.150 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271860+0000", 2024-09-11T06:51:21.150 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2639999999999994e-05 2024-09-11T06:51:21.150 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.150 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.150 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.150 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271873+0000", 2024-09-11T06:51:21.151 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3139000000000001e-05 2024-09-11T06:51:21.151 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.151 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.151 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.151 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271882+0000", 2024-09-11T06:51:21.151 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2369999999999992e-06 2024-09-11T06:51:21.151 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.151 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.151 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.152 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277150+0000", 2024-09-11T06:51:21.152 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052684419999999999 2024-09-11T06:51:21.152 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.152 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.152 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.152 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277273+0000", 2024-09-11T06:51:21.152 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012278899999999999 2024-09-11T06:51:21.152 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.152 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.153 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:21.153 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277466+0000", 2024-09-11T06:51:21.153 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193578 2024-09-11T06:51:21.153 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.153 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.153 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:21.153 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.280808+0000", 2024-09-11T06:51:21.153 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003341971 2024-09-11T06:51:21.153 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.154 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.154 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:21.154 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284419+0000", 2024-09-11T06:51:21.154 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003610982 2024-09-11T06:51:21.154 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.154 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.154 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:21.154 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284531+0000", 2024-09-11T06:51:21.154 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011121299999999999 2024-09-11T06:51:21.154 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.155 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.155 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.155 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284600+0000", 2024-09-11T06:51:21.155 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9592999999999996e-05 2024-09-11T06:51:21.155 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.155 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.155 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.155 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.155 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.156 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:21.156 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:51:21.156 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 134.870597984, 2024-09-11T06:51:21.156 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283314, 2024-09-11T06:51:21.156 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.156 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.156 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:21.156 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:21.157 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:21.157 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:21.157 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 88 2024-09-11T06:51:21.157 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.157 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.157 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.157 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.157 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:51:21.157 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.157 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.157 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.158 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.158 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:51:21.158 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.158 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.158 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.158 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.158 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271898+0000", 2024-09-11T06:51:21.158 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2999999999999999e-07 2024-09-11T06:51:21.158 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.158 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.158 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.159 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271904+0000", 2024-09-11T06:51:21.159 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.767e-06 2024-09-11T06:51:21.159 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.159 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.159 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.159 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271913+0000", 2024-09-11T06:51:21.159 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7200000000000001e-06 2024-09-11T06:51:21.159 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.159 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.159 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.159 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271920+0000", 2024-09-11T06:51:21.159 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7100000000000001e-06 2024-09-11T06:51:21.160 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.160 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.160 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.160 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278684+0000", 2024-09-11T06:51:21.160 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0067642650000000002 2024-09-11T06:51:21.160 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.160 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.160 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.160 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278838+0000", 2024-09-11T06:51:21.160 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015371499999999999 2024-09-11T06:51:21.160 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.160 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.161 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:21.161 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.279040+0000", 2024-09-11T06:51:21.161 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020207499999999999 2024-09-11T06:51:21.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.161 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.161 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:21.161 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.281448+0000", 2024-09-11T06:51:21.161 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002407686 2024-09-11T06:51:21.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.161 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.162 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:21.162 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284615+0000", 2024-09-11T06:51:21.162 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031675499999999999 2024-09-11T06:51:21.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.162 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.162 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:21.162 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284682+0000", 2024-09-11T06:51:21.162 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7032000000000005e-05 2024-09-11T06:51:21.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.162 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.162 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.162 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284730+0000", 2024-09-11T06:51:21.163 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7790000000000002e-05 2024-09-11T06:51:21.163 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.163 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.163 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.163 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.163 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.163 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:21.163 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:51:21.163 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 134.70174878399999, 2024-09-11T06:51:21.163 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014944941, 2024-09-11T06:51:21.163 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.163 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.164 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:21.164 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:21.164 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:21.164 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:21.164 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 111 2024-09-11T06:51:21.164 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.164 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.164 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.164 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.164 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:51:21.164 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.165 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.165 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.165 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:51:21.165 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.165 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.165 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.165 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440748+0000", 2024-09-11T06:51:21.165 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:51:21.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.165 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.166 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.166 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440760+0000", 2024-09-11T06:51:21.166 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1897e-05 2024-09-11T06:51:21.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.166 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.166 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.166 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440782+0000", 2024-09-11T06:51:21.166 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2552999999999999e-05 2024-09-11T06:51:21.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.167 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.167 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.167 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440793+0000", 2024-09-11T06:51:21.167 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0643e-05 2024-09-11T06:51:21.167 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.167 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.167 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.167 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449755+0000", 2024-09-11T06:51:21.167 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089622620000000004 2024-09-11T06:51:21.167 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.167 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.168 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.168 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449918+0000", 2024-09-11T06:51:21.168 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163086 2024-09-11T06:51:21.168 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.168 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.168 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:21.168 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450150+0000", 2024-09-11T06:51:21.168 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023204500000000001 2024-09-11T06:51:21.168 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.168 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.169 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:21.169 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454533+0000", 2024-09-11T06:51:21.169 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0043831349999999998 2024-09-11T06:51:21.169 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.169 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.169 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:21.169 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455511+0000", 2024-09-11T06:51:21.169 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0009776050000000001 2024-09-11T06:51:21.169 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.169 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.170 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:21.170 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455597+0000", 2024-09-11T06:51:21.170 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6446000000000005e-05 2024-09-11T06:51:21.170 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.170 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.170 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.170 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455691+0000", 2024-09-11T06:51:21.170 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3832000000000006e-05 2024-09-11T06:51:21.170 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.171 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.171 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.171 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.171 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.171 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:21.171 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:51:21.171 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 134.70167902099999, 2024-09-11T06:51:21.171 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018885411000000001, 2024-09-11T06:51:21.172 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.172 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.172 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:21.172 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:21.172 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:21.172 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:21.172 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 112 2024-09-11T06:51:21.172 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.173 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.173 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.173 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.173 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:51:21.173 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.173 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.173 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.174 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.174 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:51:21.174 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.174 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.174 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.174 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.174 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440817+0000", 2024-09-11T06:51:21.174 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:51:21.174 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.175 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.175 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.175 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441003+0000", 2024-09-11T06:51:21.175 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018661200000000001 2024-09-11T06:51:21.175 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.175 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.175 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.176 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441027+0000", 2024-09-11T06:51:21.176 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3920000000000001e-05 2024-09-11T06:51:21.176 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.176 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.177 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.177 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441062+0000", 2024-09-11T06:51:21.177 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4663000000000003e-05 2024-09-11T06:51:21.177 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.177 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.177 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.177 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451981+0000", 2024-09-11T06:51:21.177 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010918926000000001 2024-09-11T06:51:21.178 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.178 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.178 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.178 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452111+0000", 2024-09-11T06:51:21.178 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129529 2024-09-11T06:51:21.178 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.178 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.178 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:21.179 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452311+0000", 2024-09-11T06:51:21.179 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020013799999999999 2024-09-11T06:51:21.179 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.179 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.179 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:21.179 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455809+0000", 2024-09-11T06:51:21.179 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003498443 2024-09-11T06:51:21.180 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.180 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.180 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:21.180 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459566+0000", 2024-09-11T06:51:21.180 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003757271 2024-09-11T06:51:21.180 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.181 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.181 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:21.181 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459640+0000", 2024-09-11T06:51:21.181 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3949000000000006e-05 2024-09-11T06:51:21.181 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.181 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.182 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.182 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459701+0000", 2024-09-11T06:51:21.182 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1117000000000005e-05 2024-09-11T06:51:21.182 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.182 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.182 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.182 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.182 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.183 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:21.183 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:51:21.183 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 134.70140017, 2024-09-11T06:51:21.183 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018760915, 2024-09-11T06:51:21.183 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.183 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.183 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:21.183 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:21.183 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:21.184 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:21.184 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 113 2024-09-11T06:51:21.184 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.184 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.184 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.184 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.184 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:51:21.184 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.184 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.185 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.185 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.185 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:51:21.185 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.185 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.185 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.185 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.185 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441096+0000", 2024-09-11T06:51:21.185 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4130000000000001e-06 2024-09-11T06:51:21.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.186 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.186 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.186 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441110+0000", 2024-09-11T06:51:21.186 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3900000000000001e-05 2024-09-11T06:51:21.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.186 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.186 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.186 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441131+0000", 2024-09-11T06:51:21.186 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0636999999999999e-05 2024-09-11T06:51:21.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.186 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.187 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.187 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441147+0000", 2024-09-11T06:51:21.187 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6512999999999999e-05 2024-09-11T06:51:21.187 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.187 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.187 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.187 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453274+0000", 2024-09-11T06:51:21.187 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012126764999999999 2024-09-11T06:51:21.187 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.187 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.187 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.187 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453498+0000", 2024-09-11T06:51:21.188 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000223568 2024-09-11T06:51:21.188 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.188 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.188 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:21.188 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453872+0000", 2024-09-11T06:51:21.188 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037467099999999998 2024-09-11T06:51:21.188 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.188 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.188 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:21.188 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458709+0000", 2024-09-11T06:51:21.188 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0048365680000000003 2024-09-11T06:51:21.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.189 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.189 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:21.189 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459716+0000", 2024-09-11T06:51:21.189 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010075240000000001 2024-09-11T06:51:21.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.189 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.189 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:21.189 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459779+0000", 2024-09-11T06:51:21.189 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2336999999999996e-05 2024-09-11T06:51:21.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.190 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.190 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.190 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459856+0000", 2024-09-11T06:51:21.190 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7019000000000002e-05 2024-09-11T06:51:21.190 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.190 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.190 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.190 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.190 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:21.190 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:51:21.190 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 134.70132343099999, 2024-09-11T06:51:21.191 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018963114, 2024-09-11T06:51:21.191 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.191 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.191 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:21.191 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:21.191 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:21.191 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:21.191 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 114 2024-09-11T06:51:21.192 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.192 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.192 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.192 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.192 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:51:21.192 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.192 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.192 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.192 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.192 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:51:21.193 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.193 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.193 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.193 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:51:21.193 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9299999999999996e-07 2024-09-11T06:51:21.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.193 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.193 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.193 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441180+0000", 2024-09-11T06:51:21.193 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7200000000000006e-06 2024-09-11T06:51:21.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.194 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.194 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.194 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441265+0000", 2024-09-11T06:51:21.194 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4753000000000007e-05 2024-09-11T06:51:21.194 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.194 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.194 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.194 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441288+0000", 2024-09-11T06:51:21.194 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2860000000000001e-05 2024-09-11T06:51:21.194 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.195 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.195 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.195 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454585+0000", 2024-09-11T06:51:21.195 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013297286 2024-09-11T06:51:21.195 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.195 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.195 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.195 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454714+0000", 2024-09-11T06:51:21.195 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000128465 2024-09-11T06:51:21.195 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.196 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.196 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:21.196 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454909+0000", 2024-09-11T06:51:21.196 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019522499999999999 2024-09-11T06:51:21.196 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.196 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.196 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:21.196 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459868+0000", 2024-09-11T06:51:21.196 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0049591239999999997 2024-09-11T06:51:21.197 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.197 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.197 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:21.197 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459988+0000", 2024-09-11T06:51:21.197 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000120532 2024-09-11T06:51:21.197 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.198 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.198 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:21.198 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460071+0000", 2024-09-11T06:51:21.198 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2256000000000007e-05 2024-09-11T06:51:21.198 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.198 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.198 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.198 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460135+0000", 2024-09-11T06:51:21.198 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3999999999999997e-05 2024-09-11T06:51:21.199 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.199 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.199 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.199 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.199 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:21.199 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:51:21.199 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 134.70116580199999, 2024-09-11T06:51:21.199 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.019036849000000002, 2024-09-11T06:51:21.200 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.200 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.200 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:21.200 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:21.200 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:21.200 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:21.200 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 115 2024-09-11T06:51:21.200 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.200 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.200 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.201 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.201 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:51:21.201 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.201 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.201 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.201 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:51:21.201 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.202 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.202 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.202 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441330+0000", 2024-09-11T06:51:21.202 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1930000000000001e-06 2024-09-11T06:51:21.202 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.202 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.202 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.202 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441544+0000", 2024-09-11T06:51:21.202 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021405799999999999 2024-09-11T06:51:21.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.203 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.203 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.203 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441572+0000", 2024-09-11T06:51:21.203 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7317e-05 2024-09-11T06:51:21.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.203 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.203 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.203 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441593+0000", 2024-09-11T06:51:21.204 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1702999999999999e-05 2024-09-11T06:51:21.204 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.204 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.204 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.204 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455891+0000", 2024-09-11T06:51:21.204 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01429708 2024-09-11T06:51:21.204 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.204 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.204 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.205 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456046+0000", 2024-09-11T06:51:21.205 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000155682 2024-09-11T06:51:21.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.205 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.205 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:21.205 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456242+0000", 2024-09-11T06:51:21.205 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000195692 2024-09-11T06:51:21.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.205 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.205 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:21.206 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459876+0000", 2024-09-11T06:51:21.206 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036336480000000002 2024-09-11T06:51:21.206 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.206 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.206 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:21.206 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460245+0000", 2024-09-11T06:51:21.206 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036901700000000002 2024-09-11T06:51:21.206 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.206 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.206 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:21.207 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460316+0000", 2024-09-11T06:51:21.207 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1846000000000003e-05 2024-09-11T06:51:21.207 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.207 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.207 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.207 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460366+0000", 2024-09-11T06:51:21.207 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9613000000000001e-05 2024-09-11T06:51:21.207 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.207 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.207 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.207 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.208 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.208 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:21.208 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:51:21.208 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 134.700843722, 2024-09-11T06:51:21.208 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01904432, 2024-09-11T06:51:21.208 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.208 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.208 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:21.208 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:21.208 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:21.208 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:21.209 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 116 2024-09-11T06:51:21.209 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.209 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.209 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.209 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.209 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:51:21.209 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.209 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.209 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.210 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.210 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:51:21.210 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.210 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.210 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.210 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.210 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441653+0000", 2024-09-11T06:51:21.210 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:51:21.211 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.211 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.211 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.211 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441665+0000", 2024-09-11T06:51:21.211 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1653e-05 2024-09-11T06:51:21.211 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.211 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.211 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.211 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441684+0000", 2024-09-11T06:51:21.212 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9490000000000001e-05 2024-09-11T06:51:21.212 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.212 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.212 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.212 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441699+0000", 2024-09-11T06:51:21.212 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5044e-05 2024-09-11T06:51:21.212 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.212 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.213 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.213 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457297+0000", 2024-09-11T06:51:21.213 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015598241000000001 2024-09-11T06:51:21.213 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.213 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.213 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.213 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457577+0000", 2024-09-11T06:51:21.213 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027930799999999998 2024-09-11T06:51:21.213 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.214 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.214 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:21.214 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457957+0000", 2024-09-11T06:51:21.214 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00038088 2024-09-11T06:51:21.214 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.214 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.214 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:21.214 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460432+0000", 2024-09-11T06:51:21.214 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024743349999999998 2024-09-11T06:51:21.215 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.215 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.215 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:21.215 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460584+0000", 2024-09-11T06:51:21.215 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015232900000000001 2024-09-11T06:51:21.215 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.215 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.215 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:21.216 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460646+0000", 2024-09-11T06:51:21.216 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1883000000000004e-05 2024-09-11T06:51:21.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.216 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.216 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.216 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460696+0000", 2024-09-11T06:51:21.216 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9589999999999998e-05 2024-09-11T06:51:21.216 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.216 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.216 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.216 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.217 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:21.217 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:51:21.217 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 134.700771656, 2024-09-11T06:51:21.217 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.021834473, 2024-09-11T06:51:21.217 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.217 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.217 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:21.217 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:21.217 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:21.217 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:21.217 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 117 2024-09-11T06:51:21.218 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.218 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.218 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.218 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.218 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:51:21.218 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.218 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.218 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.218 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.218 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:51:21.218 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.218 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.219 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.219 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.219 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441725+0000", 2024-09-11T06:51:21.219 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5069999999999999e-06 2024-09-11T06:51:21.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.219 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.219 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.219 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441738+0000", 2024-09-11T06:51:21.219 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2813000000000001e-05 2024-09-11T06:51:21.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.219 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.220 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.220 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441759+0000", 2024-09-11T06:51:21.220 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1246999999999998e-05 2024-09-11T06:51:21.220 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.220 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.220 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.220 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441779+0000", 2024-09-11T06:51:21.220 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0392999999999999e-05 2024-09-11T06:51:21.220 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.220 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.220 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.221 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458804+0000", 2024-09-11T06:51:21.221 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.017024182999999998 2024-09-11T06:51:21.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.221 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.221 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.221 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458977+0000", 2024-09-11T06:51:21.221 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000173926 2024-09-11T06:51:21.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.221 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.221 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:21.221 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459170+0000", 2024-09-11T06:51:21.221 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000192184 2024-09-11T06:51:21.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.222 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.222 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:21.222 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.462300+0000", 2024-09-11T06:51:21.222 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031304969999999999 2024-09-11T06:51:21.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.222 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.222 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:21.223 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463408+0000", 2024-09-11T06:51:21.223 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011077839999999999 2024-09-11T06:51:21.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.223 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:21.223 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463497+0000", 2024-09-11T06:51:21.223 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8828999999999999e-05 2024-09-11T06:51:21.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.223 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.223 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463558+0000", 2024-09-11T06:51:21.224 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1110000000000003e-05 2024-09-11T06:51:21.224 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.224 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.224 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.224 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.224 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:21.224 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:51:21.224 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 134.32413015399999, 2024-09-11T06:51:21.224 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013136297, 2024-09-11T06:51:21.224 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.225 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.225 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:21.225 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:21.225 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:21.225 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:21.225 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 138 2024-09-11T06:51:21.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.225 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.225 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.225 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:51:21.226 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.226 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.226 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.226 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:51:21.226 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.226 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.226 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.226 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:51:21.226 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.89e-07 2024-09-11T06:51:21.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.227 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.227 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.227 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818369+0000", 2024-09-11T06:51:21.227 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4170000000000001e-06 2024-09-11T06:51:21.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.227 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.227 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.227 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818376+0000", 2024-09-11T06:51:21.227 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4270000000000002e-06 2024-09-11T06:51:21.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.228 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.228 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818383+0000", 2024-09-11T06:51:21.228 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3060000000000003e-06 2024-09-11T06:51:21.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.228 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.228 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824783+0000", 2024-09-11T06:51:21.229 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064003020000000001 2024-09-11T06:51:21.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.229 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.229 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.229 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825013+0000", 2024-09-11T06:51:21.229 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023039499999999999 2024-09-11T06:51:21.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.229 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.229 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:21.229 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825484+0000", 2024-09-11T06:51:21.230 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000470562 2024-09-11T06:51:21.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.230 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.230 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:21.230 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828633+0000", 2024-09-11T06:51:21.230 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031496689999999999 2024-09-11T06:51:21.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.230 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.230 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:21.231 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831413+0000", 2024-09-11T06:51:21.231 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027793219999999999 2024-09-11T06:51:21.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.232 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.232 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:21.232 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831450+0000", 2024-09-11T06:51:21.232 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7370999999999998e-05 2024-09-11T06:51:21.233 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.233 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.233 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831501+0000", 2024-09-11T06:51:21.233 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1036999999999999e-05 2024-09-11T06:51:21.233 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.233 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.233 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.233 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.234 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:21.234 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:51:21.234 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 134.32410567400001, 2024-09-11T06:51:21.234 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013223011, 2024-09-11T06:51:21.234 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.234 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.234 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:21.234 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:21.234 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:21.234 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:21.235 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 139 2024-09-11T06:51:21.235 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.235 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.235 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.235 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.235 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:51:21.235 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.235 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.235 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.236 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.236 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:51:21.236 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.236 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.236 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.236 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.236 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818390+0000", 2024-09-11T06:51:21.236 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5600000000000001e-07 2024-09-11T06:51:21.236 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.236 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.237 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.237 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818394+0000", 2024-09-11T06:51:21.237 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5929999999999999e-06 2024-09-11T06:51:21.237 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.237 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.237 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.237 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818399+0000", 2024-09-11T06:51:21.237 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7740000000000004e-06 2024-09-11T06:51:21.237 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.238 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.238 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.238 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818412+0000", 2024-09-11T06:51:21.238 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2625e-05 2024-09-11T06:51:21.238 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.238 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.238 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.238 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826205+0000", 2024-09-11T06:51:21.239 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0077930730000000002 2024-09-11T06:51:21.239 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.239 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.239 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.239 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826281+0000", 2024-09-11T06:51:21.239 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5876000000000003e-05 2024-09-11T06:51:21.239 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.239 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.240 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:21.240 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826384+0000", 2024-09-11T06:51:21.240 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103838 2024-09-11T06:51:21.240 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.240 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.240 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:21.240 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830642+0000", 2024-09-11T06:51:21.240 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004257083 2024-09-11T06:51:21.241 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.241 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.241 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:21.241 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831509+0000", 2024-09-11T06:51:21.241 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086730900000000003 2024-09-11T06:51:21.241 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.241 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.242 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:21.242 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831547+0000", 2024-09-11T06:51:21.242 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7793000000000002e-05 2024-09-11T06:51:21.242 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.242 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.242 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.243 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831612+0000", 2024-09-11T06:51:21.243 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5691000000000005e-05 2024-09-11T06:51:21.243 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.243 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.243 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.243 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.243 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.243 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:21.244 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:51:21.244 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 134.32407698, 2024-09-11T06:51:21.244 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016098024999999998, 2024-09-11T06:51:21.244 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.244 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.244 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:21.244 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:21.244 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:21.245 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:21.245 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 140 2024-09-11T06:51:21.245 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.245 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.245 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.245 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.245 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:51:21.245 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.245 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.245 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.246 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.246 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:51:21.246 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.246 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.246 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.246 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.246 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:51:21.246 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5199999999999998e-07 2024-09-11T06:51:21.246 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.247 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.247 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.247 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818479+0000", 2024-09-11T06:51:21.247 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0368000000000002e-05 2024-09-11T06:51:21.247 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.247 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.247 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.247 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818487+0000", 2024-09-11T06:51:21.247 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2339999999999994e-06 2024-09-11T06:51:21.247 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.247 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.248 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.248 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818491+0000", 2024-09-11T06:51:21.248 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4329999999999997e-06 2024-09-11T06:51:21.248 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.248 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.248 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.248 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828080+0000", 2024-09-11T06:51:21.248 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0095890449999999992 2024-09-11T06:51:21.248 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.248 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.248 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.248 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828130+0000", 2024-09-11T06:51:21.249 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9239000000000001e-05 2024-09-11T06:51:21.249 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.249 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.249 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:21.249 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828208+0000", 2024-09-11T06:51:21.249 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.852e-05 2024-09-11T06:51:21.249 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.249 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.249 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:21.249 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831625+0000", 2024-09-11T06:51:21.249 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034169410000000002 2024-09-11T06:51:21.249 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.250 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.250 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:21.250 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834411+0000", 2024-09-11T06:51:21.250 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002786111 2024-09-11T06:51:21.250 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.250 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.250 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:21.250 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834457+0000", 2024-09-11T06:51:21.250 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6041000000000002e-05 2024-09-11T06:51:21.250 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.250 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.251 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.251 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834516+0000", 2024-09-11T06:51:21.251 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8740999999999999e-05 2024-09-11T06:51:21.251 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.251 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.251 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.251 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.251 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.251 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:21.251 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:51:21.251 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 134.323995913, 2024-09-11T06:51:21.251 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016588772000000002, 2024-09-11T06:51:21.252 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.252 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.252 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:21.252 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:21.252 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:21.252 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:21.252 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 141 2024-09-11T06:51:21.252 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.252 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.252 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.252 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.253 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:51:21.253 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.253 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.253 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.253 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.253 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:51:21.253 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.253 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.253 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.253 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.253 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818500+0000", 2024-09-11T06:51:21.253 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0800000000000005e-07 2024-09-11T06:51:21.253 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.254 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.254 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.254 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818505+0000", 2024-09-11T06:51:21.254 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2429999999999996e-06 2024-09-11T06:51:21.254 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.254 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.254 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.254 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818510+0000", 2024-09-11T06:51:21.254 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9339999999999997e-06 2024-09-11T06:51:21.254 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.254 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.255 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.255 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818514+0000", 2024-09-11T06:51:21.255 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7270000000000001e-06 2024-09-11T06:51:21.255 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.255 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.255 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.255 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828711+0000", 2024-09-11T06:51:21.255 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010197031 2024-09-11T06:51:21.255 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.255 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.255 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.256 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828853+0000", 2024-09-11T06:51:21.256 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014201200000000001 2024-09-11T06:51:21.256 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.256 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.256 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:21.256 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.829115+0000", 2024-09-11T06:51:21.256 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026257500000000003 2024-09-11T06:51:21.256 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.256 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.256 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:21.256 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831629+0000", 2024-09-11T06:51:21.257 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025138169999999998 2024-09-11T06:51:21.257 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.257 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.257 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:21.257 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834991+0000", 2024-09-11T06:51:21.257 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033624459999999998 2024-09-11T06:51:21.257 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.257 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.257 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:21.257 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835032+0000", 2024-09-11T06:51:21.258 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1001000000000003e-05 2024-09-11T06:51:21.258 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.258 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.258 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.258 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835088+0000", 2024-09-11T06:51:21.258 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5478e-05 2024-09-11T06:51:21.258 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.258 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.258 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.258 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.259 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.259 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5235.0:1 2.5 2:b2cc3336:::rbd_id.testrbdnbd21607:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-11T06:51:21.259 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:51:21.259 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 46.583193891000001, 2024-09-11T06:51:21.259 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.68353462499999995, 2024-09-11T06:51:21.259 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.259 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.259 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:21.259 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:21.260 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5235", 2024-09-11T06:51:21.260 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2008977164", 2024-09-11T06:51:21.260 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:51:21.260 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.260 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.260 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.260 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.260 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:51:21.260 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.261 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.261 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.261 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.261 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:51:21.261 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.261 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.261 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.261 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.262 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559303+0000", 2024-09-11T06:51:21.262 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1830000000000001e-06 2024-09-11T06:51:21.262 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.262 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.262 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.262 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559314+0000", 2024-09-11T06:51:21.262 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0396e-05 2024-09-11T06:51:21.262 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.262 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.263 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.263 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559335+0000", 2024-09-11T06:51:21.263 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1183e-05 2024-09-11T06:51:21.263 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.263 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.263 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.263 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559354+0000", 2024-09-11T06:51:21.263 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9225999999999999e-05 2024-09-11T06:51:21.263 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.263 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.264 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.264 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559522+0000", 2024-09-11T06:51:21.264 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000168021 2024-09-11T06:51:21.264 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.264 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.264 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:51:21.264 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559543+0000", 2024-09-11T06:51:21.264 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0616999999999999e-05 2024-09-11T06:51:21.264 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.265 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.265 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.265 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.238584+0000", 2024-09-11T06:51:21.265 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.67904107499999999 2024-09-11T06:51:21.265 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.265 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.265 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for missing object", 2024-09-11T06:51:21.265 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.238933+0000", 2024-09-11T06:51:21.265 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034879600000000001 2024-09-11T06:51:21.265 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.266 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.266 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.266 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.241968+0000", 2024-09-11T06:51:21.266 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030350830000000001 2024-09-11T06:51:21.266 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.266 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.266 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.266 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.242281+0000", 2024-09-11T06:51:21.266 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00031299499999999999 2024-09-11T06:51:21.267 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.267 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.267 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.267 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.242836+0000", 2024-09-11T06:51:21.267 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00055504999999999997 2024-09-11T06:51:21.267 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.267 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.267 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.267 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.268 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.268 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5235.0:18 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 0~1024 [fadvise_sequential]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T06:51:21.268 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:51:21.268 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 45.880701451999997, 2024-09-11T06:51:21.268 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.048993760999999997, 2024-09-11T06:51:21.268 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.268 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.268 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:21.268 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:21.269 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5235", 2024-09-11T06:51:21.269 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2008977164", 2024-09-11T06:51:21.269 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 18 2024-09-11T06:51:21.269 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.269 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.269 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.269 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.269 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:51:21.269 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.270 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.270 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.270 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.270 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:51:21.270 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.270 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.270 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.270 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.270 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261797+0000", 2024-09-11T06:51:21.270 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2059999999999999e-06 2024-09-11T06:51:21.270 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.271 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.271 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.271 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261812+0000", 2024-09-11T06:51:21.271 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5159e-05 2024-09-11T06:51:21.271 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.271 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.271 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.271 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261852+0000", 2024-09-11T06:51:21.271 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.9906e-05 2024-09-11T06:51:21.271 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.271 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.271 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.272 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261865+0000", 2024-09-11T06:51:21.272 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.312e-05 2024-09-11T06:51:21.272 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.272 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.272 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.272 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.263166+0000", 2024-09-11T06:51:21.272 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001301134 2024-09-11T06:51:21.272 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.273 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.273 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for missing object", 2024-09-11T06:51:21.273 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.263529+0000", 2024-09-11T06:51:21.273 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036279900000000002 2024-09-11T06:51:21.273 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.273 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.273 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.273 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.300567+0000", 2024-09-11T06:51:21.273 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.037038326000000003 2024-09-11T06:51:21.274 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.274 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.274 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:21.274 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.300800+0000", 2024-09-11T06:51:21.274 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023323300000000001 2024-09-11T06:51:21.274 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.274 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.274 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.274 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.306709+0000", 2024-09-11T06:51:21.274 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0059087690000000003 2024-09-11T06:51:21.275 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.275 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.275 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.275 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.306850+0000", 2024-09-11T06:51:21.275 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014106200000000001 2024-09-11T06:51:21.275 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.275 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.275 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:21.275 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.307121+0000", 2024-09-11T06:51:21.275 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027083599999999999 2024-09-11T06:51:21.275 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.275 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.276 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:21.276 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.308809+0000", 2024-09-11T06:51:21.276 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0016880759999999999 2024-09-11T06:51:21.276 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.276 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.276 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:21.276 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310382+0000", 2024-09-11T06:51:21.276 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0015732840000000001 2024-09-11T06:51:21.276 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.276 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.276 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:21.277 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310501+0000", 2024-09-11T06:51:21.277 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000118276 2024-09-11T06:51:21.277 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.277 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.277 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.277 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310787+0000", 2024-09-11T06:51:21.277 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00028657500000000001 2024-09-11T06:51:21.277 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.277 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.277 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.277 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.277 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.278 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:70 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 393216~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:21.278 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596118+0000", 2024-09-11T06:51:21.278 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 32.546377049999997, 2024-09-11T06:51:21.278 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012279145, 2024-09-11T06:51:21.278 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.278 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.278 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:21.278 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:21.278 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:21.278 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:21.278 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 70 2024-09-11T06:51:21.278 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.279 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.279 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.279 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.279 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596118+0000", 2024-09-11T06:51:21.279 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.279 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.279 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.279 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.279 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596118+0000", 2024-09-11T06:51:21.279 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.279 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.279 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.280 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.280 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596119+0000", 2024-09-11T06:51:21.280 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0299999999999997e-07 2024-09-11T06:51:21.280 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.280 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.280 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.280 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596238+0000", 2024-09-11T06:51:21.280 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000118798 2024-09-11T06:51:21.280 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.280 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.280 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.280 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596250+0000", 2024-09-11T06:51:21.281 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2775999999999999e-05 2024-09-11T06:51:21.281 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.281 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.281 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.281 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596259+0000", 2024-09-11T06:51:21.281 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3100000000000001e-06 2024-09-11T06:51:21.281 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.281 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.281 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.281 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.603389+0000", 2024-09-11T06:51:21.281 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0071300759999999999 2024-09-11T06:51:21.282 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.282 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.282 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.282 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.603540+0000", 2024-09-11T06:51:21.282 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015111700000000001 2024-09-11T06:51:21.282 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.282 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.282 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:21.282 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.603766+0000", 2024-09-11T06:51:21.282 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022570899999999999 2024-09-11T06:51:21.282 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.282 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.282 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:21.283 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.606676+0000", 2024-09-11T06:51:21.283 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0029106380000000001 2024-09-11T06:51:21.283 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.283 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.283 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:21.283 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608173+0000", 2024-09-11T06:51:21.283 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014967629999999999 2024-09-11T06:51:21.283 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.283 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.283 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:21.283 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608277+0000", 2024-09-11T06:51:21.283 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000104398 2024-09-11T06:51:21.284 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.284 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.284 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.284 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608397+0000", 2024-09-11T06:51:21.284 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000119657 2024-09-11T06:51:21.284 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.284 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.284 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.284 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.284 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.284 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:71 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 524288~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:21.285 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596272+0000", 2024-09-11T06:51:21.285 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 32.546222796999999, 2024-09-11T06:51:21.285 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013390681999999999, 2024-09-11T06:51:21.285 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.285 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.285 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:21.285 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:21.285 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:21.285 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:21.285 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 71 2024-09-11T06:51:21.285 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.286 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.286 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.286 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.286 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596272+0000", 2024-09-11T06:51:21.286 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.286 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.286 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.286 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.286 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596272+0000", 2024-09-11T06:51:21.286 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.286 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.287 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.287 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.287 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596273+0000", 2024-09-11T06:51:21.287 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.9999999999999996e-07 2024-09-11T06:51:21.287 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.287 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.287 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.287 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596395+0000", 2024-09-11T06:51:21.287 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012204799999999999 2024-09-11T06:51:21.287 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.287 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.288 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.288 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596411+0000", 2024-09-11T06:51:21.288 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5746000000000001e-05 2024-09-11T06:51:21.288 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.288 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.288 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.288 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596419+0000", 2024-09-11T06:51:21.288 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2600000000000005e-06 2024-09-11T06:51:21.288 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.288 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.289 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.289 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.605311+0000", 2024-09-11T06:51:21.289 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0088915179999999993 2024-09-11T06:51:21.289 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.289 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.289 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.289 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.605436+0000", 2024-09-11T06:51:21.289 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000125203 2024-09-11T06:51:21.289 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.290 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.290 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:21.290 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.605625+0000", 2024-09-11T06:51:21.290 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018961299999999999 2024-09-11T06:51:21.290 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.290 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.290 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:21.290 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609477+0000", 2024-09-11T06:51:21.290 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0038513549999999999 2024-09-11T06:51:21.290 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.291 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.291 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:21.291 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609499+0000", 2024-09-11T06:51:21.291 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2215e-05 2024-09-11T06:51:21.291 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.291 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.291 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:21.291 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609589+0000", 2024-09-11T06:51:21.291 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0029000000000001e-05 2024-09-11T06:51:21.292 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.292 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.292 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.292 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609663+0000", 2024-09-11T06:51:21.292 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3894999999999996e-05 2024-09-11T06:51:21.292 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.292 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.292 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.293 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.293 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.293 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:72 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 655360~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:21.293 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596432+0000", 2024-09-11T06:51:21.293 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 32.546063310000001, 2024-09-11T06:51:21.293 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01414727, 2024-09-11T06:51:21.293 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.293 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.293 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:21.294 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:21.294 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:21.294 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:21.294 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 72 2024-09-11T06:51:21.294 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.294 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.294 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.294 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.294 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596432+0000", 2024-09-11T06:51:21.295 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.295 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.295 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.295 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.295 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596432+0000", 2024-09-11T06:51:21.295 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.295 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.295 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.295 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.296 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596433+0000", 2024-09-11T06:51:21.296 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1999999999999998e-07 2024-09-11T06:51:21.296 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.296 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.296 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.296 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596556+0000", 2024-09-11T06:51:21.296 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012379000000000001 2024-09-11T06:51:21.296 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.296 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.296 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.297 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596571+0000", 2024-09-11T06:51:21.297 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4749999999999999e-05 2024-09-11T06:51:21.297 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.297 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.297 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.298 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596579+0000", 2024-09-11T06:51:21.298 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2500000000000006e-06 2024-09-11T06:51:21.298 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.298 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.298 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.298 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.606784+0000", 2024-09-11T06:51:21.298 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010204757 2024-09-11T06:51:21.298 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.298 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.299 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.299 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.606950+0000", 2024-09-11T06:51:21.299 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000166229 2024-09-11T06:51:21.299 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.299 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.299 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:21.299 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.607176+0000", 2024-09-11T06:51:21.299 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022549899999999999 2024-09-11T06:51:21.299 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.300 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.300 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:21.300 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610363+0000", 2024-09-11T06:51:21.300 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031874690000000001 2024-09-11T06:51:21.300 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.300 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.300 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:21.300 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610490+0000", 2024-09-11T06:51:21.301 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012639899999999999 2024-09-11T06:51:21.301 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.301 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.301 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:21.301 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610529+0000", 2024-09-11T06:51:21.301 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.8936000000000001e-05 2024-09-11T06:51:21.301 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.301 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.301 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.301 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610579+0000", 2024-09-11T06:51:21.302 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0371000000000003e-05 2024-09-11T06:51:21.302 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.302 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.302 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.302 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.302 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.302 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:73 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 786432~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:21.302 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596592+0000", 2024-09-11T06:51:21.302 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 32.545903056999997, 2024-09-11T06:51:21.302 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014914779, 2024-09-11T06:51:21.303 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.303 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.303 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:21.303 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:21.303 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:21.303 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:21.303 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 73 2024-09-11T06:51:21.303 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.303 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.304 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.304 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.304 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596592+0000", 2024-09-11T06:51:21.304 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.304 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.304 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.304 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.304 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596592+0000", 2024-09-11T06:51:21.304 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.305 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.305 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.305 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.305 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596593+0000", 2024-09-11T06:51:21.305 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.0699999999999996e-07 2024-09-11T06:51:21.305 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.305 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.305 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.306 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596786+0000", 2024-09-11T06:51:21.306 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019278200000000001 2024-09-11T06:51:21.306 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.306 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.306 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.306 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596801+0000", 2024-09-11T06:51:21.306 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5690000000000001e-05 2024-09-11T06:51:21.306 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.306 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.306 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.306 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596815+0000", 2024-09-11T06:51:21.306 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3753e-05 2024-09-11T06:51:21.307 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.307 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.307 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.307 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608442+0000", 2024-09-11T06:51:21.307 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011627385 2024-09-11T06:51:21.307 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.307 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.307 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.307 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608570+0000", 2024-09-11T06:51:21.307 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012740399999999999 2024-09-11T06:51:21.307 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.308 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.308 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:21.308 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608754+0000", 2024-09-11T06:51:21.308 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018392899999999999 2024-09-11T06:51:21.308 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.308 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.308 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:21.308 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610370+0000", 2024-09-11T06:51:21.308 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001615998 2024-09-11T06:51:21.308 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.308 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.309 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:21.309 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611420+0000", 2024-09-11T06:51:21.309 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010502790000000001 2024-09-11T06:51:21.309 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.309 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.309 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:21.309 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611458+0000", 2024-09-11T06:51:21.309 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7749000000000001e-05 2024-09-11T06:51:21.309 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.309 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.309 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.310 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611507+0000", 2024-09-11T06:51:21.310 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9002999999999998e-05 2024-09-11T06:51:21.310 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.310 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.310 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.310 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.310 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.310 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:74 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 917504~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:21.310 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596849+0000", 2024-09-11T06:51:21.310 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 32.545646419000001, 2024-09-11T06:51:21.310 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015202963999999999, 2024-09-11T06:51:21.310 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.311 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.311 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:21.311 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:21.311 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:21.311 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:21.311 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 74 2024-09-11T06:51:21.311 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.311 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.311 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.311 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.311 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596849+0000", 2024-09-11T06:51:21.311 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.312 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.312 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.312 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.312 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596849+0000", 2024-09-11T06:51:21.312 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.312 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.312 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.312 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.312 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596850+0000", 2024-09-11T06:51:21.312 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3170000000000001e-06 2024-09-11T06:51:21.312 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.312 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.313 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.313 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.597118+0000", 2024-09-11T06:51:21.313 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026772999999999999 2024-09-11T06:51:21.313 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.313 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.313 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.313 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.597136+0000", 2024-09-11T06:51:21.313 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8467000000000001e-05 2024-09-11T06:51:21.313 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.313 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.313 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.314 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.597148+0000", 2024-09-11T06:51:21.314 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1486e-05 2024-09-11T06:51:21.314 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.314 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.314 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.314 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609714+0000", 2024-09-11T06:51:21.314 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012566124 2024-09-11T06:51:21.314 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.314 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.314 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.314 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609792+0000", 2024-09-11T06:51:21.314 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.8579999999999996e-05 2024-09-11T06:51:21.315 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.315 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.315 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:21.315 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609899+0000", 2024-09-11T06:51:21.315 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00010716499999999999 2024-09-11T06:51:21.315 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.315 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.315 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:21.315 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610865+0000", 2024-09-11T06:51:21.315 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00096540299999999999 2024-09-11T06:51:21.316 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.316 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.316 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:21.316 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611970+0000", 2024-09-11T06:51:21.316 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001105118 2024-09-11T06:51:21.316 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.316 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.316 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:21.316 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.612005+0000", 2024-09-11T06:51:21.316 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4789000000000002e-05 2024-09-11T06:51:21.317 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.317 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.317 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.317 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.612052+0000", 2024-09-11T06:51:21.317 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6785e-05 2024-09-11T06:51:21.317 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.317 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.317 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.317 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.318 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.318 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:21.320 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:51:21.320 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:21.313+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:21.420 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:21.414+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:21.493 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:21.493 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:21.494 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:51:21.494 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:21.507 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:51:21.521 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:21.518+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:21.622 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:21.618+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:21.630 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:21.630 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:21.630 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:51:21.630 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:51:21.630 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:21.645 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:51:21.723 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:21.717+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:21.769 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:21.770 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:51:21.770 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:51:21.770 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:51:21.770 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.770 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4558.0:1 3.3 3:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-11T06:51:21.770 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:51:21.770 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 64.675687044, 2024-09-11T06:51:21.770 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0085520600000000002, 2024-09-11T06:51:21.770 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.770 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.771 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:21.771 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:21.771 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4558", 2024-09-11T06:51:21.771 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1478388898", 2024-09-11T06:51:21.771 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:51:21.771 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.771 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.771 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.771 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.771 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:51:21.771 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.771 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.771 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.772 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.772 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:51:21.772 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.772 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.772 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.772 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.772 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090533+0000", 2024-09-11T06:51:21.772 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2629999999999999e-06 2024-09-11T06:51:21.772 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.772 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.772 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.772 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090546+0000", 2024-09-11T06:51:21.773 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2918999999999999e-05 2024-09-11T06:51:21.773 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.773 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.773 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.773 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090572+0000", 2024-09-11T06:51:21.773 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6412000000000001e-05 2024-09-11T06:51:21.773 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.773 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.773 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.773 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090594+0000", 2024-09-11T06:51:21.773 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2476e-05 2024-09-11T06:51:21.773 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.773 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.773 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.774 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090938+0000", 2024-09-11T06:51:21.774 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034325099999999998 2024-09-11T06:51:21.774 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.774 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.774 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.774 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.091149+0000", 2024-09-11T06:51:21.774 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021151300000000001 2024-09-11T06:51:21.774 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.774 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.774 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:21.774 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.091377+0000", 2024-09-11T06:51:21.774 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022785399999999999 2024-09-11T06:51:21.774 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.774 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.775 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:21.775 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.092912+0000", 2024-09-11T06:51:21.775 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0015346089999999999 2024-09-11T06:51:21.775 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.775 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.775 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:21.775 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098919+0000", 2024-09-11T06:51:21.775 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0060076799999999996 2024-09-11T06:51:21.775 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.775 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.775 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:21.775 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.099009+0000", 2024-09-11T06:51:21.775 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0056000000000006e-05 2024-09-11T06:51:21.776 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.776 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.776 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.776 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.099081+0000", 2024-09-11T06:51:21.776 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2026999999999993e-05 2024-09-11T06:51:21.776 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.776 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.776 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.776 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.776 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.776 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:51:21.776 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:51:21.777 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 58.634197964000002, 2024-09-11T06:51:21.777 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0071924390000000001, 2024-09-11T06:51:21.777 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.777 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.777 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:21.777 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.777 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.777 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.777 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:51:21.777 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.777 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.778 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.778 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.778 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:51:21.778 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.778 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.778 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.778 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.778 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132020+0000", 2024-09-11T06:51:21.778 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.846e-06 2024-09-11T06:51:21.778 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.778 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.778 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.779 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135688+0000", 2024-09-11T06:51:21.779 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036677350000000001 2024-09-11T06:51:21.779 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.779 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.779 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.779 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135747+0000", 2024-09-11T06:51:21.779 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.9314999999999997e-05 2024-09-11T06:51:21.779 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.779 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.779 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.779 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135797+0000", 2024-09-11T06:51:21.779 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9984000000000001e-05 2024-09-11T06:51:21.780 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.780 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.780 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.780 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136143+0000", 2024-09-11T06:51:21.780 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034594299999999998 2024-09-11T06:51:21.780 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.780 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.780 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.780 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136179+0000", 2024-09-11T06:51:21.780 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5636000000000002e-05 2024-09-11T06:51:21.780 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.780 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.781 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.781 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.139211+0000", 2024-09-11T06:51:21.781 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030319800000000001 2024-09-11T06:51:21.781 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.781 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.781 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.781 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.781 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.781 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:51:21.781 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:51:21.781 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 58.632134710000003, 2024-09-11T06:51:21.781 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064954139999999997, 2024-09-11T06:51:21.781 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.782 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.782 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:21.782 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.782 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.782 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.782 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:51:21.782 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.782 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.782 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.782 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.782 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:51:21.782 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.782 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.783 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.783 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.783 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134084+0000", 2024-09-11T06:51:21.783 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.894e-06 2024-09-11T06:51:21.783 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.783 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.783 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.783 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136513+0000", 2024-09-11T06:51:21.783 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024290969999999999 2024-09-11T06:51:21.783 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.783 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.784 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.784 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136554+0000", 2024-09-11T06:51:21.784 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.0932e-05 2024-09-11T06:51:21.784 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.784 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.784 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.784 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136584+0000", 2024-09-11T06:51:21.784 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0056000000000002e-05 2024-09-11T06:51:21.784 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.784 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.784 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.785 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136805+0000", 2024-09-11T06:51:21.785 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022086100000000001 2024-09-11T06:51:21.785 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.785 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.785 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.785 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136864+0000", 2024-09-11T06:51:21.785 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.9907000000000001e-05 2024-09-11T06:51:21.785 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.785 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.785 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.785 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.140577+0000", 2024-09-11T06:51:21.786 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0037126669999999998 2024-09-11T06:51:21.786 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.786 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.786 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.786 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.786 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.786 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:21.786 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:51:21.786 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 58.623843909999998, 2024-09-11T06:51:21.786 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035763180000000002, 2024-09-11T06:51:21.786 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.787 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.787 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:21.787 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.787 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.787 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.787 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:51:21.787 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.787 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.787 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.787 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.787 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:51:21.788 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.788 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.788 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.788 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.788 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142374+0000", 2024-09-11T06:51:21.788 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5460000000000001e-06 2024-09-11T06:51:21.788 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.788 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.788 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.788 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143902+0000", 2024-09-11T06:51:21.788 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00152808 2024-09-11T06:51:21.789 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.789 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.789 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.789 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143946+0000", 2024-09-11T06:51:21.789 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3874e-05 2024-09-11T06:51:21.789 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.789 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.789 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.789 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143978+0000", 2024-09-11T06:51:21.789 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2425999999999998e-05 2024-09-11T06:51:21.789 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.789 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.790 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.790 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.144177+0000", 2024-09-11T06:51:21.790 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019837500000000001 2024-09-11T06:51:21.790 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.790 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.790 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.790 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.144198+0000", 2024-09-11T06:51:21.790 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1639000000000001e-05 2024-09-11T06:51:21.790 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.790 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.790 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.791 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.145949+0000", 2024-09-11T06:51:21.791 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0017503779999999999 2024-09-11T06:51:21.791 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.791 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.791 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.791 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.791 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.791 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:21.791 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:51:21.791 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 58.619528207999998, 2024-09-11T06:51:21.791 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0057693220000000003, 2024-09-11T06:51:21.792 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.792 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.792 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:21.792 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.792 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.792 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.792 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:51:21.792 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.792 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.792 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.792 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.792 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:51:21.793 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.793 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.793 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.793 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.793 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146690+0000", 2024-09-11T06:51:21.793 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.607e-06 2024-09-11T06:51:21.793 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.793 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.793 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.793 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148693+0000", 2024-09-11T06:51:21.793 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0020029739999999998 2024-09-11T06:51:21.794 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.794 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.794 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.794 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148735+0000", 2024-09-11T06:51:21.794 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1958e-05 2024-09-11T06:51:21.794 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.794 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.794 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.794 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148764+0000", 2024-09-11T06:51:21.794 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8999000000000001e-05 2024-09-11T06:51:21.794 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.794 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.795 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.795 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149069+0000", 2024-09-11T06:51:21.795 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000305032 2024-09-11T06:51:21.795 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.795 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.795 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.795 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149204+0000", 2024-09-11T06:51:21.795 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013495 2024-09-11T06:51:21.795 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.795 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.795 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.796 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.152458+0000", 2024-09-11T06:51:21.796 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0032538020000000001 2024-09-11T06:51:21.796 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.796 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.796 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.796 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.796 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.796 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:21.796 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:51:21.796 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 58.618629667, 2024-09-11T06:51:21.796 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0042425329999999997, 2024-09-11T06:51:21.796 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.796 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.797 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:21.797 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.797 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.797 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.797 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:51:21.797 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.797 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.797 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.797 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.797 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:51:21.797 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.797 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.797 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.798 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.798 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147588+0000", 2024-09-11T06:51:21.798 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.277e-06 2024-09-11T06:51:21.798 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.798 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.798 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.798 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149051+0000", 2024-09-11T06:51:21.798 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001462669 2024-09-11T06:51:21.798 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.798 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.798 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.798 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149090+0000", 2024-09-11T06:51:21.798 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.8967999999999997e-05 2024-09-11T06:51:21.799 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.799 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.799 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.799 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149120+0000", 2024-09-11T06:51:21.799 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0766000000000003e-05 2024-09-11T06:51:21.799 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.799 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.799 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.799 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149395+0000", 2024-09-11T06:51:21.799 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027484900000000002 2024-09-11T06:51:21.799 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.799 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.799 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.800 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149439+0000", 2024-09-11T06:51:21.800 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3940999999999998e-05 2024-09-11T06:51:21.800 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.800 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.800 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.800 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.151829+0000", 2024-09-11T06:51:21.800 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002390063 2024-09-11T06:51:21.800 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.800 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.800 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.800 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.800 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.800 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:21.801 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:51:21.801 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 58.612489035999999, 2024-09-11T06:51:21.801 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0046568219999999997, 2024-09-11T06:51:21.801 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.801 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.801 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:21.801 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.801 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.801 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.801 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:51:21.801 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.801 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.801 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.801 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.802 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:51:21.802 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.802 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.802 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.802 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.802 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153729+0000", 2024-09-11T06:51:21.802 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5629999999999999e-06 2024-09-11T06:51:21.802 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.802 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.802 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.802 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155179+0000", 2024-09-11T06:51:21.802 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001450469 2024-09-11T06:51:21.802 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.802 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.803 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.803 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155221+0000", 2024-09-11T06:51:21.803 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1448e-05 2024-09-11T06:51:21.803 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.803 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.803 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.803 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155249+0000", 2024-09-11T06:51:21.803 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8433e-05 2024-09-11T06:51:21.803 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.803 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.803 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.803 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155420+0000", 2024-09-11T06:51:21.803 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017070599999999999 2024-09-11T06:51:21.803 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.804 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.804 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.804 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155445+0000", 2024-09-11T06:51:21.804 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4813000000000001e-05 2024-09-11T06:51:21.804 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.804 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.804 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.804 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.158384+0000", 2024-09-11T06:51:21.804 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0029393900000000001 2024-09-11T06:51:21.804 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.804 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.804 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.804 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.805 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.805 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:21.805 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:51:21.805 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 58.610462286999997, 2024-09-11T06:51:21.805 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0046826020000000001, 2024-09-11T06:51:21.805 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.805 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.805 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:21.805 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.805 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.805 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.805 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:51:21.805 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.806 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.806 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.806 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.806 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:51:21.806 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.806 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.806 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.806 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.806 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155756+0000", 2024-09-11T06:51:21.806 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.384e-06 2024-09-11T06:51:21.806 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.807 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.807 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.807 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157205+0000", 2024-09-11T06:51:21.807 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014496260000000001 2024-09-11T06:51:21.807 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.807 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.807 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.807 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157252+0000", 2024-09-11T06:51:21.807 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6368000000000001e-05 2024-09-11T06:51:21.807 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.807 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.807 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.808 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157286+0000", 2024-09-11T06:51:21.808 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4944999999999998e-05 2024-09-11T06:51:21.808 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.808 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.808 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.808 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157540+0000", 2024-09-11T06:51:21.808 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00025307999999999998 2024-09-11T06:51:21.808 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.808 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.808 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.809 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157576+0000", 2024-09-11T06:51:21.809 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.6878999999999997e-05 2024-09-11T06:51:21.809 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.809 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.809 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.809 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.160437+0000", 2024-09-11T06:51:21.809 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00286032 2024-09-11T06:51:21.809 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.809 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.809 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.809 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.810 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.810 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:51:21.810 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.280679+0000", 2024-09-11T06:51:21.810 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 46.485537297999997, 2024-09-11T06:51:21.810 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0040934309999999998, 2024-09-11T06:51:21.810 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.810 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.810 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:51:21.810 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.810 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.810 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.811 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280679+0000", 2024-09-11T06:51:21.811 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.811 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.811 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.811 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.811 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280679+0000", 2024-09-11T06:51:21.811 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.811 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.811 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.811 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.811 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280681+0000", 2024-09-11T06:51:21.812 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2529999999999999e-06 2024-09-11T06:51:21.812 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.812 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.812 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.812 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280713+0000", 2024-09-11T06:51:21.812 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.1936000000000001e-05 2024-09-11T06:51:21.812 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.812 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.812 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.812 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280758+0000", 2024-09-11T06:51:21.812 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5124999999999999e-05 2024-09-11T06:51:21.813 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.813 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.813 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.813 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280779+0000", 2024-09-11T06:51:21.813 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0936000000000001e-05 2024-09-11T06:51:21.813 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.813 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.813 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.813 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280932+0000", 2024-09-11T06:51:21.813 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015220899999999999 2024-09-11T06:51:21.813 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.814 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.814 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.814 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.284773+0000", 2024-09-11T06:51:21.814 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0038409719999999998 2024-09-11T06:51:21.814 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.814 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.814 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.814 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.814 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.814 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:51:21.814 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.293438+0000", 2024-09-11T06:51:21.814 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 46.472778562000002, 2024-09-11T06:51:21.815 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0061479350000000002, 2024-09-11T06:51:21.815 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.815 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.815 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:51:21.815 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.815 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.815 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.815 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293438+0000", 2024-09-11T06:51:21.815 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.815 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.815 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.815 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.816 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293438+0000", 2024-09-11T06:51:21.816 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.816 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.816 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.816 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.816 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293439+0000", 2024-09-11T06:51:21.816 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:51:21.816 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.816 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.816 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.816 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293459+0000", 2024-09-11T06:51:21.817 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9922e-05 2024-09-11T06:51:21.817 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.817 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.817 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.817 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293482+0000", 2024-09-11T06:51:21.817 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2969e-05 2024-09-11T06:51:21.817 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.817 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.817 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.817 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293494+0000", 2024-09-11T06:51:21.817 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.131e-05 2024-09-11T06:51:21.818 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.818 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.818 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.818 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.295901+0000", 2024-09-11T06:51:21.818 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024077949999999999 2024-09-11T06:51:21.818 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.818 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.818 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.818 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.299586+0000", 2024-09-11T06:51:21.818 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036845020000000001 2024-09-11T06:51:21.818 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.819 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.819 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.819 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.819 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.819 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(client.5239.0:37 2.7 e91/69 2:eab0161e:::rbd_data.1314f0a2bfa2.0000000000000001:head v 91'134, mlcod=91'130)", 2024-09-11T06:51:21.819 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:08.068368+0000", 2024-09-11T06:51:21.819 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 13.697848484, 2024-09-11T06:51:21.819 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003850118, 2024-09-11T06:51:21.819 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.819 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.819 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:21.820 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.820 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.820 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.820 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068368+0000", 2024-09-11T06:51:21.820 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.820 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.820 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.820 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.820 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068368+0000", 2024-09-11T06:51:21.820 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.820 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.820 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.821 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.821 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068369+0000", 2024-09-11T06:51:21.821 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2699999999999998e-07 2024-09-11T06:51:21.821 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.821 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.821 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.821 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068458+0000", 2024-09-11T06:51:21.821 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9075999999999999e-05 2024-09-11T06:51:21.821 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.821 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.821 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.821 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068471+0000", 2024-09-11T06:51:21.822 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.269e-05 2024-09-11T06:51:21.822 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.822 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.822 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.822 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068480+0000", 2024-09-11T06:51:21.822 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.6199999999999994e-06 2024-09-11T06:51:21.822 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.822 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.822 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.822 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.069523+0000", 2024-09-11T06:51:21.822 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001042465 2024-09-11T06:51:21.822 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.823 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.823 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.823 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.069542+0000", 2024-09-11T06:51:21.823 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9386e-05 2024-09-11T06:51:21.823 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.823 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.823 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:21.823 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072122+0000", 2024-09-11T06:51:21.823 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025799540000000002 2024-09-11T06:51:21.823 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.823 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.824 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.824 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072218+0000", 2024-09-11T06:51:21.824 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.6100000000000005e-05 2024-09-11T06:51:21.824 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.824 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.824 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.824 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.824 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.824 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(client.5239.0:38 2.7 e91/69 2:eab0161e:::rbd_data.1314f0a2bfa2.0000000000000001:head v 91'135, mlcod=91'130)", 2024-09-11T06:51:21.824 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:08.069503+0000", 2024-09-11T06:51:21.824 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 13.696713709000001, 2024-09-11T06:51:21.824 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003356295, 2024-09-11T06:51:21.824 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.825 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.825 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:21.825 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.825 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.825 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.825 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.069503+0000", 2024-09-11T06:51:21.825 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.825 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.825 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.825 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.825 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.069503+0000", 2024-09-11T06:51:21.825 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.825 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.825 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.826 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.826 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.069504+0000", 2024-09-11T06:51:21.826 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.178e-06 2024-09-11T06:51:21.826 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.826 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.826 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.826 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.069638+0000", 2024-09-11T06:51:21.826 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013433499999999999 2024-09-11T06:51:21.826 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.826 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.826 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.826 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.069655+0000", 2024-09-11T06:51:21.826 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6359e-05 2024-09-11T06:51:21.826 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.827 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.827 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.827 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.069664+0000", 2024-09-11T06:51:21.827 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9049999999999992e-06 2024-09-11T06:51:21.827 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.827 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.827 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.827 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.070018+0000", 2024-09-11T06:51:21.827 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00035356100000000001 2024-09-11T06:51:21.827 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.827 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.827 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.827 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.070042+0000", 2024-09-11T06:51:21.828 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4119999999999999e-05 2024-09-11T06:51:21.828 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.828 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.828 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:21.828 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072772+0000", 2024-09-11T06:51:21.828 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027300929999999998 2024-09-11T06:51:21.828 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.828 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.828 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.828 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072859+0000", 2024-09-11T06:51:21.828 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6743999999999995e-05 2024-09-11T06:51:21.828 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.829 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.829 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.829 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.829 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.829 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(client.5239.0:41 2.7 e91/69 2:eab0161e:::rbd_data.1314f0a2bfa2.0000000000000001:head v 91'138, mlcod=91'132)", 2024-09-11T06:51:21.829 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:08.071899+0000", 2024-09-11T06:51:21.829 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 13.694317706, 2024-09-11T06:51:21.829 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0037511240000000002, 2024-09-11T06:51:21.829 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.829 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.829 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:21.829 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.830 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.830 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.830 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.071899+0000", 2024-09-11T06:51:21.830 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.830 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.830 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.830 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.830 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.071899+0000", 2024-09-11T06:51:21.830 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.830 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.830 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.830 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.830 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.071900+0000", 2024-09-11T06:51:21.830 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3500000000000005e-07 2024-09-11T06:51:21.830 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.831 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.831 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.831 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072027+0000", 2024-09-11T06:51:21.831 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012720299999999999 2024-09-11T06:51:21.831 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.831 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.831 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.831 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072043+0000", 2024-09-11T06:51:21.831 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6475999999999999e-05 2024-09-11T06:51:21.831 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.831 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.831 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.832 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072053+0000", 2024-09-11T06:51:21.832 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2860000000000005e-06 2024-09-11T06:51:21.832 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.832 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.832 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.832 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072256+0000", 2024-09-11T06:51:21.832 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000203876 2024-09-11T06:51:21.832 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.832 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.832 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.832 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072282+0000", 2024-09-11T06:51:21.832 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5811000000000001e-05 2024-09-11T06:51:21.832 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.833 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.833 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:21.833 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.075544+0000", 2024-09-11T06:51:21.833 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0032621500000000001 2024-09-11T06:51:21.833 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.833 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.833 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.833 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.075650+0000", 2024-09-11T06:51:21.833 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000105487 2024-09-11T06:51:21.833 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.833 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.834 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.834 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.834 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.834 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.4 101/100 [PushOp(2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head, version: 54'23, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head@54'23, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:51:21.834 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.525431+0000", 2024-09-11T06:51:21.834 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 1.240785144, 2024-09-11T06:51:21.834 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0042697100000000003, 2024-09-11T06:51:21.834 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.834 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.834 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:21.835 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.835 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.835 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.835 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.525431+0000", 2024-09-11T06:51:21.835 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.835 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.835 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.835 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.835 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.525431+0000", 2024-09-11T06:51:21.835 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.836 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.836 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.836 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.836 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.525433+0000", 2024-09-11T06:51:21.836 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5829999999999999e-06 2024-09-11T06:51:21.836 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.836 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.836 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.836 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.527430+0000", 2024-09-11T06:51:21.836 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0019966350000000001 2024-09-11T06:51:21.837 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.837 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.837 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.837 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.527477+0000", 2024-09-11T06:51:21.837 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7681e-05 2024-09-11T06:51:21.837 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.837 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.837 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.837 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.527525+0000", 2024-09-11T06:51:21.838 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.8244000000000001e-05 2024-09-11T06:51:21.838 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.838 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.838 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.838 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.527795+0000", 2024-09-11T06:51:21.838 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026935199999999999 2024-09-11T06:51:21.838 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.838 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.838 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.838 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.527816+0000", 2024-09-11T06:51:21.839 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1112e-05 2024-09-11T06:51:21.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.839 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.839 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.839 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.529701+0000", 2024-09-11T06:51:21.839 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001885103 2024-09-11T06:51:21.839 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.839 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.839 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.840 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.840 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.840 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.4 101/100 [PushOp(2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head, version: 54'23, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head@54'23, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:21.840 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.534714+0000", 2024-09-11T06:51:21.840 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 1.231502761, 2024-09-11T06:51:21.840 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003413124, 2024-09-11T06:51:21.840 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.840 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.840 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:21.841 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.841 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.841 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.841 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.534714+0000", 2024-09-11T06:51:21.841 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.841 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.841 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.841 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.841 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.534714+0000", 2024-09-11T06:51:21.841 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.841 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.534715+0000", 2024-09-11T06:51:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6199999999999999e-06 2024-09-11T06:51:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.536302+0000", 2024-09-11T06:51:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001586269 2024-09-11T06:51:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.536347+0000", 2024-09-11T06:51:21.842 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5101000000000001e-05 2024-09-11T06:51:21.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.843 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.843 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.843 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.536379+0000", 2024-09-11T06:51:21.843 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2546000000000003e-05 2024-09-11T06:51:21.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.843 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.843 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.843 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.536541+0000", 2024-09-11T06:51:21.843 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000161812 2024-09-11T06:51:21.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.844 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.844 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.844 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.536561+0000", 2024-09-11T06:51:21.844 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0400000000000001e-05 2024-09-11T06:51:21.844 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.844 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.844 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.844 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.538127+0000", 2024-09-11T06:51:21.844 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0015653760000000001 2024-09-11T06:51:21.844 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.845 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.845 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.848 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.848 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(client.6081.0:30 2.4 e101/100 2:2d66d25d:::journal_data.2.1314f0a2bfa2.2:head v 101'171, mlcod=0'0)", 2024-09-11T06:51:21.848 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.536426+0000", 2024-09-11T06:51:21.849 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 1.2297903510000001, 2024-09-11T06:51:21.849 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033841259999999999, 2024-09-11T06:51:21.849 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.849 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.849 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:21.849 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.849 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.849 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.850 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.536426+0000", 2024-09-11T06:51:21.850 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.850 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.850 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.850 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.850 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.536426+0000", 2024-09-11T06:51:21.850 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.850 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.851 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.851 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.851 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.536427+0000", 2024-09-11T06:51:21.851 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2839999999999999e-06 2024-09-11T06:51:21.851 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.851 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.851 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.851 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.536441+0000", 2024-09-11T06:51:21.851 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3682e-05 2024-09-11T06:51:21.851 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.852 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.852 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.852 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.536457+0000", 2024-09-11T06:51:21.852 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5775999999999999e-05 2024-09-11T06:51:21.852 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.852 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.852 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.852 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.536466+0000", 2024-09-11T06:51:21.852 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0000000000000002e-06 2024-09-11T06:51:21.852 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.852 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.853 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.853 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.538179+0000", 2024-09-11T06:51:21.853 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0017128639999999999 2024-09-11T06:51:21.853 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.853 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.853 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.853 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.538222+0000", 2024-09-11T06:51:21.853 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.2894e-05 2024-09-11T06:51:21.853 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.853 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.853 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:21.853 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.539719+0000", 2024-09-11T06:51:21.854 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00149752 2024-09-11T06:51:21.854 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.854 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.854 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.854 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.539810+0000", 2024-09-11T06:51:21.854 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.1106000000000005e-05 2024-09-11T06:51:21.854 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.854 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.854 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.854 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.854 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.854 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.4 101/100 [PushOp(2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head, version: 54'23, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head@54'23, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:21.854 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.540925+0000", 2024-09-11T06:51:21.855 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 1.2252917480000001, 2024-09-11T06:51:21.855 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0040908539999999997, 2024-09-11T06:51:21.855 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.855 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.855 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:21.855 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.855 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.855 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.855 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.540925+0000", 2024-09-11T06:51:21.855 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.855 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.855 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.856 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.856 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.540925+0000", 2024-09-11T06:51:21.856 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.856 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.856 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.856 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.856 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.540926+0000", 2024-09-11T06:51:21.856 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.813e-06 2024-09-11T06:51:21.856 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.856 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.856 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.856 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542405+0000", 2024-09-11T06:51:21.856 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001478011 2024-09-11T06:51:21.856 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.857 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.857 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.857 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542453+0000", 2024-09-11T06:51:21.857 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.8677999999999997e-05 2024-09-11T06:51:21.857 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.857 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.857 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.857 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542483+0000", 2024-09-11T06:51:21.857 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9921999999999999e-05 2024-09-11T06:51:21.857 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.857 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.857 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.857 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542758+0000", 2024-09-11T06:51:21.858 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027447499999999999 2024-09-11T06:51:21.858 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.858 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.858 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.858 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542802+0000", 2024-09-11T06:51:21.858 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4288e-05 2024-09-11T06:51:21.858 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.858 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.858 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.858 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.545016+0000", 2024-09-11T06:51:21.858 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0022136669999999999 2024-09-11T06:51:21.858 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.858 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.858 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.859 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.859 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.859 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.4 101/100 [PushOp(2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head, version: 54'23, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head@54'23, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:21.859 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.547946+0000", 2024-09-11T06:51:21.859 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 1.218270261, 2024-09-11T06:51:21.859 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035631679999999998, 2024-09-11T06:51:21.859 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.859 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.859 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:21.859 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.859 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.859 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.859 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.547946+0000", 2024-09-11T06:51:21.859 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.860 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.860 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.860 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.860 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.547946+0000", 2024-09-11T06:51:21.860 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.860 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.860 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.860 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.860 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.547948+0000", 2024-09-11T06:51:21.860 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4529999999999999e-06 2024-09-11T06:51:21.860 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.860 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.860 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.861 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549382+0000", 2024-09-11T06:51:21.861 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014345759999999999 2024-09-11T06:51:21.861 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.861 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.861 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.861 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549425+0000", 2024-09-11T06:51:21.861 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.2528999999999999e-05 2024-09-11T06:51:21.861 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.861 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.861 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.861 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549453+0000", 2024-09-11T06:51:21.861 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7980999999999999e-05 2024-09-11T06:51:21.861 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.862 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.862 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.862 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549623+0000", 2024-09-11T06:51:21.862 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000170753 2024-09-11T06:51:21.862 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.862 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.862 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.862 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549644+0000", 2024-09-11T06:51:21.862 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0628999999999998e-05 2024-09-11T06:51:21.862 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.862 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.862 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.862 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.551509+0000", 2024-09-11T06:51:21.862 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001865247 2024-09-11T06:51:21.863 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.863 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.863 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.863 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.863 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.863 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(client.6081.0:37 2.4 e101/100 2:27499dce:::rbd_object_map.1314f0a2bfa2:head v 101'172, mlcod=54'22)", 2024-09-11T06:51:21.863 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.549499+0000", 2024-09-11T06:51:21.863 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 1.216717394, 2024-09-11T06:51:21.863 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0087992420000000005, 2024-09-11T06:51:21.863 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.863 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.863 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:21.863 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.863 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.864 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.864 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549499+0000", 2024-09-11T06:51:21.864 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.864 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.864 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.864 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.864 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549499+0000", 2024-09-11T06:51:21.864 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.864 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.864 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.864 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.864 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549500+0000", 2024-09-11T06:51:21.864 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.217e-06 2024-09-11T06:51:21.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.865 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.865 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.865 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549511+0000", 2024-09-11T06:51:21.865 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1143e-05 2024-09-11T06:51:21.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.865 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.865 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.865 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549532+0000", 2024-09-11T06:51:21.865 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0513000000000002e-05 2024-09-11T06:51:21.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.866 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.866 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549545+0000", 2024-09-11T06:51:21.866 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3209e-05 2024-09-11T06:51:21.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.866 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.866 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.551649+0000", 2024-09-11T06:51:21.866 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002103696 2024-09-11T06:51:21.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.866 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.866 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.551694+0000", 2024-09-11T06:51:21.867 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5674999999999999e-05 2024-09-11T06:51:21.867 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.867 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.867 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:21.867 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.558099+0000", 2024-09-11T06:51:21.867 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064040679999999997 2024-09-11T06:51:21.867 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.867 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.867 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.867 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.558298+0000", 2024-09-11T06:51:21.867 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019972100000000001 2024-09-11T06:51:21.868 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.868 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.868 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.868 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.868 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.868 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.4 101/100 [PushOp(2:272abc33:ns::rbd_data.11adc335555f.000000000000000c:head, version: 54'24, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:272abc33:ns::rbd_data.11adc335555f.000000000000000c:head@54'24, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:21.868 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.572958+0000", 2024-09-11T06:51:21.868 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 1.1932579679999999, 2024-09-11T06:51:21.868 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035425080000000002, 2024-09-11T06:51:21.868 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:21.868 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:21.869 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:21.869 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:21.869 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.869 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:21.869 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.572958+0000", 2024-09-11T06:51:21.869 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.869 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.869 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.869 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:21.869 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.572958+0000", 2024-09-11T06:51:21.869 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:21.869 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.870 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.870 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:21.870 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.572960+0000", 2024-09-11T06:51:21.870 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.333e-06 2024-09-11T06:51:21.870 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.870 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.870 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:21.870 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.574357+0000", 2024-09-11T06:51:21.870 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0013975540000000001 2024-09-11T06:51:21.870 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.870 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.871 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:21.871 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.574398+0000", 2024-09-11T06:51:21.871 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.0908000000000002e-05 2024-09-11T06:51:21.871 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.871 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.871 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:21.871 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.574425+0000", 2024-09-11T06:51:21.871 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6899000000000001e-05 2024-09-11T06:51:21.871 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.871 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.871 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:21.871 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.574604+0000", 2024-09-11T06:51:21.872 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000178359 2024-09-11T06:51:21.872 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.872 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.872 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:21.872 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.574626+0000", 2024-09-11T06:51:21.872 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2768999999999998e-05 2024-09-11T06:51:21.872 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:21.872 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:21.872 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:21.872 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.576501+0000", 2024-09-11T06:51:21.872 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001874686 2024-09-11T06:51:21.873 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.873 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.873 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.873 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:21.873 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:21.873 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:21.875 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:51:21.875 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:21.870+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:21.975 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:21.970+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:22.046 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:22.047 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:22.047 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:51:22.047 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:22.061 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:51:22.076 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:22.070+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:22.177 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:22.173+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:22.182 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:22.182 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:22.182 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:51:22.182 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:51:22.182 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:22.197 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:51:22.285 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:22.281+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:22.370 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:22.370 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:51:22.370 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:51:22.370 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:51:22.370 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.371 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:51:22.371 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:51:22.371 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 131.13564409200001, 2024-09-11T06:51:22.371 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010553425999999999, 2024-09-11T06:51:22.371 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:22.371 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:22.371 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:22.371 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:22.371 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:51:22.371 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:51:22.371 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:51:22.371 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.371 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:51:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:51:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:22.372 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230274+0000", 2024-09-11T06:51:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.533e-06 2024-09-11T06:51:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230285+0000", 2024-09-11T06:51:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0540000000000001e-05 2024-09-11T06:51:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230309+0000", 2024-09-11T06:51:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4223000000000002e-05 2024-09-11T06:51:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.373 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:22.374 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230327+0000", 2024-09-11T06:51:22.374 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:51:22.374 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.374 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.374 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.374 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230478+0000", 2024-09-11T06:51:22.374 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000151459 2024-09-11T06:51:22.374 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.374 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.374 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:22.374 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230490+0000", 2024-09-11T06:51:22.374 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.147e-05 2024-09-11T06:51:22.374 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231803+0000", 2024-09-11T06:51:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001313158 2024-09-11T06:51:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:51:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231817+0000", 2024-09-11T06:51:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4197e-05 2024-09-11T06:51:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.235945+0000", 2024-09-11T06:51:22.375 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0041282480000000002 2024-09-11T06:51:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236312+0000", 2024-09-11T06:51:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036652700000000001 2024-09-11T06:51:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236689+0000", 2024-09-11T06:51:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000377117 2024-09-11T06:51:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:22.376 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238460+0000", 2024-09-11T06:51:22.377 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001770918 2024-09-11T06:51:22.377 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.377 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.377 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:22.377 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240627+0000", 2024-09-11T06:51:22.377 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021670779999999998 2024-09-11T06:51:22.377 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.377 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.377 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:22.377 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240751+0000", 2024-09-11T06:51:22.377 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012384900000000001 2024-09-11T06:51:22.377 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.377 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.377 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240821+0000", 2024-09-11T06:51:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0516e-05 2024-09-11T06:51:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-11T06:51:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:51:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 130.13347324899999, 2024-09-11T06:51:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069904770000000001, 2024-09-11T06:51:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:22.378 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:51:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:51:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 5 2024-09-11T06:51:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:51:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:51:22.379 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232442+0000", 2024-09-11T06:51:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.044e-06 2024-09-11T06:51:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232460+0000", 2024-09-11T06:51:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7863000000000002e-05 2024-09-11T06:51:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232492+0000", 2024-09-11T06:51:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2812999999999999e-05 2024-09-11T06:51:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232510+0000", 2024-09-11T06:51:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7552999999999999e-05 2024-09-11T06:51:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232640+0000", 2024-09-11T06:51:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013029900000000001 2024-09-11T06:51:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232650+0000", 2024-09-11T06:51:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0023999999999999e-05 2024-09-11T06:51:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235641+0000", 2024-09-11T06:51:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00299089 2024-09-11T06:51:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235980+0000", 2024-09-11T06:51:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000338907 2024-09-11T06:51:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.236351+0000", 2024-09-11T06:51:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037087699999999998 2024-09-11T06:51:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238074+0000", 2024-09-11T06:51:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001722859 2024-09-11T06:51:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239160+0000", 2024-09-11T06:51:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001086074 2024-09-11T06:51:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239299+0000", 2024-09-11T06:51:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013933500000000001 2024-09-11T06:51:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239429+0000", 2024-09-11T06:51:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129939 2024-09-11T06:51:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-11T06:51:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:51:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 129.11938529700001, 2024-09-11T06:51:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069253250000000004, 2024-09-11T06:51:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:51:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:51:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 7 2024-09-11T06:51:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:51:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:51:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246530+0000", 2024-09-11T06:51:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9469999999999999e-06 2024-09-11T06:51:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246545+0000", 2024-09-11T06:51:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5973000000000001e-05 2024-09-11T06:51:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246578+0000", 2024-09-11T06:51:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2759999999999998e-05 2024-09-11T06:51:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246595+0000", 2024-09-11T06:51:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6939000000000002e-05 2024-09-11T06:51:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246770+0000", 2024-09-11T06:51:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000174723 2024-09-11T06:51:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246789+0000", 2024-09-11T06:51:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9159000000000001e-05 2024-09-11T06:51:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250337+0000", 2024-09-11T06:51:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035480759999999998 2024-09-11T06:51:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250549+0000", 2024-09-11T06:51:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021192199999999999 2024-09-11T06:51:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250750+0000", 2024-09-11T06:51:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020104199999999999 2024-09-11T06:51:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.252455+0000", 2024-09-11T06:51:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001705249 2024-09-11T06:51:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253305+0000", 2024-09-11T06:51:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00084941200000000002 2024-09-11T06:51:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253388+0000", 2024-09-11T06:51:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3642999999999999e-05 2024-09-11T06:51:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253452+0000", 2024-09-11T06:51:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3479999999999995e-05 2024-09-11T06:51:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:57 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.568529+0000", 2024-09-11T06:51:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 33.797382376999998, 2024-09-11T06:51:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004228229, 2024-09-11T06:51:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 57 2024-09-11T06:51:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568529+0000", 2024-09-11T06:51:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568529+0000", 2024-09-11T06:51:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568532+0000", 2024-09-11T06:51:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0000000000000001e-06 2024-09-11T06:51:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568547+0000", 2024-09-11T06:51:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4219999999999999e-05 2024-09-11T06:51:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568580+0000", 2024-09-11T06:51:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.3328999999999999e-05 2024-09-11T06:51:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568596+0000", 2024-09-11T06:51:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5990000000000001e-05 2024-09-11T06:51:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568728+0000", 2024-09-11T06:51:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013175400000000001 2024-09-11T06:51:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568738+0000", 2024-09-11T06:51:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0196e-05 2024-09-11T06:51:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.572464+0000", 2024-09-11T06:51:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0037264160000000002 2024-09-11T06:51:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.572584+0000", 2024-09-11T06:51:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000119438 2024-09-11T06:51:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.572758+0000", 2024-09-11T06:51:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017388599999999999 2024-09-11T06:51:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5374.0:21 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [watch unwatch cookie 94119567275296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:12.469500+0000", 2024-09-11T06:51:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 9.8964115489999998, 2024-09-11T06:51:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0026485419999999998, 2024-09-11T06:51:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5374", 2024-09-11T06:51:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1937077541", 2024-09-11T06:51:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 21 2024-09-11T06:51:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469500+0000", 2024-09-11T06:51:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469500+0000", 2024-09-11T06:51:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469501+0000", 2024-09-11T06:51:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.33e-07 2024-09-11T06:51:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469506+0000", 2024-09-11T06:51:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.8690000000000003e-06 2024-09-11T06:51:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469519+0000", 2024-09-11T06:51:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3077e-05 2024-09-11T06:51:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469529+0000", 2024-09-11T06:51:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.8060000000000006e-06 2024-09-11T06:51:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469853+0000", 2024-09-11T06:51:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00032408399999999998 2024-09-11T06:51:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469915+0000", 2024-09-11T06:51:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2379999999999996e-05 2024-09-11T06:51:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.470021+0000", 2024-09-11T06:51:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00010504400000000001 2024-09-11T06:51:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.471499+0000", 2024-09-11T06:51:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014781989999999999 2024-09-11T06:51:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.471932+0000", 2024-09-11T06:51:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00043362499999999998 2024-09-11T06:51:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.472003+0000", 2024-09-11T06:51:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1060000000000001e-05 2024-09-11T06:51:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.472149+0000", 2024-09-11T06:51:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000145465 2024-09-11T06:51:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.404 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.404 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.404 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5826.0:125 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.client_list in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:22.404 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:12.469719+0000", 2024-09-11T06:51:22.404 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 9.8961928950000004, 2024-09-11T06:51:22.404 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0026625149999999998, 2024-09-11T06:51:22.404 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:22.404 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:22.404 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:22.404 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:22.404 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5826", 2024-09-11T06:51:22.404 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2236190565", 2024-09-11T06:51:22.404 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 125 2024-09-11T06:51:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469719+0000", 2024-09-11T06:51:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469719+0000", 2024-09-11T06:51:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.405 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:22.406 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469720+0000", 2024-09-11T06:51:22.406 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.097e-06 2024-09-11T06:51:22.406 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.406 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.406 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:22.406 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469726+0000", 2024-09-11T06:51:22.406 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2870000000000001e-06 2024-09-11T06:51:22.406 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.406 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.406 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:22.406 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469739+0000", 2024-09-11T06:51:22.406 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2789000000000001e-05 2024-09-11T06:51:22.406 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.407 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.407 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:22.407 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469749+0000", 2024-09-11T06:51:22.407 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.6879999999999994e-06 2024-09-11T06:51:22.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.407 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.407 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.407 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.470359+0000", 2024-09-11T06:51:22.407 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00060988299999999997 2024-09-11T06:51:22.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.407 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.407 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:22.407 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.470428+0000", 2024-09-11T06:51:22.407 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9696999999999994e-05 2024-09-11T06:51:22.408 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.408 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.408 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.408 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.472195+0000", 2024-09-11T06:51:22.408 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001767034 2024-09-11T06:51:22.408 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.408 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.408 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:22.408 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.472244+0000", 2024-09-11T06:51:22.408 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9038e-05 2024-09-11T06:51:22.408 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.408 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.408 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:22.409 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.472381+0000", 2024-09-11T06:51:22.409 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000137002 2024-09-11T06:51:22.409 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.409 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:22.409 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.409 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.409 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.409 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5374.0:22 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.client_list in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:22.409 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:12.469984+0000", 2024-09-11T06:51:22.409 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 9.8959281200000007, 2024-09-11T06:51:22.409 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027625599999999998, 2024-09-11T06:51:22.409 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:22.409 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:22.410 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:22.410 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:22.410 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5374", 2024-09-11T06:51:22.410 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1937077541", 2024-09-11T06:51:22.410 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 22 2024-09-11T06:51:22.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.410 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:22.410 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.410 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:22.410 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469984+0000", 2024-09-11T06:51:22.410 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.410 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.410 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469984+0000", 2024-09-11T06:51:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469985+0000", 2024-09-11T06:51:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.569999999999999e-07 2024-09-11T06:51:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469990+0000", 2024-09-11T06:51:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5160000000000002e-06 2024-09-11T06:51:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.411 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.470003+0000", 2024-09-11T06:51:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2933e-05 2024-09-11T06:51:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.470012+0000", 2024-09-11T06:51:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2e-06 2024-09-11T06:51:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.470453+0000", 2024-09-11T06:51:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000440963 2024-09-11T06:51:22.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.470509+0000", 2024-09-11T06:51:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5937999999999999e-05 2024-09-11T06:51:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.472419+0000", 2024-09-11T06:51:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0019100289999999999 2024-09-11T06:51:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.472523+0000", 2024-09-11T06:51:22.413 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103665 2024-09-11T06:51:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.472746+0000", 2024-09-11T06:51:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022335899999999999 2024-09-11T06:51:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5386.0:17 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [watch unwatch cookie 94226507267584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:12.563293+0000", 2024-09-11T06:51:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 9.802618635, 2024-09-11T06:51:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0028623239999999999, 2024-09-11T06:51:22.414 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5386", 2024-09-11T06:51:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3377056437", 2024-09-11T06:51:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 17 2024-09-11T06:51:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563293+0000", 2024-09-11T06:51:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.415 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563293+0000", 2024-09-11T06:51:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563294+0000", 2024-09-11T06:51:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5300000000000004e-07 2024-09-11T06:51:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563296+0000", 2024-09-11T06:51:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6079999999999998e-06 2024-09-11T06:51:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.416 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.417 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:22.417 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563302+0000", 2024-09-11T06:51:22.417 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2549999999999997e-06 2024-09-11T06:51:22.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.417 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.417 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:22.417 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563305+0000", 2024-09-11T06:51:22.417 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2949999999999998e-06 2024-09-11T06:51:22.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.418 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.418 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.418 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563792+0000", 2024-09-11T06:51:22.418 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00048731399999999998 2024-09-11T06:51:22.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.418 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.418 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:22.418 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563899+0000", 2024-09-11T06:51:22.418 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000106977 2024-09-11T06:51:22.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.418 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.418 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:22.418 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.564145+0000", 2024-09-11T06:51:22.418 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00024587600000000002 2024-09-11T06:51:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.564760+0000", 2024-09-11T06:51:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00061473099999999998 2024-09-11T06:51:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566046+0000", 2024-09-11T06:51:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012856810000000001 2024-09-11T06:51:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566085+0000", 2024-09-11T06:51:22.419 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.9499000000000002e-05 2024-09-11T06:51:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566156+0000", 2024-09-11T06:51:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0434999999999999e-05 2024-09-11T06:51:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5826.0:131 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.client_list in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:12.563833+0000", 2024-09-11T06:51:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 9.8020790790000003, 2024-09-11T06:51:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0028764329999999999, 2024-09-11T06:51:22.420 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5826", 2024-09-11T06:51:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2236190565", 2024-09-11T06:51:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 131 2024-09-11T06:51:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563833+0000", 2024-09-11T06:51:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.422 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.422 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:22.422 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563833+0000", 2024-09-11T06:51:22.422 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.422 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.422 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:22.422 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563835+0000", 2024-09-11T06:51:22.422 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8109999999999999e-06 2024-09-11T06:51:22.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.422 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.422 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:22.422 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563840+0000", 2024-09-11T06:51:22.422 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7599999999999999e-06 2024-09-11T06:51:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563854+0000", 2024-09-11T06:51:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3332999999999999e-05 2024-09-11T06:51:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563874+0000", 2024-09-11T06:51:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0630999999999999e-05 2024-09-11T06:51:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.564553+0000", 2024-09-11T06:51:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00067821199999999998 2024-09-11T06:51:22.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.424 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.424 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:22.424 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.564610+0000", 2024-09-11T06:51:22.424 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7494999999999998e-05 2024-09-11T06:51:22.424 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.424 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.424 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.424 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566240+0000", 2024-09-11T06:51:22.424 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0016295940000000001 2024-09-11T06:51:22.424 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.424 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.424 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:22.424 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566365+0000", 2024-09-11T06:51:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000124876 2024-09-11T06:51:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566709+0000", 2024-09-11T06:51:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034472099999999998 2024-09-11T06:51:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5447.0:10 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T06:51:22.425 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:13.624024+0000", 2024-09-11T06:51:22.426 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 8.7418881860000006, 2024-09-11T06:51:22.426 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0038665190000000001, 2024-09-11T06:51:22.426 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:22.426 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:22.426 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:22.426 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:22.426 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5447", 2024-09-11T06:51:22.426 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3862443986", 2024-09-11T06:51:22.426 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 10 2024-09-11T06:51:22.426 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.426 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:22.426 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.426 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:22.426 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624024+0000", 2024-09-11T06:51:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624024+0000", 2024-09-11T06:51:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624026+0000", 2024-09-11T06:51:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8090000000000001e-06 2024-09-11T06:51:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.427 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.428 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:22.428 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624037+0000", 2024-09-11T06:51:22.428 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.061e-05 2024-09-11T06:51:22.428 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.428 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.428 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:22.428 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624059+0000", 2024-09-11T06:51:22.428 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2269e-05 2024-09-11T06:51:22.428 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.428 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.428 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:22.428 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624074+0000", 2024-09-11T06:51:22.428 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4982e-05 2024-09-11T06:51:22.428 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624238+0000", 2024-09-11T06:51:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163191 2024-09-11T06:51:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624261+0000", 2024-09-11T06:51:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3779000000000001e-05 2024-09-11T06:51:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.627590+0000", 2024-09-11T06:51:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033287450000000001 2024-09-11T06:51:22.429 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.430 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.430 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:22.430 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.627723+0000", 2024-09-11T06:51:22.430 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013311599999999999 2024-09-11T06:51:22.430 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.430 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.430 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:22.430 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.627890+0000", 2024-09-11T06:51:22.430 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016701800000000001 2024-09-11T06:51:22.430 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.430 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:22.430 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.430 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.430 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5453.0:16 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [watch unwatch cookie 93828371656000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-11T06:51:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:13.655839+0000", 2024-09-11T06:51:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 8.710072534, 2024-09-11T06:51:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002630959, 2024-09-11T06:51:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5453", 2024-09-11T06:51:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1091178110", 2024-09-11T06:51:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 16 2024-09-11T06:51:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:22.431 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.432 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:22.432 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.655839+0000", 2024-09-11T06:51:22.432 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.432 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.432 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:22.432 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.655839+0000", 2024-09-11T06:51:22.432 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.432 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.432 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:22.432 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.655841+0000", 2024-09-11T06:51:22.432 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7799999999999999e-06 2024-09-11T06:51:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.655850+0000", 2024-09-11T06:51:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8559999999999996e-06 2024-09-11T06:51:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.655869+0000", 2024-09-11T06:51:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9446e-05 2024-09-11T06:51:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.655890+0000", 2024-09-11T06:51:22.434 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0292000000000002e-05 2024-09-11T06:51:22.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.434 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.434 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.434 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656678+0000", 2024-09-11T06:51:22.434 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00078840500000000005 2024-09-11T06:51:22.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.434 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.434 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:22.434 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656785+0000", 2024-09-11T06:51:22.434 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00010641699999999999 2024-09-11T06:51:22.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.434 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.434 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656993+0000", 2024-09-11T06:51:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000208156 2024-09-11T06:51:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.658118+0000", 2024-09-11T06:51:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011256829999999999 2024-09-11T06:51:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.658242+0000", 2024-09-11T06:51:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012334999999999999 2024-09-11T06:51:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.435 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.658322+0000", 2024-09-11T06:51:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.0092000000000005e-05 2024-09-11T06:51:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.658470+0000", 2024-09-11T06:51:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014848199999999999 2024-09-11T06:51:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.436 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5453.0:17 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.client_list in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T06:51:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:13.656142+0000", 2024-09-11T06:51:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 8.709769627, 2024-09-11T06:51:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027779549999999999, 2024-09-11T06:51:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5453", 2024-09-11T06:51:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1091178110", 2024-09-11T06:51:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 17 2024-09-11T06:51:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:22.438 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656142+0000", 2024-09-11T06:51:22.438 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.438 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.438 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:22.438 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656142+0000", 2024-09-11T06:51:22.438 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.438 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.438 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:22.438 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656144+0000", 2024-09-11T06:51:22.438 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.68e-06 2024-09-11T06:51:22.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.439 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.439 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:22.439 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656154+0000", 2024-09-11T06:51:22.439 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0159999999999999e-05 2024-09-11T06:51:22.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.439 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.439 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:22.439 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656173+0000", 2024-09-11T06:51:22.439 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9324999999999999e-05 2024-09-11T06:51:22.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.439 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.439 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:22.439 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656193+0000", 2024-09-11T06:51:22.440 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9692999999999999e-05 2024-09-11T06:51:22.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.440 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.440 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.440 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.657563+0000", 2024-09-11T06:51:22.440 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0013697099999999999 2024-09-11T06:51:22.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.440 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.440 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:22.440 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.657779+0000", 2024-09-11T06:51:22.440 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021642100000000001 2024-09-11T06:51:22.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.440 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.440 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.441 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.658528+0000", 2024-09-11T06:51:22.441 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00074905000000000002 2024-09-11T06:51:22.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.441 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.441 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:22.441 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.658639+0000", 2024-09-11T06:51:22.441 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000110788 2024-09-11T06:51:22.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.441 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.441 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:22.441 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.658920+0000", 2024-09-11T06:51:22.441 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00028112800000000001 2024-09-11T06:51:22.441 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.442 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:22.442 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.442 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.442 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5826.0:137 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.client_list in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T06:51:22.442 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:13.656886+0000", 2024-09-11T06:51:22.442 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 8.7090258160000005, 2024-09-11T06:51:22.442 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002654478, 2024-09-11T06:51:22.442 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:22.442 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:22.442 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:22.442 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:22.442 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5826", 2024-09-11T06:51:22.443 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2236190565", 2024-09-11T06:51:22.443 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 137 2024-09-11T06:51:22.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.443 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:22.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.443 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:22.443 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656886+0000", 2024-09-11T06:51:22.443 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.443 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:22.443 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656886+0000", 2024-09-11T06:51:22.443 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656888+0000", 2024-09-11T06:51:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.66e-06 2024-09-11T06:51:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656898+0000", 2024-09-11T06:51:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0042e-05 2024-09-11T06:51:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656917+0000", 2024-09-11T06:51:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9522e-05 2024-09-11T06:51:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656928+0000", 2024-09-11T06:51:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1097000000000001e-05 2024-09-11T06:51:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.657882+0000", 2024-09-11T06:51:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00095349499999999995 2024-09-11T06:51:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:22.445 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.658007+0000", 2024-09-11T06:51:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000125337 2024-09-11T06:51:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.659031+0000", 2024-09-11T06:51:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001024248 2024-09-11T06:51:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.659149+0000", 2024-09-11T06:51:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000117971 2024-09-11T06:51:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:22.446 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.659541+0000", 2024-09-11T06:51:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00039110600000000001 2024-09-11T06:51:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.6018.0:17 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [watch unwatch cookie 93970929245696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-11T06:51:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:16.021477+0000", 2024-09-11T06:51:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 6.3444351990000003, 2024-09-11T06:51:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0026652009999999999, 2024-09-11T06:51:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:22.447 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.6018", 2024-09-11T06:51:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1879900433", 2024-09-11T06:51:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 17 2024-09-11T06:51:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:16.021477+0000", 2024-09-11T06:51:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:16.021477+0000", 2024-09-11T06:51:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:16.021479+0000", 2024-09-11T06:51:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9800000000000001e-06 2024-09-11T06:51:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:16.021488+0000", 2024-09-11T06:51:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.6330000000000008e-06 2024-09-11T06:51:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:16.021519+0000", 2024-09-11T06:51:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.1208999999999999e-05 2024-09-11T06:51:22.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:16.021538+0000", 2024-09-11T06:51:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8919e-05 2024-09-11T06:51:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:16.022335+0000", 2024-09-11T06:51:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00079702600000000005 2024-09-11T06:51:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:16.022457+0000", 2024-09-11T06:51:22.451 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121494 2024-09-11T06:51:22.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.451 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.451 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:22.451 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:16.022666+0000", 2024-09-11T06:51:22.451 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020869399999999999 2024-09-11T06:51:22.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.451 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.451 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:22.451 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:16.023772+0000", 2024-09-11T06:51:22.451 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011067410000000001 2024-09-11T06:51:22.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.451 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.451 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:22.452 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:16.023872+0000", 2024-09-11T06:51:22.452 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9702000000000002e-05 2024-09-11T06:51:22.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.452 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:22.452 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:16.023957+0000", 2024-09-11T06:51:22.452 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.5093000000000007e-05 2024-09-11T06:51:22.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.452 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:22.452 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:16.024142+0000", 2024-09-11T06:51:22.452 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018471000000000001 2024-09-11T06:51:22.452 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.452 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.6081.0:32 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.set_minimum_set in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-11T06:51:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.541248+0000", 2024-09-11T06:51:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 1.8246638590000002, 2024-09-11T06:51:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0028300869999999998, 2024-09-11T06:51:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.6081", 2024-09-11T06:51:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3800899964", 2024-09-11T06:51:22.453 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 32 2024-09-11T06:51:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541248+0000", 2024-09-11T06:51:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541248+0000", 2024-09-11T06:51:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:22.454 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541251+0000", 2024-09-11T06:51:22.455 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5299999999999999e-06 2024-09-11T06:51:22.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.455 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:22.455 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541266+0000", 2024-09-11T06:51:22.455 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5139e-05 2024-09-11T06:51:22.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.455 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:22.455 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541298+0000", 2024-09-11T06:51:22.455 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2152e-05 2024-09-11T06:51:22.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.455 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541313+0000", 2024-09-11T06:51:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5532999999999998e-05 2024-09-11T06:51:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541554+0000", 2024-09-11T06:51:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000240306 2024-09-11T06:51:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541957+0000", 2024-09-11T06:51:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00040357799999999998 2024-09-11T06:51:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.544078+0000", 2024-09-11T06:51:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021208490000000002 2024-09-11T06:51:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.6081.0:33 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-11T06:51:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.541352+0000", 2024-09-11T06:51:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 1.8245597340000002, 2024-09-11T06:51:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034084850000000002, 2024-09-11T06:51:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:22.457 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:22.458 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:22.458 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:22.458 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.6081", 2024-09-11T06:51:22.458 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3800899964", 2024-09-11T06:51:22.458 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 33 2024-09-11T06:51:22.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.458 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:22.458 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.458 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:22.458 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541352+0000", 2024-09-11T06:51:22.458 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.458 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.458 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541352+0000", 2024-09-11T06:51:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541353+0000", 2024-09-11T06:51:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7000000000000003e-07 2024-09-11T06:51:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541358+0000", 2024-09-11T06:51:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7400000000000004e-06 2024-09-11T06:51:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.459 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:22.460 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541374+0000", 2024-09-11T06:51:22.460 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5670000000000001e-05 2024-09-11T06:51:22.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.460 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.460 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:22.460 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541386+0000", 2024-09-11T06:51:22.460 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2359e-05 2024-09-11T06:51:22.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.460 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.460 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.460 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542684+0000", 2024-09-11T06:51:22.460 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012984590000000001 2024-09-11T06:51:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542820+0000", 2024-09-11T06:51:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013536700000000001 2024-09-11T06:51:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542972+0000", 2024-09-11T06:51:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000151843 2024-09-11T06:51:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.544110+0000", 2024-09-11T06:51:22.461 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011381060000000001 2024-09-11T06:51:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.544510+0000", 2024-09-11T06:51:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00040019799999999999 2024-09-11T06:51:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.544643+0000", 2024-09-11T06:51:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013313499999999999 2024-09-11T06:51:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.544761+0000", 2024-09-11T06:51:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000117638 2024-09-11T06:51:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5542.0:10 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-11T06:51:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:22.002114+0000", 2024-09-11T06:51:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.363798285, 2024-09-11T06:51:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00564235, 2024-09-11T06:51:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:22.463 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5542", 2024-09-11T06:51:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1546942093", 2024-09-11T06:51:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 10 2024-09-11T06:51:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002114+0000", 2024-09-11T06:51:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002114+0000", 2024-09-11T06:51:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.464 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002115+0000", 2024-09-11T06:51:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.872e-06 2024-09-11T06:51:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002123+0000", 2024-09-11T06:51:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3309999999999998e-06 2024-09-11T06:51:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002136+0000", 2024-09-11T06:51:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2816000000000001e-05 2024-09-11T06:51:22.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002145+0000", 2024-09-11T06:51:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.8570000000000007e-06 2024-09-11T06:51:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002214+0000", 2024-09-11T06:51:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.8832999999999996e-05 2024-09-11T06:51:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002220+0000", 2024-09-11T06:51:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4249999999999997e-06 2024-09-11T06:51:22.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.007445+0000", 2024-09-11T06:51:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052251659999999998 2024-09-11T06:51:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.007586+0000", 2024-09-11T06:51:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014103100000000001 2024-09-11T06:51:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.007756+0000", 2024-09-11T06:51:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017001900000000001 2024-09-11T06:51:22.467 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.468 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:22.468 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.468 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.468 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.468 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5542.0:21 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.create in=13b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-11T06:51:22.468 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:22.028810+0000", 2024-09-11T06:51:22.468 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.33710224100000002, 2024-09-11T06:51:22.468 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034172709999999999, 2024-09-11T06:51:22.468 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:22.468 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:22.468 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:22.468 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:22.468 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5542", 2024-09-11T06:51:22.468 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1546942093", 2024-09-11T06:51:22.469 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 21 2024-09-11T06:51:22.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.469 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:22.469 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.469 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:22.469 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.028810+0000", 2024-09-11T06:51:22.469 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.469 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.469 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:22.469 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.028810+0000", 2024-09-11T06:51:22.469 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.469 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.028812+0000", 2024-09-11T06:51:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8100000000000002e-06 2024-09-11T06:51:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.028826+0000", 2024-09-11T06:51:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3882e-05 2024-09-11T06:51:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.028859+0000", 2024-09-11T06:51:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2225999999999999e-05 2024-09-11T06:51:22.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.028880+0000", 2024-09-11T06:51:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1755000000000001e-05 2024-09-11T06:51:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.029048+0000", 2024-09-11T06:51:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016811699999999999 2024-09-11T06:51:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.029198+0000", 2024-09-11T06:51:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000149753 2024-09-11T06:51:22.471 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.029612+0000", 2024-09-11T06:51:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00041426899999999999 2024-09-11T06:51:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.030882+0000", 2024-09-11T06:51:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012693030000000001 2024-09-11T06:51:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.032087+0000", 2024-09-11T06:51:22.472 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012054279999999999 2024-09-11T06:51:22.473 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.473 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.473 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:22.473 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.032167+0000", 2024-09-11T06:51:22.473 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.0177000000000002e-05 2024-09-11T06:51:22.473 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.473 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.473 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:22.473 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.032227+0000", 2024-09-11T06:51:22.473 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.9551e-05 2024-09-11T06:51:22.473 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.473 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:22.473 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.473 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5542.0:23 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.tag_create in=17b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-11T06:51:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:22.033580+0000", 2024-09-11T06:51:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.33233179099999999, 2024-09-11T06:51:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002804384, 2024-09-11T06:51:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5542", 2024-09-11T06:51:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1546942093", 2024-09-11T06:51:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 23 2024-09-11T06:51:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:22.474 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033580+0000", 2024-09-11T06:51:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033580+0000", 2024-09-11T06:51:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033581+0000", 2024-09-11T06:51:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.217e-06 2024-09-11T06:51:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.476 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:22.476 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033591+0000", 2024-09-11T06:51:22.476 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2660000000000007e-06 2024-09-11T06:51:22.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.476 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:22.476 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033611+0000", 2024-09-11T06:51:22.476 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0418999999999999e-05 2024-09-11T06:51:22.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.476 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:22.476 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033628+0000", 2024-09-11T06:51:22.476 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7316000000000002e-05 2024-09-11T06:51:22.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033803+0000", 2024-09-11T06:51:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017436200000000001 2024-09-11T06:51:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033944+0000", 2024-09-11T06:51:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000141688 2024-09-11T06:51:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.034475+0000", 2024-09-11T06:51:22.477 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00053032399999999999 2024-09-11T06:51:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.035684+0000", 2024-09-11T06:51:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001209452 2024-09-11T06:51:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.036254+0000", 2024-09-11T06:51:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00056982599999999995 2024-09-11T06:51:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.036328+0000", 2024-09-11T06:51:22.478 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4499999999999995e-05 2024-09-11T06:51:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.036384+0000", 2024-09-11T06:51:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.6014000000000003e-05 2024-09-11T06:51:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5542.0:26 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.client_register in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-11T06:51:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:22.038666+0000", 2024-09-11T06:51:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.32724586999999999, 2024-09-11T06:51:22.479 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027244439999999999, 2024-09-11T06:51:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5542", 2024-09-11T06:51:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1546942093", 2024-09-11T06:51:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 26 2024-09-11T06:51:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.038666+0000", 2024-09-11T06:51:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.481 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:22.481 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.038666+0000", 2024-09-11T06:51:22.481 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:22.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.481 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:22.481 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.038668+0000", 2024-09-11T06:51:22.481 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.516e-06 2024-09-11T06:51:22.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.481 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:22.481 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.038675+0000", 2024-09-11T06:51:22.481 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2030000000000003e-06 2024-09-11T06:51:22.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.038694+0000", 2024-09-11T06:51:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9276e-05 2024-09-11T06:51:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.038711+0000", 2024-09-11T06:51:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7119e-05 2024-09-11T06:51:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.038831+0000", 2024-09-11T06:51:22.482 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000119929 2024-09-11T06:51:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.038954+0000", 2024-09-11T06:51:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012327099999999999 2024-09-11T06:51:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.039374+0000", 2024-09-11T06:51:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000419372 2024-09-11T06:51:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.040347+0000", 2024-09-11T06:51:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00097351800000000004 2024-09-11T06:51:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.041254+0000", 2024-09-11T06:51:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00090697200000000003 2024-09-11T06:51:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.041334+0000", 2024-09-11T06:51:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.0302999999999993e-05 2024-09-11T06:51:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.041390+0000", 2024-09-11T06:51:22.484 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5964999999999997e-05 2024-09-11T06:51:22.485 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.485 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:22.485 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.485 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:22.485 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:22.485 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:22.485 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:51:22.487 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:22.482+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:22.487 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:51:22.588 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:22.582+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:22.618 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:22.618 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:22.618 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:51:22.618 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:22.633 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:51:22.634 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:51:22.634 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:51:22.651 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:51:22.651 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:51:22.668 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:51:22.668 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:51:22.689 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:51:22.689 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:51:22.689 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:22.682+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:22.709 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:51:22.710 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:51:22.730 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:51:22.730 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:51:22.748 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:22.748 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:22.763 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:51:22.810 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:22.806+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:22.891 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:22.886+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:22.924 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:22.924 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:22.940 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:51:23.003 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 3, 5] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T06:51:23.004 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T06:51:23.004 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-11T06:51:23.004 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-11T06:51:23.005 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T06:51:23.026 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:23.022+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:23.093 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:23.086+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:23.107 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:23.107 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:23.123 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:51:23.209 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:23.206+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:23.280 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:23.281 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:23.293 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:51:23.331 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:51:23.331 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:23.331 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 103, 2024-09-11T06:51:23.331 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "bd5dcb33-e9f7-4a5f-a26c-a451452e8aa2", 2024-09-11T06:51:23.331 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2024-09-11T06:48:52.191706+0000", 2024-09-11T06:51:23.331 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2024-09-11T06:51:21.977212+0000", 2024-09-11T06:51:23.331 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2024-09-11T06:49:04.007203+0000", 2024-09-11T06:51:23.331 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2024-09-11T06:51:17.949107+0000", 2024-09-11T06:51:23.331 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-11T06:51:23.331 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5801984, 2024-09-11T06:51:23.332 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2024-09-11T06:51:23.332 INFO:teuthology.orchestra.run.smithi071.stdout: "noscrub", 2024-09-11T06:51:23.332 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2024-09-11T06:51:23.332 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2024-09-11T06:51:23.332 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2024-09-11T06:51:23.332 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2024-09-11T06:51:23.332 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-11T06:51:23.332 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2024-09-11T06:51:23.332 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.94999998807907104, 2024-09-11T06:51:23.332 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-11T06:51:23.332 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-11T06:51:23.332 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2024-09-11T06:51:23.333 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 3, 2024-09-11T06:51:23.333 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 6, 2024-09-11T06:51:23.333 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2024-09-11T06:51:23.333 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "luminous", 2024-09-11T06:51:23.333 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "squid", 2024-09-11T06:51:23.333 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2024-09-11T06:51:23.333 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2024-09-11T06:51:23.333 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.333 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2024-09-11T06:51:23.333 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2024-09-11T06:51:23.333 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2024-09-11T06:49:05.084515+0000", 2024-09-11T06:51:23.333 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-11T06:51:23.333 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2024-09-11T06:51:23.334 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2024-09-11T06:51:23.334 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2024-09-11T06:51:23.334 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2024-09-11T06:51:23.334 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2024-09-11T06:51:23.334 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2024-09-11T06:51:23.334 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2024-09-11T06:51:23.334 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T06:51:23.334 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T06:51:23.334 INFO:teuthology.orchestra.run.smithi071.stdout: "is_stretch_pool": false, 2024-09-11T06:51:23.334 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2024-09-11T06:51:23.334 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2024-09-11T06:51:23.334 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2024-09-11T06:51:23.334 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2024-09-11T06:51:23.335 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2024-09-11T06:51:23.335 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2024-09-11T06:51:23.335 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2024-09-11T06:51:23.335 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2024-09-11T06:51:23.335 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2024-09-11T06:51:23.335 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2024-09-11T06:51:23.335 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2024-09-11T06:51:23.335 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2024-09-11T06:51:23.335 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2024-09-11T06:51:23.335 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2024-09-11T06:51:23.335 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.335 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "14", 2024-09-11T06:51:23.335 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2024-09-11T06:51:23.335 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T06:51:23.336 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T06:51:23.336 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2024-09-11T06:51:23.336 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2024-09-11T06:51:23.336 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2024-09-11T06:51:23.336 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2024-09-11T06:51:23.336 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2024-09-11T06:51:23.336 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2024-09-11T06:51:23.336 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2024-09-11T06:51:23.336 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2024-09-11T06:51:23.336 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2024-09-11T06:51:23.336 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2024-09-11T06:51:23.336 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2024-09-11T06:51:23.336 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2024-09-11T06:51:23.336 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2024-09-11T06:51:23.337 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2024-09-11T06:51:23.337 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2024-09-11T06:51:23.337 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T06:51:23.337 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T06:51:23.337 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T06:51:23.337 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2024-09-11T06:51:23.337 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2024-09-11T06:51:23.337 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2024-09-11T06:51:23.337 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2024-09-11T06:51:23.337 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2024-09-11T06:51:23.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.337 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2024-09-11T06:51:23.337 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2024-09-11T06:51:23.338 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2024-09-11T06:51:23.338 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2024-09-11T06:51:23.338 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2024-09-11T06:51:23.338 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T06:51:23.338 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2024-09-11T06:51:23.338 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2024-09-11T06:51:23.338 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2024-09-11T06:51:23.338 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2024-09-11T06:51:23.338 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2024-09-11T06:51:23.338 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2024-09-11T06:51:23.338 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2024-09-11T06:51:23.338 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2024-09-11T06:51:23.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.338 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2024-09-11T06:51:23.339 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2024-09-11T06:51:23.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.339 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2024-09-11T06:51:23.339 INFO:teuthology.orchestra.run.smithi071.stdout: "score_type": "Fair distribution", 2024-09-11T06:51:23.339 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 6.059999942779541, 2024-09-11T06:51:23.339 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 6.059999942779541, 2024-09-11T06:51:23.339 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.33000001311302185, 2024-09-11T06:51:23.339 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2024-09-11T06:51:23.339 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2024-09-11T06:51:23.339 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 0.85340124368667603, 2024-09-11T06:51:23.339 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 0.85000002384185791, 2024-09-11T06:51:23.339 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 0.85000002384185791 2024-09-11T06:51:23.339 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.340 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.340 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 2, 2024-09-11T06:51:23.340 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "rbd", 2024-09-11T06:51:23.340 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2024-09-11T06:49:09.530167+0000", 2024-09-11T06:51:23.340 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 8193, 2024-09-11T06:51:23.340 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T06:51:23.340 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2024-09-11T06:51:23.340 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2024-09-11T06:51:23.340 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2024-09-11T06:51:23.340 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2024-09-11T06:51:23.340 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2024-09-11T06:51:23.340 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2024-09-11T06:51:23.340 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T06:51:23.341 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T06:51:23.341 INFO:teuthology.orchestra.run.smithi071.stdout: "is_stretch_pool": false, 2024-09-11T06:51:23.341 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2024-09-11T06:51:23.341 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2024-09-11T06:51:23.341 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 8, 2024-09-11T06:51:23.341 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 8, 2024-09-11T06:51:23.341 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 8, 2024-09-11T06:51:23.341 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 8, 2024-09-11T06:51:23.341 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 8, 2024-09-11T06:51:23.341 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2024-09-11T06:51:23.341 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2024-09-11T06:51:23.341 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2024-09-11T06:51:23.341 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2024-09-11T06:51:23.342 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2024-09-11T06:51:23.342 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2024-09-11T06:51:23.342 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2024-09-11T06:51:23.342 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.342 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "91", 2024-09-11T06:51:23.342 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2024-09-11T06:51:23.342 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T06:51:23.342 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T06:51:23.342 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2024-09-11T06:51:23.342 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2024-09-11T06:51:23.342 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 7, 2024-09-11T06:51:23.342 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 91, 2024-09-11T06:51:23.342 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2024-09-11T06:51:23.343 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2024-09-11T06:51:23.343 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2024-09-11T06:51:23.343 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2024-09-11T06:51:23.343 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2024-09-11T06:51:23.343 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2024-09-11T06:51:23.343 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2024-09-11T06:51:23.343 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2024-09-11T06:51:23.343 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2024-09-11T06:51:23.343 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2024-09-11T06:51:23.343 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2024-09-11T06:51:23.343 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T06:51:23.343 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T06:51:23.343 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T06:51:23.344 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2024-09-11T06:51:23.344 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2024-09-11T06:51:23.344 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2024-09-11T06:51:23.344 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2024-09-11T06:51:23.344 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2024-09-11T06:51:23.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.344 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2024-09-11T06:51:23.344 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2024-09-11T06:51:23.344 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2024-09-11T06:51:23.344 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2024-09-11T06:51:23.344 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2024-09-11T06:51:23.344 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T06:51:23.345 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2024-09-11T06:51:23.345 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2024-09-11T06:51:23.345 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2024-09-11T06:51:23.345 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2024-09-11T06:51:23.345 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2024-09-11T06:51:23.345 INFO:teuthology.orchestra.run.smithi071.stdout: "options": {}, 2024-09-11T06:51:23.345 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2024-09-11T06:51:23.345 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd": {} 2024-09-11T06:51:23.345 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.345 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2024-09-11T06:51:23.345 INFO:teuthology.orchestra.run.smithi071.stdout: "score_type": "Fair distribution", 2024-09-11T06:51:23.345 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 3.0099999904632568, 2024-09-11T06:51:23.345 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 3.0099999904632568, 2024-09-11T06:51:23.346 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.82999998331069946, 2024-09-11T06:51:23.346 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2.5, 2024-09-11T06:51:23.346 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2.5, 2024-09-11T06:51:23.346 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 0.85340124368667603, 2024-09-11T06:51:23.346 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 0.85000002384185791, 2024-09-11T06:51:23.346 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 0.85000002384185791 2024-09-11T06:51:23.346 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.346 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.346 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 3, 2024-09-11T06:51:23.346 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "new_default_pool21607", 2024-09-11T06:51:23.346 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2024-09-11T06:50:15.393063+0000", 2024-09-11T06:51:23.346 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 8193, 2024-09-11T06:51:23.347 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T06:51:23.347 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2024-09-11T06:51:23.347 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2024-09-11T06:51:23.347 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2024-09-11T06:51:23.347 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2024-09-11T06:51:23.347 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2024-09-11T06:51:23.347 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2024-09-11T06:51:23.347 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T06:51:23.347 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T06:51:23.347 INFO:teuthology.orchestra.run.smithi071.stdout: "is_stretch_pool": false, 2024-09-11T06:51:23.347 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2024-09-11T06:51:23.347 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2024-09-11T06:51:23.348 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 8, 2024-09-11T06:51:23.348 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 8, 2024-09-11T06:51:23.348 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 8, 2024-09-11T06:51:23.348 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 8, 2024-09-11T06:51:23.348 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 8, 2024-09-11T06:51:23.348 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2024-09-11T06:51:23.348 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2024-09-11T06:51:23.348 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2024-09-11T06:51:23.348 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2024-09-11T06:51:23.348 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2024-09-11T06:51:23.348 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2024-09-11T06:51:23.348 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2024-09-11T06:51:23.349 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.349 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "58", 2024-09-11T06:51:23.349 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2024-09-11T06:51:23.349 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T06:51:23.349 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T06:51:23.349 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2024-09-11T06:51:23.349 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2024-09-11T06:51:23.350 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 2, 2024-09-11T06:51:23.350 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 58, 2024-09-11T06:51:23.350 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2024-09-11T06:51:23.350 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2024-09-11T06:51:23.350 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2024-09-11T06:51:23.350 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2024-09-11T06:51:23.350 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2024-09-11T06:51:23.350 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2024-09-11T06:51:23.351 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2024-09-11T06:51:23.351 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2024-09-11T06:51:23.351 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2024-09-11T06:51:23.351 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2024-09-11T06:51:23.351 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2024-09-11T06:51:23.351 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T06:51:23.351 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T06:51:23.351 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T06:51:23.351 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2024-09-11T06:51:23.351 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2024-09-11T06:51:23.351 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2024-09-11T06:51:23.351 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2024-09-11T06:51:23.351 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2024-09-11T06:51:23.352 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.352 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2024-09-11T06:51:23.352 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2024-09-11T06:51:23.352 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2024-09-11T06:51:23.352 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2024-09-11T06:51:23.352 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2024-09-11T06:51:23.352 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T06:51:23.352 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2024-09-11T06:51:23.352 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2024-09-11T06:51:23.352 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2024-09-11T06:51:23.352 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2024-09-11T06:51:23.352 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2024-09-11T06:51:23.352 INFO:teuthology.orchestra.run.smithi071.stdout: "options": {}, 2024-09-11T06:51:23.353 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2024-09-11T06:51:23.353 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd": {} 2024-09-11T06:51:23.353 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.353 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2024-09-11T06:51:23.353 INFO:teuthology.orchestra.run.smithi071.stdout: "score_type": "Fair distribution", 2024-09-11T06:51:23.353 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 3.0099999904632568, 2024-09-11T06:51:23.353 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 3.0099999904632568, 2024-09-11T06:51:23.353 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.82999998331069946, 2024-09-11T06:51:23.353 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2.5, 2024-09-11T06:51:23.353 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2.5, 2024-09-11T06:51:23.353 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 0.85340124368667603, 2024-09-11T06:51:23.353 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 0.85000002384185791, 2024-09-11T06:51:23.353 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 0.85000002384185791 2024-09-11T06:51:23.353 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.354 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.354 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-11T06:51:23.354 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2024-09-11T06:51:23.354 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.354 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2024-09-11T06:51:23.354 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "3c9e35bf-d3db-49db-b10c-d2cf38c7eb7e", 2024-09-11T06:51:23.354 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-11T06:51:23.354 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-11T06:51:23.354 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-11T06:51:23.354 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 0.1204071044921875, 2024-09-11T06:51:23.354 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-11T06:51:23.354 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-11T06:51:23.354 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 11, 2024-09-11T06:51:23.355 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2024-09-11T06:51:23.355 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-11T06:51:23.355 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-11T06:51:23.355 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-11T06:51:23.355 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:23.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.355 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:23.355 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2024-09-11T06:51:23.355 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1158147777 2024-09-11T06:51:23.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.355 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:23.355 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2024-09-11T06:51:23.356 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1158147777 2024-09-11T06:51:23.356 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.356 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.356 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-11T06:51:23.356 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:23.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.356 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:23.356 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2024-09-11T06:51:23.356 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1158147777 2024-09-11T06:51:23.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.356 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:23.357 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2024-09-11T06:51:23.357 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1158147777 2024-09-11T06:51:23.357 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.357 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.357 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-11T06:51:23.357 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:23.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.357 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:23.357 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2024-09-11T06:51:23.357 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1158147777 2024-09-11T06:51:23.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.358 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:23.358 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2024-09-11T06:51:23.358 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1158147777 2024-09-11T06:51:23.358 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.358 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.358 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-11T06:51:23.358 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:23.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.358 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:23.358 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2024-09-11T06:51:23.358 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1158147777 2024-09-11T06:51:23.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.359 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:23.359 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2024-09-11T06:51:23.359 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1158147777 2024-09-11T06:51:23.359 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.359 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.359 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/1158147777", 2024-09-11T06:51:23.359 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/1158147777", 2024-09-11T06:51:23.359 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/1158147777", 2024-09-11T06:51:23.359 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/1158147777", 2024-09-11T06:51:23.359 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-11T06:51:23.359 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-11T06:51:23.359 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-11T06:51:23.360 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.360 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2024-09-11T06:51:23.360 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "693649dc-db7c-43cf-bdbe-87f6128ef311", 2024-09-11T06:51:23.360 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-11T06:51:23.360 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 0, 2024-09-11T06:51:23.360 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 0, 2024-09-11T06:51:23.360 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-11T06:51:23.360 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-11T06:51:23.360 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-11T06:51:23.360 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 11, 2024-09-11T06:51:23.360 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 86, 2024-09-11T06:51:23.360 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-11T06:51:23.361 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-11T06:51:23.361 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-11T06:51:23.361 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:23.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.361 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:23.361 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2024-09-11T06:51:23.361 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 620364628 2024-09-11T06:51:23.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.361 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:23.361 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2024-09-11T06:51:23.361 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 620364628 2024-09-11T06:51:23.361 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.362 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.362 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.362 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-11T06:51:23.362 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:23.362 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.362 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:23.362 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2024-09-11T06:51:23.362 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 620364628 2024-09-11T06:51:23.362 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.362 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.362 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:23.362 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2024-09-11T06:51:23.362 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 620364628 2024-09-11T06:51:23.363 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.363 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.363 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-11T06:51:23.363 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:23.363 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.363 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:23.363 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2024-09-11T06:51:23.363 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 620364628 2024-09-11T06:51:23.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.363 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.363 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:23.363 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2024-09-11T06:51:23.363 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 620364628 2024-09-11T06:51:23.364 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.364 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.364 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-11T06:51:23.364 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:23.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.364 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:23.364 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2024-09-11T06:51:23.364 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 620364628 2024-09-11T06:51:23.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.364 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:23.364 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2024-09-11T06:51:23.365 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 620364628 2024-09-11T06:51:23.365 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.365 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.365 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/620364628", 2024-09-11T06:51:23.365 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6811/620364628", 2024-09-11T06:51:23.365 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6815/620364628", 2024-09-11T06:51:23.365 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6813/620364628", 2024-09-11T06:51:23.365 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-11T06:51:23.365 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-11T06:51:23.365 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-11T06:51:23.365 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.366 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2024-09-11T06:51:23.366 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "89421ae7-fead-409c-83f7-ede424db6004", 2024-09-11T06:51:23.366 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-11T06:51:23.366 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-11T06:51:23.366 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-11T06:51:23.366 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-11T06:51:23.366 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-11T06:51:23.366 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-11T06:51:23.366 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 11, 2024-09-11T06:51:23.366 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 102, 2024-09-11T06:51:23.366 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-11T06:51:23.366 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-11T06:51:23.366 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-11T06:51:23.366 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:23.367 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.367 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:23.367 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2024-09-11T06:51:23.367 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926882521 2024-09-11T06:51:23.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.367 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.367 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:23.367 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2024-09-11T06:51:23.367 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926882521 2024-09-11T06:51:23.367 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.367 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.367 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-11T06:51:23.367 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:23.368 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.368 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:23.368 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2024-09-11T06:51:23.368 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926882521 2024-09-11T06:51:23.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.368 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.368 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:23.368 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2024-09-11T06:51:23.368 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926882521 2024-09-11T06:51:23.368 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.368 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.368 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-11T06:51:23.368 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:23.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.369 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:23.369 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2024-09-11T06:51:23.369 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926882521 2024-09-11T06:51:23.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.369 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:23.369 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2024-09-11T06:51:23.369 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926882521 2024-09-11T06:51:23.369 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.369 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.369 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-11T06:51:23.369 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:23.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.370 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:23.370 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2024-09-11T06:51:23.370 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926882521 2024-09-11T06:51:23.370 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.370 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:23.370 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2024-09-11T06:51:23.370 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926882521 2024-09-11T06:51:23.370 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.370 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.370 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.370 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6817/2926882521", 2024-09-11T06:51:23.370 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6819/2926882521", 2024-09-11T06:51:23.371 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/2926882521", 2024-09-11T06:51:23.371 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6821/2926882521", 2024-09-11T06:51:23.371 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-11T06:51:23.371 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-11T06:51:23.371 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-11T06:51:23.371 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.371 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2024-09-11T06:51:23.371 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "f4512e48-63f1-42b6-81be-594bd82e3210", 2024-09-11T06:51:23.371 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-11T06:51:23.371 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-11T06:51:23.371 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-11T06:51:23.371 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-11T06:51:23.372 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-11T06:51:23.372 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-11T06:51:23.372 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 11, 2024-09-11T06:51:23.372 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 99, 2024-09-11T06:51:23.372 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-11T06:51:23.372 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-11T06:51:23.372 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-11T06:51:23.372 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:23.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.372 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:23.372 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6808", 2024-09-11T06:51:23.372 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 558942315 2024-09-11T06:51:23.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.373 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:23.373 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6809", 2024-09-11T06:51:23.373 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 558942315 2024-09-11T06:51:23.373 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.373 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.373 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-11T06:51:23.373 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:23.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.373 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:23.373 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6810", 2024-09-11T06:51:23.373 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 558942315 2024-09-11T06:51:23.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.374 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:23.374 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6811", 2024-09-11T06:51:23.374 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 558942315 2024-09-11T06:51:23.374 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.374 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.374 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-11T06:51:23.374 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:23.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.374 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:23.374 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6814", 2024-09-11T06:51:23.374 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 558942315 2024-09-11T06:51:23.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.375 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:23.375 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6815", 2024-09-11T06:51:23.375 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 558942315 2024-09-11T06:51:23.375 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.375 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.375 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-11T06:51:23.375 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:23.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.376 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:23.376 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6812", 2024-09-11T06:51:23.376 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 558942315 2024-09-11T06:51:23.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.376 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:23.376 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6813", 2024-09-11T06:51:23.376 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 558942315 2024-09-11T06:51:23.376 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.376 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.376 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.89:6809/558942315", 2024-09-11T06:51:23.377 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.89:6811/558942315", 2024-09-11T06:51:23.377 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.89:6815/558942315", 2024-09-11T06:51:23.377 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.89:6813/558942315", 2024-09-11T06:51:23.377 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-11T06:51:23.377 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-11T06:51:23.377 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-11T06:51:23.377 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.377 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.377 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.377 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2024-09-11T06:51:23.377 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "52d47b6f-af30-455e-8326-e53c598911ae", 2024-09-11T06:51:23.377 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-11T06:51:23.377 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-11T06:51:23.377 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-11T06:51:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-11T06:51:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-11T06:51:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-11T06:51:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 11, 2024-09-11T06:51:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 102, 2024-09-11T06:51:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-11T06:51:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-11T06:51:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-11T06:51:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6816", 2024-09-11T06:51:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3878488003 2024-09-11T06:51:23.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.379 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:23.379 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6817", 2024-09-11T06:51:23.379 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3878488003 2024-09-11T06:51:23.379 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.379 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.379 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-11T06:51:23.379 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:23.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.379 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:23.379 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6818", 2024-09-11T06:51:23.379 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3878488003 2024-09-11T06:51:23.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.380 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:23.380 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6819", 2024-09-11T06:51:23.380 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3878488003 2024-09-11T06:51:23.380 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.380 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.380 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-11T06:51:23.380 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:23.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.380 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:23.380 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6822", 2024-09-11T06:51:23.380 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3878488003 2024-09-11T06:51:23.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.381 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:23.381 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6823", 2024-09-11T06:51:23.381 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3878488003 2024-09-11T06:51:23.381 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.381 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.381 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-11T06:51:23.381 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:23.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.381 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:23.381 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6820", 2024-09-11T06:51:23.382 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3878488003 2024-09-11T06:51:23.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.382 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:23.382 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6821", 2024-09-11T06:51:23.382 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3878488003 2024-09-11T06:51:23.382 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.382 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.382 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.89:6817/3878488003", 2024-09-11T06:51:23.382 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.89:6819/3878488003", 2024-09-11T06:51:23.382 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.89:6823/3878488003", 2024-09-11T06:51:23.382 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.89:6821/3878488003", 2024-09-11T06:51:23.382 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-11T06:51:23.383 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-11T06:51:23.383 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-11T06:51:23.383 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.383 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2024-09-11T06:51:23.383 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "6b4f9416-0ba0-4c58-883e-8cbdb362adcc", 2024-09-11T06:51:23.383 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-11T06:51:23.383 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-11T06:51:23.383 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-11T06:51:23.383 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-11T06:51:23.383 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-11T06:51:23.383 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-11T06:51:23.383 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 11, 2024-09-11T06:51:23.384 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 99, 2024-09-11T06:51:23.384 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-11T06:51:23.384 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-11T06:51:23.384 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-11T06:51:23.384 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:23.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.384 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:23.384 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6800", 2024-09-11T06:51:23.384 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2840204107 2024-09-11T06:51:23.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.384 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:23.384 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6801", 2024-09-11T06:51:23.385 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2840204107 2024-09-11T06:51:23.385 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.385 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.385 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-11T06:51:23.385 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:23.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.385 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:23.385 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6802", 2024-09-11T06:51:23.385 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2840204107 2024-09-11T06:51:23.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.386 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:23.386 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6803", 2024-09-11T06:51:23.386 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2840204107 2024-09-11T06:51:23.386 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.386 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.386 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-11T06:51:23.386 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:23.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.386 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:23.387 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6806", 2024-09-11T06:51:23.387 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2840204107 2024-09-11T06:51:23.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.387 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:23.387 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6807", 2024-09-11T06:51:23.387 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2840204107 2024-09-11T06:51:23.387 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.387 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.387 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-11T06:51:23.387 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-11T06:51:23.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.388 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-11T06:51:23.388 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6804", 2024-09-11T06:51:23.388 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2840204107 2024-09-11T06:51:23.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.388 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-11T06:51:23.388 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.89:6805", 2024-09-11T06:51:23.388 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2840204107 2024-09-11T06:51:23.388 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.388 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.388 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.89:6801/2840204107", 2024-09-11T06:51:23.389 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.89:6803/2840204107", 2024-09-11T06:51:23.389 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.89:6807/2840204107", 2024-09-11T06:51:23.389 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.89:6805/2840204107", 2024-09-11T06:51:23.389 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-11T06:51:23.389 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-11T06:51:23.389 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-11T06:51:23.389 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.389 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.389 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-11T06:51:23.389 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2024-09-11T06:51:23.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.389 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2024-09-11T06:51:23.389 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-11T06:51:23.389 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-11T06:51:23.390 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-11T06:51:23.390 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540701547738038271, 2024-09-11T06:51:23.390 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-11T06:51:23.390 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-11T06:49:00.282785+0000", 2024-09-11T06:51:23.390 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-11T06:51:23.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.390 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2024-09-11T06:51:23.390 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-11T06:51:23.390 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-11T06:51:23.390 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-11T06:51:23.390 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540701547738038271, 2024-09-11T06:51:23.390 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 65536, 2024-09-11T06:51:23.391 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-11T06:49:00.381259+0000", 2024-09-11T06:51:23.391 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-11T06:51:23.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.391 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2024-09-11T06:51:23.391 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-11T06:51:23.391 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-11T06:51:23.391 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-11T06:51:23.391 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540701547738038271, 2024-09-11T06:51:23.391 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-11T06:51:23.391 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-11T06:49:00.986977+0000", 2024-09-11T06:51:23.391 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-11T06:51:23.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.392 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2024-09-11T06:51:23.392 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-11T06:51:23.392 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-11T06:51:23.392 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-11T06:51:23.392 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540701547738038271, 2024-09-11T06:51:23.392 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-11T06:51:23.392 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-11T06:49:00.732329+0000", 2024-09-11T06:51:23.392 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-11T06:51:23.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.392 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2024-09-11T06:51:23.392 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-11T06:51:23.393 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-11T06:51:23.393 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-11T06:51:23.393 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540701547738038271, 2024-09-11T06:51:23.393 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-11T06:51:23.393 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-11T06:49:00.844988+0000", 2024-09-11T06:51:23.393 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-11T06:51:23.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.393 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2024-09-11T06:51:23.393 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-11T06:51:23.393 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-11T06:51:23.393 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-11T06:51:23.393 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540701547738038271, 2024-09-11T06:51:23.393 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-11T06:51:23.394 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-11T06:49:00.639989+0000", 2024-09-11T06:51:23.394 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-11T06:51:23.394 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.394 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-11T06:51:23.394 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [ 2024-09-11T06:51:23.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.394 INFO:teuthology.orchestra.run.smithi071.stdout: "pgid": "3.3", 2024-09-11T06:51:23.394 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2024-09-11T06:51:23.394 INFO:teuthology.orchestra.run.smithi071.stdout: 2, 2024-09-11T06:51:23.394 INFO:teuthology.orchestra.run.smithi071.stdout: 0 2024-09-11T06:51:23.394 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.394 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.394 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-11T06:51:23.395 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [ 2024-09-11T06:51:23.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.395 INFO:teuthology.orchestra.run.smithi071.stdout: "pgid": "3.5", 2024-09-11T06:51:23.395 INFO:teuthology.orchestra.run.smithi071.stdout: "mappings": [ 2024-09-11T06:51:23.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:23.395 INFO:teuthology.orchestra.run.smithi071.stdout: "from": 2, 2024-09-11T06:51:23.395 INFO:teuthology.orchestra.run.smithi071.stdout: "to": 3 2024-09-11T06:51:23.395 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.395 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:23.395 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.395 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-11T06:51:23.395 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2024-09-11T06:51:23.395 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2024-09-11T06:51:23.396 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2024-09-11T06:51:23.396 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2024-09-11T06:51:23.396 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2024-09-11T06:51:23.396 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2024-09-11T06:51:23.396 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2024-09-11T06:51:23.396 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2024-09-11T06:51:23.396 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2024-09-11T06:51:23.396 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2024-09-11T06:51:23.396 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2024-09-11T06:51:23.396 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2024-09-11T06:51:23.396 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:23.397 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2024-09-11T06:51:23.397 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2024-09-11T06:51:23.397 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2024-09-11T06:51:23.397 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2024-09-11T06:51:23.397 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2024-09-11T06:51:23.397 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2024-09-11T06:51:23.397 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2024-09-11T06:51:23.397 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2024-09-11T06:51:23.397 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2024-09-11T06:51:23.397 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2024-09-11T06:51:23.397 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2024-09-11T06:51:23.397 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:23.397 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:23.398 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:23.326+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:23.399 INFO:tasks.thrashosds.thrasher:j is {'epoch': 103, 'fsid': 'bd5dcb33-e9f7-4a5f-a26c-a451452e8aa2', 'created': '2024-09-11T06:48:52.191706+0000', 'modified': '2024-09-11T06:51:21.977212+0000', 'last_up_change': '2024-09-11T06:49:04.007203+0000', 'last_in_change': '2024-09-11T06:51:17.949107+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T06:49:05.084515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.853401243686676, 'average_primary_affinity': 0.8500000238418579, 'average_primary_affinity_weighted': 0.8500000238418579}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-11T06:49:09.530167+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 7, 'snap_epoch': 91, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.009999990463257, 'score_stable': 3.009999990463257, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 0.853401243686676, 'average_primary_affinity': 0.8500000238418579, 'average_primary_affinity_weighted': 0.8500000238418579}}, {'pool': 3, 'pool_name': 'new_default_pool21607', 'create_time': '2024-09-11T06:50:15.393063+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 58, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.009999990463257, 'score_stable': 3.009999990463257, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 0.853401243686676, 'average_primary_affinity': 0.8500000238418579, 'average_primary_affinity_weighted': 0.8500000238418579}}], 'osds': [{'osd': 0, 'uuid': '3c9e35bf-d3db-49db-b10c-d2cf38c7eb7e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.1204071044921875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 101, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6800', 'nonce': 1158147777}, {'type': 'v1', 'addr': '172.21.15.71:6801', 'nonce': 1158147777}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6802', 'nonce': 1158147777}, {'type': 'v1', 'addr': '172.21.15.71:6803', 'nonce': 1158147777}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6806', 'nonce': 1158147777}, {'type': 'v1', 'addr': '172.21.15.71:6807', 'nonce': 1158147777}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6804', 'nonce': 1158147777}, {'type': 'v1', 'addr': '172.21.15.71:6805', 'nonce': 1158147777}]}, 'public_addr': '172.21.15.71:6801/1158147777', 'cluster_addr': '172.21.15.71:6803/1158147777', 'heartbeat_back_addr': '172.21.15.71:6807/1158147777', 'heartbeat_front_addr': '172.21.15.71:6805/1158147777', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '693649dc-db7c-43cf-bdbe-87f6128ef311', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6808', 'nonce': 620364628}, {'type': 'v1', 'addr': '172.21.15.71:6809', 'nonce': 620364628}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6810', 'nonce': 620364628}, {'type': 'v1', 'addr': '172.21.15.71:6811', 'nonce': 620364628}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6814', 'nonce': 620364628}, {'type': 'v1', 'addr': '172.21.15.71:6815', 'nonce': 620364628}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6812', 'nonce': 620364628}, {'type': 'v1', 'addr': '172.21.15.71:6813', 'nonce': 620364628}]}, 'public_addr': '172.21.15.71:6809/620364628', 'cluster_addr': '172.21.15.71:6811/620364628', 'heartbeat_back_addr': '172.21.15.71:6815/620364628', 'heartbeat_front_addr': '172.21.15.71:6813/620364628', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '89421ae7-fead-409c-83f7-ede424db6004', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 102, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6816', 'nonce': 2926882521}, {'type': 'v1', 'addr': '172.21.15.71:6817', 'nonce': 2926882521}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6818', 'nonce': 2926882521}, {'type': 'v1', 'addr': '172.21.15.71:6819', 'nonce': 2926882521}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6822', 'nonce': 2926882521}, {'type': 'v1', 'addr': '172.21.15.71:6823', 'nonce': 2926882521}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6820', 'nonce': 2926882521}, {'type': 'v1', 'addr': '172.21.15.71:6821', 'nonce': 2926882521}]}, 'public_addr': '172.21.15.71:6817/2926882521', 'cluster_addr': '172.21.15.71:6819/2926882521', 'heartbeat_back_addr': '172.21.15.71:6823/2926882521', 'heartbeat_front_addr': '172.21.15.71:6821/2926882521', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'f4512e48-63f1-42b6-81be-594bd82e3210', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6808', 'nonce': 558942315}, {'type': 'v1', 'addr': '172.21.15.89:6809', 'nonce': 558942315}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6810', 'nonce': 558942315}, {'type': 'v1', 'addr': '172.21.15.89:6811', 'nonce': 558942315}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6814', 'nonce': 558942315}, {'type': 'v1', 'addr': '172.21.15.89:6815', 'nonce': 558942315}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6812', 'nonce': 558942315}, {'type': 'v1', 'addr': '172.21.15.89:6813', 'nonce': 558942315}]}, 'public_addr': '172.21.15.89:6809/558942315', 'cluster_addr': '172.21.15.89:6811/558942315', 'heartbeat_back_addr': '172.21.15.89:6815/558942315', 'heartbeat_front_addr': '172.21.15.89:6813/558942315', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '52d47b6f-af30-455e-8326-e53c598911ae', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 102, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6816', 'nonce': 3878488003}, {'type': 'v1', 'addr': '172.21.15.89:6817', 'nonce': 3878488003}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6818', 'nonce': 3878488003}, {'type': 'v1', 'addr': '172.21.15.89:6819', 'nonce': 3878488003}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6822', 'nonce': 3878488003}, {'type': 'v1', 'addr': '172.21.15.89:6823', 'nonce': 3878488003}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6820', 'nonce': 3878488003}, {'type': 'v1', 'addr': '172.21.15.89:6821', 'nonce': 3878488003}]}, 'public_addr': '172.21.15.89:6817/3878488003', 'cluster_addr': '172.21.15.89:6819/3878488003', 'heartbeat_back_addr': '172.21.15.89:6823/3878488003', 'heartbeat_front_addr': '172.21.15.89:6821/3878488003', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '6b4f9416-0ba0-4c58-883e-8cbdb362adcc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6800', 'nonce': 2840204107}, {'type': 'v1', 'addr': '172.21.15.89:6801', 'nonce': 2840204107}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6802', 'nonce': 2840204107}, {'type': 'v1', 'addr': '172.21.15.89:6803', 'nonce': 2840204107}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6806', 'nonce': 2840204107}, {'type': 'v1', 'addr': '172.21.15.89:6807', 'nonce': 2840204107}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6804', 'nonce': 2840204107}, {'type': 'v1', 'addr': '172.21.15.89:6805', 'nonce': 2840204107}]}, 'public_addr': '172.21.15.89:6801/2840204107', 'cluster_addr': '172.21.15.89:6803/2840204107', 'heartbeat_back_addr': '172.21.15.89:6807/2840204107', 'heartbeat_front_addr': '172.21.15.89:6805/2840204107', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T06:49:00.282785+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-11T06:49:00.381259+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T06:49:00.986977+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T06:49:00.732329+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T06:49:00.844988+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T06:49:00.639989+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.3', 'osds': [2, 0]}], 'pg_upmap_items': [{'pgid': '3.5', 'mappings': [{'from': 2, 'to': 3}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-11T06:51:23.400 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:51:23.441 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:23.438+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:23.503 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:23.497+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:23.504 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:23.505 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:23.518 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:51:23.604 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:23.597+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:23.625 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:23.625 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:23.638 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:51:23.705 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:23.698+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:23.724 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:51:23.725 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:51:23.741 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2024-09-11T06:51:23.043216+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":151655643,"num_objects":82,"num_object_clones":20,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":12,"num_read":5554,"num_read_kb":116073,"num_write":992,"num_write_kb":275324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":207882919,"num_keys_recovered":35,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":966,"ondisk_log_size":966,"up":34,"acting":32,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":363748,"kb_used_data":211968,"kb_used_omap":7,"kb_used_meta":151672,"kb_avail":471495452,"statfs":{"total":483183820800,"available":482811342848,"internally_reserved":0,"allocated":217055232,"data_stored":234398587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7971,"internal_metadata":155312349},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":410,"num_read_kb":342,"num_write":14,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":20971744,"num_keys_recovered":13,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.072810"},"pg_stats":[{"pgid":"1.0","version":"14'117","reported_seq":338,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-11T06:51:18.973277+0000","last_change":"2024-09-11T06:50:37.941212+0000","last_active":"2024-09-11T06:51:18.973277+0000","last_peered":"2024-09-11T06:51:18.973277+0000","last_clean":"2024-09-11T06:51:18.973277+0000","last_became_active":"2024-09-11T06:50:37.940448+0000","last_became_peered":"2024-09-11T06:50:37.940448+0000","last_unstale":"2024-09-11T06:51:18.973277+0000","last_undegraded":"2024-09-11T06:51:18.973277+0000","last_fullsized":"2024-09-11T06:51:18.973277+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_clean_scrub_stamp":"2024-09-11T06:49:05.088839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:48:59.833858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'95","reported_seq":299,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-11T06:51:18.979969+0000","last_change":"2024-09-11T06:50:47.401354+0000","last_active":"2024-09-11T06:51:18.979969+0000","last_peered":"2024-09-11T06:51:18.979969+0000","last_clean":"2024-09-11T06:51:18.979969+0000","last_became_active":"2024-09-11T06:50:47.306211+0000","last_became_peered":"2024-09-11T06:50:47.306211+0000","last_unstale":"2024-09-11T06:51:18.979969+0000","last_undegraded":"2024-09-11T06:51:18.979969+0000","last_fullsized":"2024-09-11T06:51:18.979969+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:06:28.664588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011212469999999999,"stat_sum":{"num_bytes":12582950,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":334,"num_read_kb":8458,"num_write":77,"num_write_kb":28698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":25165862,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"58'2","reported_seq":101,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-11T06:51:19.003253+0000","last_change":"2024-09-11T06:51:19.002589+0000","last_active":"2024-09-11T06:51:19.003253+0000","last_peered":"2024-09-11T06:51:19.003253+0000","last_clean":"2024-09-11T06:51:19.003253+0000","last_became_active":"2024-09-11T06:51:18.986483+0000","last_became_peered":"2024-09-11T06:51:18.986483+0000","last_unstale":"2024-09-11T06:51:19.003253+0000","last_undegraded":"2024-09-11T06:51:19.003253+0000","last_fullsized":"2024-09-11T06:51:19.003253+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:14:27.491460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011764950000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"89'63","reported_seq":245,"reported_epoch":100,"state":"active+recovering","last_fresh":"2024-09-11T06:51:19.086242+0000","last_change":"2024-09-11T06:51:19.064773+0000","last_active":"2024-09-11T06:51:19.086242+0000","last_peered":"2024-09-11T06:51:19.086242+0000","last_clean":"2024-09-11T06:51:15.987353+0000","last_became_active":"2024-09-11T06:51:18.986870+0000","last_became_peered":"2024-09-11T06:51:18.986870+0000","last_unstale":"2024-09-11T06:51:19.086242+0000","last_undegraded":"2024-09-11T06:51:19.086242+0000","last_fullsized":"2024-09-11T06:51:19.086242+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077615599999999996,"stat_sum":{"num_bytes":20971520,"num_objects":9,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":150,"num_read_kb":23060,"num_write":59,"num_write_kb":45056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":46137344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"0,3","objects":2},{"shards":"1,3","objects":7}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":102,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-11T06:51:18.978267+0000","last_change":"2024-09-11T06:50:19.703255+0000","last_active":"2024-09-11T06:51:18.978267+0000","last_peered":"2024-09-11T06:51:18.978267+0000","last_clean":"2024-09-11T06:51:18.978267+0000","last_became_active":"2024-09-11T06:50:17.089296+0000","last_became_peered":"2024-09-11T06:50:17.089296+0000","last_unstale":"2024-09-11T06:51:18.978267+0000","last_undegraded":"2024-09-11T06:51:18.978267+0000","last_fullsized":"2024-09-11T06:51:18.978267+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:42:18.684413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001159373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"98'175","reported_seq":1718,"reported_epoch":100,"state":"remapped+peering","last_fresh":"2024-09-11T06:51:18.973734+0000","last_change":"2024-09-11T06:51:18.973734+0000","last_active":"2024-09-11T06:51:17.967242+0000","last_peered":"2024-09-11T06:51:17.965682+0000","last_clean":"2024-09-11T06:51:17.965682+0000","last_became_active":"2024-09-11T06:49:55.948499+0000","last_became_peered":"2024-09-11T06:49:55.948499+0000","last_unstale":"2024-09-11T06:51:18.973734+0000","last_undegraded":"2024-09-11T06:51:18.973734+0000","last_fullsized":"2024-09-11T06:51:18.973734+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010163780000000001,"stat_sum":{"num_bytes":29360244,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":3146,"num_read_kb":25965,"num_write":90,"num_write_kb":41039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"79'1","reported_seq":75,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-11T06:51:19.057824+0000","last_change":"2024-09-11T06:51:19.056939+0000","last_active":"2024-09-11T06:51:19.057824+0000","last_peered":"2024-09-11T06:51:19.057824+0000","last_clean":"2024-09-11T06:51:19.057824+0000","last_became_active":"2024-09-11T06:51:18.984308+0000","last_became_peered":"2024-09-11T06:51:18.984308+0000","last_unstale":"2024-09-11T06:51:19.057824+0000","last_undegraded":"2024-09-11T06:51:19.057824+0000","last_fullsized":"2024-09-11T06:51:19.057824+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:40:22.221668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088028100000000001,"stat_sum":{"num_bytes":35,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":35,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"93'6","reported_seq":136,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-11T06:51:19.062114+0000","last_change":"2024-09-11T06:51:19.062114+0000","last_active":"2024-09-11T06:51:19.062114+0000","last_peered":"2024-09-11T06:51:19.062114+0000","last_clean":"2024-09-11T06:51:19.062114+0000","last_became_active":"2024-09-11T06:51:18.987217+0000","last_became_peered":"2024-09-11T06:51:18.987217+0000","last_unstale":"2024-09-11T06:51:19.062114+0000","last_undegraded":"2024-09-11T06:51:19.062114+0000","last_fullsized":"2024-09-11T06:51:19.062114+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:09.047370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001910244,"stat_sum":{"num_bytes":8388743,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":58,"num_read_kb":57,"num_write":6,"num_write_kb":8196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":16777427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"56'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-11T06:51:18.972294+0000","last_change":"2024-09-11T06:51:05.893192+0000","last_active":"2024-09-11T06:51:18.972294+0000","last_peered":"2024-09-11T06:51:18.972294+0000","last_clean":"2024-09-11T06:51:18.972294+0000","last_became_active":"2024-09-11T06:51:05.889184+0000","last_became_peered":"2024-09-11T06:51:05.889184+0000","last_unstale":"2024-09-11T06:51:18.972294+0000","last_undegraded":"2024-09-11T06:51:18.972294+0000","last_fullsized":"2024-09-11T06:51:18.972294+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:44:46.365158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-11T06:51:18.994195+0000","last_change":"2024-09-11T06:51:18.988850+0000","last_active":"2024-09-11T06:51:18.994195+0000","last_peered":"2024-09-11T06:51:18.994195+0000","last_clean":"2024-09-11T06:51:18.994195+0000","last_became_active":"2024-09-11T06:51:18.981623+0000","last_became_peered":"2024-09-11T06:51:18.981623+0000","last_unstale":"2024-09-11T06:51:18.994195+0000","last_undegraded":"2024-09-11T06:51:18.994195+0000","last_fullsized":"2024-09-11T06:51:18.994195+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:06.338033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":32,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"98'169","reported_seq":391,"reported_epoch":98,"state":"remapped+peering","last_fresh":"2024-09-11T06:51:17.192824+0000","last_change":"2024-09-11T06:51:18.980548+0000","last_active":"2024-09-11T06:51:18.980548+0000","last_peered":"2024-09-11T06:51:17.192824+0000","last_clean":"2024-09-11T06:51:17.192824+0000","last_became_active":"2024-09-11T06:49:56.121998+0000","last_became_peered":"2024-09-11T06:49:56.121998+0000","last_unstale":"2024-09-11T06:51:17.192824+0000","last_undegraded":"2024-09-11T06:51:17.192824+0000","last_fullsized":"2024-09-11T06:51:17.192824+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583028,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":802,"num_read_kb":8829,"num_write":273,"num_write_kb":20700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":4,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"79'4","reported_seq":90,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-11T06:51:18.998151+0000","last_change":"2024-09-11T06:51:18.997451+0000","last_active":"2024-09-11T06:51:18.998151+0000","last_peered":"2024-09-11T06:51:18.998151+0000","last_clean":"2024-09-11T06:51:18.998151+0000","last_became_active":"2024-09-11T06:51:18.982117+0000","last_became_peered":"2024-09-11T06:51:18.982117+0000","last_unstale":"2024-09-11T06:51:18.998151+0000","last_undegraded":"2024-09-11T06:51:18.998151+0000","last_fullsized":"2024-09-11T06:51:18.998151+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:03:43.947723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085501399999999997,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"93'145","reported_seq":468,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-11T06:51:18.973581+0000","last_change":"2024-09-11T06:50:35.337013+0000","last_active":"2024-09-11T06:51:18.973581+0000","last_peered":"2024-09-11T06:51:18.973581+0000","last_clean":"2024-09-11T06:51:18.973581+0000","last_became_active":"2024-09-11T06:50:35.236333+0000","last_became_peered":"2024-09-11T06:50:35.236333+0000","last_unstale":"2024-09-11T06:51:18.973581+0000","last_undegraded":"2024-09-11T06:51:18.973581+0000","last_fullsized":"2024-09-11T06:51:18.973581+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:36:17.699073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095172900000000001,"stat_sum":{"num_bytes":29425734,"num_objects":13,"num_object_clones":5,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":495,"num_read_kb":19494,"num_write":140,"num_write_kb":45180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":41943104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":108,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-11T06:51:18.984652+0000","last_change":"2024-09-11T06:51:18.984652+0000","last_active":"2024-09-11T06:51:18.984652+0000","last_peered":"2024-09-11T06:51:18.984652+0000","last_clean":"2024-09-11T06:51:18.984652+0000","last_became_active":"2024-09-11T06:51:18.983773+0000","last_became_peered":"2024-09-11T06:51:18.983773+0000","last_unstale":"2024-09-11T06:51:18.984652+0000","last_undegraded":"2024-09-11T06:51:18.984652+0000","last_fullsized":"2024-09-11T06:51:18.984652+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:40:49.527081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053029399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"89'45","reported_seq":245,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-11T06:51:19.053797+0000","last_change":"2024-09-11T06:51:19.052980+0000","last_active":"2024-09-11T06:51:19.053797+0000","last_peered":"2024-09-11T06:51:19.053797+0000","last_clean":"2024-09-11T06:51:19.053797+0000","last_became_active":"2024-09-11T06:51:18.984613+0000","last_became_peered":"2024-09-11T06:51:18.984613+0000","last_unstale":"2024-09-11T06:51:19.053797+0000","last_undegraded":"2024-09-11T06:51:19.053797+0000","last_fullsized":"2024-09-11T06:51:19.053797+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:13:12.126436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388646,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":118,"num_read_kb":8286,"num_write":60,"num_write_kb":16420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":25165938,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":108,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-11T06:51:18.987943+0000","last_change":"2024-09-11T06:51:18.987943+0000","last_active":"2024-09-11T06:51:18.987943+0000","last_peered":"2024-09-11T06:51:18.987943+0000","last_clean":"2024-09-11T06:51:18.987943+0000","last_became_active":"2024-09-11T06:51:18.987264+0000","last_became_peered":"2024-09-11T06:51:18.987264+0000","last_unstale":"2024-09-11T06:51:18.987943+0000","last_undegraded":"2024-09-11T06:51:18.987943+0000","last_fullsized":"2024-09-11T06:51:18.987943+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:50:16.062872+0000","last_clean_scrub_stamp":"2024-09-11T06:50:16.062872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:51:46.244258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059833200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"93'142","reported_seq":252,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-11T06:51:18.973451+0000","last_change":"2024-09-11T06:50:35.519269+0000","last_active":"2024-09-11T06:51:18.973451+0000","last_peered":"2024-09-11T06:51:18.973451+0000","last_clean":"2024-09-11T06:51:18.973451+0000","last_became_active":"2024-09-11T06:50:35.238011+0000","last_became_peered":"2024-09-11T06:50:35.238011+0000","last_unstale":"2024-09-11T06:51:18.973451+0000","last_undegraded":"2024-09-11T06:51:18.973451+0000","last_fullsized":"2024-09-11T06:51:18.973451+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:49:49.483725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001403348,"stat_sum":{"num_bytes":29364340,"num_objects":15,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":305,"num_read_kb":21800,"num_write":122,"num_write_kb":67613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":50331648,"num_keys_recovered":16,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":70,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":30,"num_read_kb":24,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":124,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":16,"acting":16,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":151065205,"num_objects":74,"num_object_clones":20,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":12,"num_read":5408,"num_read_kb":115949,"num_write":827,"num_write_kb":272902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":205521323,"num_keys_recovered":25,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290856960,"data_stored":311772394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":840,"ondisk_log_size":840,"up":16,"acting":14,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640285,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48668,"kb_used_data":21640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94323172,"statfs":{"total":96636764160,"available":96586928128,"internally_reserved":0,"allocated":22159360,"data_stored":21422048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640286,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58976,"kb_used_data":27788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94312864,"statfs":{"total":96636764160,"available":96576372736,"internally_reserved":0,"allocated":28454912,"data_stored":27778964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640286,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":114932,"kb_used_data":83744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94256908,"statfs":{"total":96636764160,"available":96519073792,"internally_reserved":0,"allocated":85753856,"data_stored":95519280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640287,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69272,"kb_used_data":38084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94302568,"statfs":{"total":96636764160,"available":96565829632,"internally_reserved":0,"allocated":38998016,"data_stored":38264797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":47244640287,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640288,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71900,"kb_used_data":40712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94299940,"statfs":{"total":96636764160,"available":96563138560,"internally_reserved":0,"allocated":41689088,"data_stored":51413498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53571584,"data_stored":64004417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":58802176,"data_stored":58786024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":37826560,"data_stored":37814426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":83988480,"data_stored":94478560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":35680256,"data_stored":35717274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20987904,"data_stored":20971693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":35,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":35,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":35,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":35,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:51:23.743 INFO:tasks.thrashosds.thrasher:Setting 3.7 to [2, 4] 2024-09-11T06:51:23.743 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.7', '2', '4'] 2024-09-11T06:51:23.743 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.7 2 4 2024-09-11T06:51:23.751 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:23.751 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:23.764 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:51:23.805 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:23.805+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:23.905 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:23.902+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:23.932 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:23.933 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:23.945 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:51:24.006 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:24.001+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:24.009 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is set 2024-09-11T06:51:24.079 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:24.080 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:24.093 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:51:24.137 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:24.133+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:24.207 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:24.201+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:24.258 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:24.258 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:24.272 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:51:24.308 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:24.302+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:24.410 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:24.406+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:24.436 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:24.437 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:24.451 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:51:24.511 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:24.505+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:24.576 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:24.576 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:24.590 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:51:24.633 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:24.629+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:24.713 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:24.709+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:24.748 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:51:24.749 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:24.749 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:24.762 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:51:24.813 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:24.810+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:24.900 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:51:24.900 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:51:24.915 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:24.910+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:24.917 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:51:24.918 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:51:24.921 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:24.921 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:24.934 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:51:24.935 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:51:24.936 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:51:24.952 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:51:24.953 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:51:24.972 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:51:24.973 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:51:24.989 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:51:24.989 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:51:25.015 INFO:teuthology.orchestra.run.smithi071.stderr:set 3.7 pg_upmap mapping to [2,4] 2024-09-11T06:51:25.016 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:25.009+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:25.062 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:25.062 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:25.062 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:51:25.062 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:25.076 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:51:25.117 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:25.113+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:25.218 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:25.214+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:25.235 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:25.236 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:25.236 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:51:25.236 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:51:25.236 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:25.250 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:51:25.318 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:25.313+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:25.371 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:25.372 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:51:25.372 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:51:25.372 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:51:25.372 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.372 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:51:25.372 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:51:25.372 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 134.137706321, 2024-09-11T06:51:25.372 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010553425999999999, 2024-09-11T06:51:25.372 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:25.372 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:25.372 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:25.373 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:25.373 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:51:25.373 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:51:25.373 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:51:25.373 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.373 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:25.373 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.373 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:25.373 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:51:25.373 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.373 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.373 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.373 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:25.373 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:51:25.374 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.374 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.374 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.374 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:25.374 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230274+0000", 2024-09-11T06:51:25.374 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.533e-06 2024-09-11T06:51:25.374 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.374 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.374 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:25.374 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230285+0000", 2024-09-11T06:51:25.374 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0540000000000001e-05 2024-09-11T06:51:25.374 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.374 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.374 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:25.375 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230309+0000", 2024-09-11T06:51:25.375 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4223000000000002e-05 2024-09-11T06:51:25.375 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.375 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.375 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.375 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230327+0000", 2024-09-11T06:51:25.375 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:51:25.375 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.375 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.375 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.375 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230478+0000", 2024-09-11T06:51:25.375 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000151459 2024-09-11T06:51:25.375 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.376 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.376 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:25.376 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230490+0000", 2024-09-11T06:51:25.376 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.147e-05 2024-09-11T06:51:25.376 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.376 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.376 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.376 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231803+0000", 2024-09-11T06:51:25.376 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001313158 2024-09-11T06:51:25.376 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.376 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.376 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:51:25.376 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231817+0000", 2024-09-11T06:51:25.376 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4197e-05 2024-09-11T06:51:25.377 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.377 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.377 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.377 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.235945+0000", 2024-09-11T06:51:25.377 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0041282480000000002 2024-09-11T06:51:25.377 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.377 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.377 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:25.377 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236312+0000", 2024-09-11T06:51:25.377 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036652700000000001 2024-09-11T06:51:25.377 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.377 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.377 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:25.377 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236689+0000", 2024-09-11T06:51:25.378 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000377117 2024-09-11T06:51:25.378 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.378 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.378 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:25.378 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238460+0000", 2024-09-11T06:51:25.378 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001770918 2024-09-11T06:51:25.378 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.378 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.378 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:25.378 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240627+0000", 2024-09-11T06:51:25.378 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021670779999999998 2024-09-11T06:51:25.378 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.378 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.378 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:25.378 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240751+0000", 2024-09-11T06:51:25.379 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012384900000000001 2024-09-11T06:51:25.379 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.379 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.379 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:25.379 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240821+0000", 2024-09-11T06:51:25.379 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0516e-05 2024-09-11T06:51:25.379 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.379 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:25.379 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.379 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.379 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.379 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-11T06:51:25.379 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:51:25.379 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 133.13553547800001, 2024-09-11T06:51:25.379 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069904770000000001, 2024-09-11T06:51:25.380 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:25.380 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:25.380 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:25.380 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:25.380 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:51:25.380 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:51:25.380 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 5 2024-09-11T06:51:25.380 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.380 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:25.380 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.380 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:25.380 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:51:25.380 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.381 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.381 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.381 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:25.381 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:51:25.381 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.381 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.381 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.381 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:25.381 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232442+0000", 2024-09-11T06:51:25.381 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.044e-06 2024-09-11T06:51:25.381 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.381 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.381 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:25.381 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232460+0000", 2024-09-11T06:51:25.382 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7863000000000002e-05 2024-09-11T06:51:25.382 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.382 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.382 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:25.382 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232492+0000", 2024-09-11T06:51:25.382 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2812999999999999e-05 2024-09-11T06:51:25.382 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.382 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.382 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.382 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232510+0000", 2024-09-11T06:51:25.382 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7552999999999999e-05 2024-09-11T06:51:25.382 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.382 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.382 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.382 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232640+0000", 2024-09-11T06:51:25.382 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013029900000000001 2024-09-11T06:51:25.383 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.383 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.383 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:25.383 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232650+0000", 2024-09-11T06:51:25.383 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0023999999999999e-05 2024-09-11T06:51:25.383 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.383 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.383 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.383 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235641+0000", 2024-09-11T06:51:25.383 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00299089 2024-09-11T06:51:25.383 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.383 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.383 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:25.383 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235980+0000", 2024-09-11T06:51:25.384 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000338907 2024-09-11T06:51:25.384 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.384 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.384 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:25.384 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.236351+0000", 2024-09-11T06:51:25.384 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037087699999999998 2024-09-11T06:51:25.384 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.384 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.384 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:25.384 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238074+0000", 2024-09-11T06:51:25.384 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001722859 2024-09-11T06:51:25.384 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.384 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.385 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:25.385 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239160+0000", 2024-09-11T06:51:25.385 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001086074 2024-09-11T06:51:25.385 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.385 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.385 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:25.385 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239299+0000", 2024-09-11T06:51:25.385 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013933500000000001 2024-09-11T06:51:25.385 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.385 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.385 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:25.385 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239429+0000", 2024-09-11T06:51:25.385 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129939 2024-09-11T06:51:25.385 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.386 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:25.386 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.386 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.386 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.386 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-11T06:51:25.386 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:51:25.386 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 132.121447526, 2024-09-11T06:51:25.386 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069253250000000004, 2024-09-11T06:51:25.386 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:25.386 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:25.386 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:25.386 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:25.386 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:51:25.387 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:51:25.387 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 7 2024-09-11T06:51:25.387 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.387 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:25.387 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.387 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:25.387 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:51:25.387 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.387 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.387 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.387 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:25.387 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:51:25.387 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.387 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.388 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.388 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:25.388 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246530+0000", 2024-09-11T06:51:25.388 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9469999999999999e-06 2024-09-11T06:51:25.388 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.388 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.388 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:25.388 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246545+0000", 2024-09-11T06:51:25.388 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5973000000000001e-05 2024-09-11T06:51:25.388 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.388 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.388 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:25.388 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246578+0000", 2024-09-11T06:51:25.388 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2759999999999998e-05 2024-09-11T06:51:25.389 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.389 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.389 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.389 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246595+0000", 2024-09-11T06:51:25.389 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6939000000000002e-05 2024-09-11T06:51:25.389 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.389 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.389 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.389 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246770+0000", 2024-09-11T06:51:25.389 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000174723 2024-09-11T06:51:25.389 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.389 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.389 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:25.389 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246789+0000", 2024-09-11T06:51:25.389 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9159000000000001e-05 2024-09-11T06:51:25.390 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.390 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.390 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.390 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250337+0000", 2024-09-11T06:51:25.390 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035480759999999998 2024-09-11T06:51:25.390 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.390 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.390 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:25.390 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250549+0000", 2024-09-11T06:51:25.390 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021192199999999999 2024-09-11T06:51:25.390 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.390 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.390 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:25.390 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250750+0000", 2024-09-11T06:51:25.390 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020104199999999999 2024-09-11T06:51:25.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.391 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.391 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:25.391 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.252455+0000", 2024-09-11T06:51:25.391 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001705249 2024-09-11T06:51:25.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.391 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.391 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:25.391 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253305+0000", 2024-09-11T06:51:25.391 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00084941200000000002 2024-09-11T06:51:25.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.391 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.391 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:25.391 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253388+0000", 2024-09-11T06:51:25.391 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3642999999999999e-05 2024-09-11T06:51:25.392 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.392 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.392 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:25.392 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253452+0000", 2024-09-11T06:51:25.392 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3479999999999995e-05 2024-09-11T06:51:25.392 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.392 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:25.392 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.392 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.392 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.392 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:57 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:25.392 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.568529+0000", 2024-09-11T06:51:25.392 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 36.799444606000002, 2024-09-11T06:51:25.393 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004228229, 2024-09-11T06:51:25.393 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:25.393 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:25.393 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:25.393 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:25.393 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:25.393 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:25.393 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 57 2024-09-11T06:51:25.393 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.393 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:25.393 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.393 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:25.393 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568529+0000", 2024-09-11T06:51:25.393 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.393 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.394 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.394 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:25.394 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568529+0000", 2024-09-11T06:51:25.394 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.394 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.394 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.394 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:25.394 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568532+0000", 2024-09-11T06:51:25.394 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0000000000000001e-06 2024-09-11T06:51:25.394 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.394 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.394 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:25.394 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568547+0000", 2024-09-11T06:51:25.394 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4219999999999999e-05 2024-09-11T06:51:25.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.395 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.395 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:25.395 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568580+0000", 2024-09-11T06:51:25.395 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.3328999999999999e-05 2024-09-11T06:51:25.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.395 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.395 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.395 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568596+0000", 2024-09-11T06:51:25.395 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5990000000000001e-05 2024-09-11T06:51:25.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.395 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.395 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.395 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568728+0000", 2024-09-11T06:51:25.396 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013175400000000001 2024-09-11T06:51:25.396 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.396 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:25.396 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568738+0000", 2024-09-11T06:51:25.396 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0196e-05 2024-09-11T06:51:25.396 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.396 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.396 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.572464+0000", 2024-09-11T06:51:25.396 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0037264160000000002 2024-09-11T06:51:25.396 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.397 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:25.397 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.572584+0000", 2024-09-11T06:51:25.397 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000119438 2024-09-11T06:51:25.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.397 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.397 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:25.397 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.572758+0000", 2024-09-11T06:51:25.397 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017388599999999999 2024-09-11T06:51:25.397 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.397 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:25.397 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.397 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.397 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5374.0:21 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [watch unwatch cookie 94119567275296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:25.398 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:12.469500+0000", 2024-09-11T06:51:25.398 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 12.898473778, 2024-09-11T06:51:25.398 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0026485419999999998, 2024-09-11T06:51:25.398 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:25.398 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:25.398 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:25.398 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:25.398 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5374", 2024-09-11T06:51:25.398 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1937077541", 2024-09-11T06:51:25.398 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 21 2024-09-11T06:51:25.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.398 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:25.398 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.399 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:25.399 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469500+0000", 2024-09-11T06:51:25.399 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.399 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.399 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:25.399 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469500+0000", 2024-09-11T06:51:25.399 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.399 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.399 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:25.399 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469501+0000", 2024-09-11T06:51:25.399 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.33e-07 2024-09-11T06:51:25.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.400 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.400 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:25.400 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469506+0000", 2024-09-11T06:51:25.400 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.8690000000000003e-06 2024-09-11T06:51:25.400 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.400 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.400 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:25.400 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469519+0000", 2024-09-11T06:51:25.400 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3077e-05 2024-09-11T06:51:25.400 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.400 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.400 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.400 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469529+0000", 2024-09-11T06:51:25.401 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.8060000000000006e-06 2024-09-11T06:51:25.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.401 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.401 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469853+0000", 2024-09-11T06:51:25.401 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00032408399999999998 2024-09-11T06:51:25.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.401 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:25.401 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469915+0000", 2024-09-11T06:51:25.401 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2379999999999996e-05 2024-09-11T06:51:25.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.401 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:25.402 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.470021+0000", 2024-09-11T06:51:25.402 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00010504400000000001 2024-09-11T06:51:25.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.402 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:25.402 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.471499+0000", 2024-09-11T06:51:25.402 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014781989999999999 2024-09-11T06:51:25.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.402 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:25.402 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.471932+0000", 2024-09-11T06:51:25.402 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00043362499999999998 2024-09-11T06:51:25.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.402 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:25.402 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.472003+0000", 2024-09-11T06:51:25.403 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1060000000000001e-05 2024-09-11T06:51:25.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.403 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.403 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:25.403 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.472149+0000", 2024-09-11T06:51:25.403 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000145465 2024-09-11T06:51:25.403 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.403 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:25.403 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.403 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.403 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5826.0:125 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.client_list in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:25.403 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:12.469719+0000", 2024-09-11T06:51:25.404 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 12.898255124, 2024-09-11T06:51:25.404 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0026625149999999998, 2024-09-11T06:51:25.404 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:25.404 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:25.404 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:25.404 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:25.404 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5826", 2024-09-11T06:51:25.404 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2236190565", 2024-09-11T06:51:25.404 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 125 2024-09-11T06:51:25.404 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.404 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:25.404 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.404 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:25.405 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469719+0000", 2024-09-11T06:51:25.405 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.405 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.405 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:25.405 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469719+0000", 2024-09-11T06:51:25.405 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.405 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.405 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:25.405 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469720+0000", 2024-09-11T06:51:25.405 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.097e-06 2024-09-11T06:51:25.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.406 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.406 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:25.406 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469726+0000", 2024-09-11T06:51:25.406 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2870000000000001e-06 2024-09-11T06:51:25.406 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.406 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.406 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:25.406 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469739+0000", 2024-09-11T06:51:25.406 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2789000000000001e-05 2024-09-11T06:51:25.406 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.406 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.406 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.406 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469749+0000", 2024-09-11T06:51:25.406 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.6879999999999994e-06 2024-09-11T06:51:25.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.407 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.407 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.407 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.470359+0000", 2024-09-11T06:51:25.407 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00060988299999999997 2024-09-11T06:51:25.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.407 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.407 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:25.407 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.470428+0000", 2024-09-11T06:51:25.407 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9696999999999994e-05 2024-09-11T06:51:25.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.407 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.407 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.407 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.472195+0000", 2024-09-11T06:51:25.408 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001767034 2024-09-11T06:51:25.408 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.408 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.408 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:25.408 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.472244+0000", 2024-09-11T06:51:25.408 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9038e-05 2024-09-11T06:51:25.408 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.408 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.408 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:25.408 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.472381+0000", 2024-09-11T06:51:25.408 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000137002 2024-09-11T06:51:25.408 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.408 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:25.408 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.409 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.409 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.409 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5374.0:22 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.client_list in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:25.409 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:12.469984+0000", 2024-09-11T06:51:25.409 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 12.897990349000001, 2024-09-11T06:51:25.409 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027625599999999998, 2024-09-11T06:51:25.409 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:25.409 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:25.409 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:25.409 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:25.409 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5374", 2024-09-11T06:51:25.409 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1937077541", 2024-09-11T06:51:25.409 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 22 2024-09-11T06:51:25.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.410 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:25.410 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.410 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:25.410 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469984+0000", 2024-09-11T06:51:25.410 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.410 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.410 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:25.410 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469984+0000", 2024-09-11T06:51:25.410 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.410 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.410 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:25.411 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469985+0000", 2024-09-11T06:51:25.411 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.569999999999999e-07 2024-09-11T06:51:25.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.411 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.411 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:25.411 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.469990+0000", 2024-09-11T06:51:25.411 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5160000000000002e-06 2024-09-11T06:51:25.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.411 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.411 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:25.411 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.470003+0000", 2024-09-11T06:51:25.411 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2933e-05 2024-09-11T06:51:25.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.411 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.412 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.412 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.470012+0000", 2024-09-11T06:51:25.412 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2e-06 2024-09-11T06:51:25.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.412 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.412 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.412 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.470453+0000", 2024-09-11T06:51:25.412 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000440963 2024-09-11T06:51:25.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.412 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.412 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:25.412 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.470509+0000", 2024-09-11T06:51:25.413 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5937999999999999e-05 2024-09-11T06:51:25.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.413 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.413 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.472419+0000", 2024-09-11T06:51:25.413 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0019100289999999999 2024-09-11T06:51:25.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.413 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:25.413 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.472523+0000", 2024-09-11T06:51:25.413 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103665 2024-09-11T06:51:25.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.413 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:25.414 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.472746+0000", 2024-09-11T06:51:25.414 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022335899999999999 2024-09-11T06:51:25.414 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.414 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:25.414 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.414 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.414 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5386.0:17 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [watch unwatch cookie 94226507267584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:25.414 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:12.563293+0000", 2024-09-11T06:51:25.414 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 12.804680864, 2024-09-11T06:51:25.414 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0028623239999999999, 2024-09-11T06:51:25.414 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:25.414 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:25.414 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:25.415 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:25.415 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5386", 2024-09-11T06:51:25.415 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3377056437", 2024-09-11T06:51:25.415 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 17 2024-09-11T06:51:25.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.415 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:25.415 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.415 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:25.415 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563293+0000", 2024-09-11T06:51:25.415 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.415 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.415 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:25.416 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563293+0000", 2024-09-11T06:51:25.416 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.416 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.416 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:25.416 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563294+0000", 2024-09-11T06:51:25.416 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5300000000000004e-07 2024-09-11T06:51:25.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.416 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.416 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:25.416 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563296+0000", 2024-09-11T06:51:25.416 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6079999999999998e-06 2024-09-11T06:51:25.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.416 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.417 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:25.417 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563302+0000", 2024-09-11T06:51:25.417 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2549999999999997e-06 2024-09-11T06:51:25.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.417 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.417 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.417 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563305+0000", 2024-09-11T06:51:25.417 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2949999999999998e-06 2024-09-11T06:51:25.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.417 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.417 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.417 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563792+0000", 2024-09-11T06:51:25.417 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00048731399999999998 2024-09-11T06:51:25.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.418 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.418 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:25.418 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563899+0000", 2024-09-11T06:51:25.418 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000106977 2024-09-11T06:51:25.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.418 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.418 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:25.418 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.564145+0000", 2024-09-11T06:51:25.418 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00024587600000000002 2024-09-11T06:51:25.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.418 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.418 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:25.418 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.564760+0000", 2024-09-11T06:51:25.418 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00061473099999999998 2024-09-11T06:51:25.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.419 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.419 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:25.419 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566046+0000", 2024-09-11T06:51:25.419 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012856810000000001 2024-09-11T06:51:25.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.419 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.419 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:25.419 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566085+0000", 2024-09-11T06:51:25.419 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.9499000000000002e-05 2024-09-11T06:51:25.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.419 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.419 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:25.419 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566156+0000", 2024-09-11T06:51:25.419 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0434999999999999e-05 2024-09-11T06:51:25.420 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.420 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:25.420 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.420 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.420 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5826.0:131 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.client_list in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:25.420 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:12.563833+0000", 2024-09-11T06:51:25.420 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 12.804141308, 2024-09-11T06:51:25.420 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0028764329999999999, 2024-09-11T06:51:25.420 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:25.420 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:25.420 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:25.420 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:25.421 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5826", 2024-09-11T06:51:25.421 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2236190565", 2024-09-11T06:51:25.421 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 131 2024-09-11T06:51:25.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.421 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:25.421 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.421 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:25.421 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563833+0000", 2024-09-11T06:51:25.421 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.421 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.421 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:25.421 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563833+0000", 2024-09-11T06:51:25.421 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.422 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.422 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:25.422 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563835+0000", 2024-09-11T06:51:25.422 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8109999999999999e-06 2024-09-11T06:51:25.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.422 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.422 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:25.422 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563840+0000", 2024-09-11T06:51:25.422 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7599999999999999e-06 2024-09-11T06:51:25.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.422 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.422 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:25.423 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563854+0000", 2024-09-11T06:51:25.423 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3332999999999999e-05 2024-09-11T06:51:25.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.423 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.423 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.423 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563874+0000", 2024-09-11T06:51:25.423 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0630999999999999e-05 2024-09-11T06:51:25.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.423 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.423 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.423 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.564553+0000", 2024-09-11T06:51:25.423 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00067821199999999998 2024-09-11T06:51:25.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.423 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.423 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:25.424 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.564610+0000", 2024-09-11T06:51:25.424 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7494999999999998e-05 2024-09-11T06:51:25.424 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.424 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.424 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.424 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566240+0000", 2024-09-11T06:51:25.424 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0016295940000000001 2024-09-11T06:51:25.424 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.424 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.424 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:25.424 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566365+0000", 2024-09-11T06:51:25.424 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000124876 2024-09-11T06:51:25.424 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.425 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.425 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:25.425 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566709+0000", 2024-09-11T06:51:25.425 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034472099999999998 2024-09-11T06:51:25.425 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.425 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:25.425 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.425 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.425 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5447.0:10 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T06:51:25.425 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:13.624024+0000", 2024-09-11T06:51:25.425 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 11.743950415, 2024-09-11T06:51:25.425 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0038665190000000001, 2024-09-11T06:51:25.425 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:25.426 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:25.426 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:25.426 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:25.426 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5447", 2024-09-11T06:51:25.426 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3862443986", 2024-09-11T06:51:25.426 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 10 2024-09-11T06:51:25.426 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.426 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:25.426 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.426 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:25.426 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624024+0000", 2024-09-11T06:51:25.426 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.426 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.426 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.426 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:25.427 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624024+0000", 2024-09-11T06:51:25.427 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.427 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.427 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:25.427 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624026+0000", 2024-09-11T06:51:25.427 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8090000000000001e-06 2024-09-11T06:51:25.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.427 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.427 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:25.427 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624037+0000", 2024-09-11T06:51:25.427 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.061e-05 2024-09-11T06:51:25.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.428 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.428 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:25.428 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624059+0000", 2024-09-11T06:51:25.428 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2269e-05 2024-09-11T06:51:25.428 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.428 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.428 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.428 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624074+0000", 2024-09-11T06:51:25.428 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4982e-05 2024-09-11T06:51:25.428 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.428 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.428 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.428 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624238+0000", 2024-09-11T06:51:25.428 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163191 2024-09-11T06:51:25.429 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.429 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.429 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:25.429 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624261+0000", 2024-09-11T06:51:25.429 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3779000000000001e-05 2024-09-11T06:51:25.429 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.429 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.429 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.429 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.627590+0000", 2024-09-11T06:51:25.429 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033287450000000001 2024-09-11T06:51:25.429 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.429 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.429 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:25.429 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.627723+0000", 2024-09-11T06:51:25.430 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013311599999999999 2024-09-11T06:51:25.430 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.430 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.430 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:25.430 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.627890+0000", 2024-09-11T06:51:25.430 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016701800000000001 2024-09-11T06:51:25.430 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.430 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:25.430 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.430 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.430 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.430 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5453.0:16 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [watch unwatch cookie 93828371656000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-11T06:51:25.430 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:13.655839+0000", 2024-09-11T06:51:25.430 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 11.712134763, 2024-09-11T06:51:25.431 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002630959, 2024-09-11T06:51:25.431 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:25.431 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:25.431 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:25.431 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:25.431 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5453", 2024-09-11T06:51:25.431 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1091178110", 2024-09-11T06:51:25.431 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 16 2024-09-11T06:51:25.431 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.431 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:25.431 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.431 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:25.431 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.655839+0000", 2024-09-11T06:51:25.431 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.432 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.432 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:25.432 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.655839+0000", 2024-09-11T06:51:25.432 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.432 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.432 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:25.432 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.655841+0000", 2024-09-11T06:51:25.432 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7799999999999999e-06 2024-09-11T06:51:25.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.432 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.432 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:25.433 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.655850+0000", 2024-09-11T06:51:25.433 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8559999999999996e-06 2024-09-11T06:51:25.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.433 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.433 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:25.433 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.655869+0000", 2024-09-11T06:51:25.433 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9446e-05 2024-09-11T06:51:25.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.433 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.433 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.433 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.655890+0000", 2024-09-11T06:51:25.433 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0292000000000002e-05 2024-09-11T06:51:25.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.433 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.434 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.434 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656678+0000", 2024-09-11T06:51:25.434 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00078840500000000005 2024-09-11T06:51:25.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.434 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.434 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:25.434 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656785+0000", 2024-09-11T06:51:25.434 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00010641699999999999 2024-09-11T06:51:25.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.434 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.434 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:25.434 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656993+0000", 2024-09-11T06:51:25.434 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000208156 2024-09-11T06:51:25.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.435 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.435 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:25.435 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.658118+0000", 2024-09-11T06:51:25.435 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011256829999999999 2024-09-11T06:51:25.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.435 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.435 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:25.435 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.658242+0000", 2024-09-11T06:51:25.435 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012334999999999999 2024-09-11T06:51:25.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.435 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.435 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:25.435 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.658322+0000", 2024-09-11T06:51:25.436 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.0092000000000005e-05 2024-09-11T06:51:25.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.436 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.436 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:25.436 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.658470+0000", 2024-09-11T06:51:25.436 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014848199999999999 2024-09-11T06:51:25.436 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.436 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:25.436 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.436 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.436 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5453.0:17 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.client_list in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T06:51:25.436 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:13.656142+0000", 2024-09-11T06:51:25.436 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 11.711831856, 2024-09-11T06:51:25.437 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027779549999999999, 2024-09-11T06:51:25.437 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:25.437 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:25.437 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:25.437 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:25.437 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5453", 2024-09-11T06:51:25.437 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1091178110", 2024-09-11T06:51:25.437 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 17 2024-09-11T06:51:25.437 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.437 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:25.437 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.437 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:25.437 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656142+0000", 2024-09-11T06:51:25.437 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.437 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.438 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.438 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:25.438 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656142+0000", 2024-09-11T06:51:25.438 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.438 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.438 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:25.438 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656144+0000", 2024-09-11T06:51:25.438 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.68e-06 2024-09-11T06:51:25.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.438 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.438 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:25.438 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656154+0000", 2024-09-11T06:51:25.438 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0159999999999999e-05 2024-09-11T06:51:25.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.439 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.439 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:25.439 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656173+0000", 2024-09-11T06:51:25.439 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9324999999999999e-05 2024-09-11T06:51:25.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.439 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.439 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.439 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656193+0000", 2024-09-11T06:51:25.439 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9692999999999999e-05 2024-09-11T06:51:25.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.439 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.439 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.440 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.657563+0000", 2024-09-11T06:51:25.440 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0013697099999999999 2024-09-11T06:51:25.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.440 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.440 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:25.440 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.657779+0000", 2024-09-11T06:51:25.440 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021642100000000001 2024-09-11T06:51:25.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.440 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.440 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.440 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.658528+0000", 2024-09-11T06:51:25.440 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00074905000000000002 2024-09-11T06:51:25.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.440 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.441 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:25.441 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.658639+0000", 2024-09-11T06:51:25.441 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000110788 2024-09-11T06:51:25.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.441 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.441 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:25.441 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.658920+0000", 2024-09-11T06:51:25.441 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00028112800000000001 2024-09-11T06:51:25.441 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.441 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:25.441 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.441 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.442 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5826.0:137 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.client_list in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T06:51:25.442 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:13.656886+0000", 2024-09-11T06:51:25.442 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 11.711088045, 2024-09-11T06:51:25.442 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002654478, 2024-09-11T06:51:25.442 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:25.442 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:25.442 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:25.442 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:25.442 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5826", 2024-09-11T06:51:25.442 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2236190565", 2024-09-11T06:51:25.442 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 137 2024-09-11T06:51:25.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.442 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:25.442 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.443 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:25.443 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656886+0000", 2024-09-11T06:51:25.443 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.443 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:25.443 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656886+0000", 2024-09-11T06:51:25.443 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.443 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:25.443 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656888+0000", 2024-09-11T06:51:25.443 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.66e-06 2024-09-11T06:51:25.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.444 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.444 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:25.444 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656898+0000", 2024-09-11T06:51:25.444 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0042e-05 2024-09-11T06:51:25.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.444 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.444 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:25.444 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656917+0000", 2024-09-11T06:51:25.444 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9522e-05 2024-09-11T06:51:25.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.444 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.444 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.444 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.656928+0000", 2024-09-11T06:51:25.445 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1097000000000001e-05 2024-09-11T06:51:25.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.445 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.445 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.657882+0000", 2024-09-11T06:51:25.445 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00095349499999999995 2024-09-11T06:51:25.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.445 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:25.445 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.658007+0000", 2024-09-11T06:51:25.445 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000125337 2024-09-11T06:51:25.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.445 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.446 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.659031+0000", 2024-09-11T06:51:25.446 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001024248 2024-09-11T06:51:25.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.446 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.446 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:25.446 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.659149+0000", 2024-09-11T06:51:25.446 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000117971 2024-09-11T06:51:25.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.446 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.446 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:25.446 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.659541+0000", 2024-09-11T06:51:25.446 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00039110600000000001 2024-09-11T06:51:25.446 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.446 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:25.446 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.447 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.6018.0:17 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [watch unwatch cookie 93970929245696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-11T06:51:25.447 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:16.021477+0000", 2024-09-11T06:51:25.447 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 9.3464974279999993, 2024-09-11T06:51:25.447 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0026652009999999999, 2024-09-11T06:51:25.447 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:25.447 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:25.447 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:25.447 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:25.447 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.6018", 2024-09-11T06:51:25.447 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1879900433", 2024-09-11T06:51:25.447 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 17 2024-09-11T06:51:25.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.448 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:25.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.448 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:25.448 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:16.021477+0000", 2024-09-11T06:51:25.448 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.448 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:25.448 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:16.021477+0000", 2024-09-11T06:51:25.448 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.448 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:25.449 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:16.021479+0000", 2024-09-11T06:51:25.449 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9800000000000001e-06 2024-09-11T06:51:25.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.449 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:25.449 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:16.021488+0000", 2024-09-11T06:51:25.449 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.6330000000000008e-06 2024-09-11T06:51:25.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.449 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:25.449 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:16.021519+0000", 2024-09-11T06:51:25.449 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.1208999999999999e-05 2024-09-11T06:51:25.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.449 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.450 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:16.021538+0000", 2024-09-11T06:51:25.450 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8919e-05 2024-09-11T06:51:25.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.450 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.450 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.450 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:16.022335+0000", 2024-09-11T06:51:25.450 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00079702600000000005 2024-09-11T06:51:25.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.450 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.450 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:25.450 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:16.022457+0000", 2024-09-11T06:51:25.450 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000121494 2024-09-11T06:51:25.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.450 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.451 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:25.451 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:16.022666+0000", 2024-09-11T06:51:25.451 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020869399999999999 2024-09-11T06:51:25.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.451 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.451 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:25.451 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:16.023772+0000", 2024-09-11T06:51:25.451 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011067410000000001 2024-09-11T06:51:25.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.451 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.451 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:25.451 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:16.023872+0000", 2024-09-11T06:51:25.451 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.9702000000000002e-05 2024-09-11T06:51:25.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.452 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:25.452 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:16.023957+0000", 2024-09-11T06:51:25.452 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.5093000000000007e-05 2024-09-11T06:51:25.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.452 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:25.452 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:16.024142+0000", 2024-09-11T06:51:25.452 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018471000000000001 2024-09-11T06:51:25.452 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.452 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:25.452 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.453 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.6081.0:32 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.set_minimum_set in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-11T06:51:25.453 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.541248+0000", 2024-09-11T06:51:25.453 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.826726088, 2024-09-11T06:51:25.453 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0028300869999999998, 2024-09-11T06:51:25.453 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:25.453 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:25.453 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:25.453 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:25.453 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.6081", 2024-09-11T06:51:25.453 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3800899964", 2024-09-11T06:51:25.453 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 32 2024-09-11T06:51:25.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.454 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:25.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.454 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:25.454 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541248+0000", 2024-09-11T06:51:25.454 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.454 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:25.454 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541248+0000", 2024-09-11T06:51:25.454 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.454 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:25.454 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541251+0000", 2024-09-11T06:51:25.455 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5299999999999999e-06 2024-09-11T06:51:25.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.455 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:25.455 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541266+0000", 2024-09-11T06:51:25.455 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5139e-05 2024-09-11T06:51:25.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.455 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:25.455 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541298+0000", 2024-09-11T06:51:25.455 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2152e-05 2024-09-11T06:51:25.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.455 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.456 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541313+0000", 2024-09-11T06:51:25.456 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5532999999999998e-05 2024-09-11T06:51:25.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.456 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.456 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541554+0000", 2024-09-11T06:51:25.456 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000240306 2024-09-11T06:51:25.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.456 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:25.456 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541957+0000", 2024-09-11T06:51:25.456 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00040357799999999998 2024-09-11T06:51:25.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.457 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:25.457 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.544078+0000", 2024-09-11T06:51:25.457 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021208490000000002 2024-09-11T06:51:25.457 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.457 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:25.457 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.457 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.457 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.6081.0:33 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-11T06:51:25.457 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.541352+0000", 2024-09-11T06:51:25.457 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.826621963, 2024-09-11T06:51:25.457 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034084850000000002, 2024-09-11T06:51:25.457 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:25.458 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:25.458 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:25.458 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:25.458 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.6081", 2024-09-11T06:51:25.458 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3800899964", 2024-09-11T06:51:25.458 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 33 2024-09-11T06:51:25.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.458 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:25.458 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.458 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:25.458 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541352+0000", 2024-09-11T06:51:25.458 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.458 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.458 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:25.459 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541352+0000", 2024-09-11T06:51:25.459 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.459 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:25.459 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541353+0000", 2024-09-11T06:51:25.459 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7000000000000003e-07 2024-09-11T06:51:25.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.459 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:25.459 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541358+0000", 2024-09-11T06:51:25.459 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7400000000000004e-06 2024-09-11T06:51:25.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.460 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:25.460 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541374+0000", 2024-09-11T06:51:25.460 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5670000000000001e-05 2024-09-11T06:51:25.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.460 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.460 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.460 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541386+0000", 2024-09-11T06:51:25.460 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2359e-05 2024-09-11T06:51:25.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.460 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.460 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.460 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542684+0000", 2024-09-11T06:51:25.460 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012984590000000001 2024-09-11T06:51:25.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.460 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.461 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:25.461 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542820+0000", 2024-09-11T06:51:25.461 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013536700000000001 2024-09-11T06:51:25.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.461 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.461 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:25.461 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542972+0000", 2024-09-11T06:51:25.461 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000151843 2024-09-11T06:51:25.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.461 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.461 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:25.461 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.544110+0000", 2024-09-11T06:51:25.461 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011381060000000001 2024-09-11T06:51:25.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.462 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.462 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:25.462 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.544510+0000", 2024-09-11T06:51:25.462 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00040019799999999999 2024-09-11T06:51:25.462 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.462 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.462 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:25.462 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.544643+0000", 2024-09-11T06:51:25.462 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013313499999999999 2024-09-11T06:51:25.462 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.462 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.462 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:25.462 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.544761+0000", 2024-09-11T06:51:25.462 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000117638 2024-09-11T06:51:25.462 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.463 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:25.463 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.463 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.463 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.463 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5542.0:10 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-11T06:51:25.463 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:22.002114+0000", 2024-09-11T06:51:25.463 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 3.365860514, 2024-09-11T06:51:25.463 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00564235, 2024-09-11T06:51:25.463 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:25.463 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:25.463 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:25.463 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:25.463 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5542", 2024-09-11T06:51:25.463 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1546942093", 2024-09-11T06:51:25.464 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 10 2024-09-11T06:51:25.464 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.464 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:25.464 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.464 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:25.464 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002114+0000", 2024-09-11T06:51:25.464 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.464 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.464 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.464 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:25.464 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002114+0000", 2024-09-11T06:51:25.464 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.464 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.465 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.465 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:25.465 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002115+0000", 2024-09-11T06:51:25.465 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.872e-06 2024-09-11T06:51:25.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.465 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.465 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:25.465 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002123+0000", 2024-09-11T06:51:25.465 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3309999999999998e-06 2024-09-11T06:51:25.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.465 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.465 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:25.465 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002136+0000", 2024-09-11T06:51:25.465 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2816000000000001e-05 2024-09-11T06:51:25.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.466 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.466 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.466 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002145+0000", 2024-09-11T06:51:25.466 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.8570000000000007e-06 2024-09-11T06:51:25.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.466 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.466 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.466 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002214+0000", 2024-09-11T06:51:25.466 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.8832999999999996e-05 2024-09-11T06:51:25.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.466 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.466 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:25.466 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002220+0000", 2024-09-11T06:51:25.467 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4249999999999997e-06 2024-09-11T06:51:25.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.467 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.467 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.467 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.007445+0000", 2024-09-11T06:51:25.467 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052251659999999998 2024-09-11T06:51:25.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.467 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.467 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:25.467 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.007586+0000", 2024-09-11T06:51:25.467 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014103100000000001 2024-09-11T06:51:25.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.467 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.467 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:25.467 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.007756+0000", 2024-09-11T06:51:25.468 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017001900000000001 2024-09-11T06:51:25.468 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.468 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:25.468 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.468 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.468 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.468 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5542.0:21 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.create in=13b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-11T06:51:25.468 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:22.028810+0000", 2024-09-11T06:51:25.468 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 3.3391644700000001, 2024-09-11T06:51:25.468 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034172709999999999, 2024-09-11T06:51:25.468 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:25.468 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:25.468 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:25.468 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:25.468 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5542", 2024-09-11T06:51:25.469 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1546942093", 2024-09-11T06:51:25.469 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 21 2024-09-11T06:51:25.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.469 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:25.469 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.469 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:25.469 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.028810+0000", 2024-09-11T06:51:25.469 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.469 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.469 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:25.469 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.028810+0000", 2024-09-11T06:51:25.469 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.470 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.470 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:25.470 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.028812+0000", 2024-09-11T06:51:25.470 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8100000000000002e-06 2024-09-11T06:51:25.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.470 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.470 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:25.470 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.028826+0000", 2024-09-11T06:51:25.470 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3882e-05 2024-09-11T06:51:25.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.470 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.470 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:25.470 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.028859+0000", 2024-09-11T06:51:25.471 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2225999999999999e-05 2024-09-11T06:51:25.471 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.471 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.471 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.471 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.028880+0000", 2024-09-11T06:51:25.471 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1755000000000001e-05 2024-09-11T06:51:25.471 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.471 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.471 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.471 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.029048+0000", 2024-09-11T06:51:25.471 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016811699999999999 2024-09-11T06:51:25.471 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.471 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.471 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:25.471 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.029198+0000", 2024-09-11T06:51:25.472 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000149753 2024-09-11T06:51:25.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.472 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.472 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:25.472 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.029612+0000", 2024-09-11T06:51:25.472 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00041426899999999999 2024-09-11T06:51:25.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.472 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.472 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:25.472 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.030882+0000", 2024-09-11T06:51:25.472 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012693030000000001 2024-09-11T06:51:25.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.472 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.472 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:25.472 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.032087+0000", 2024-09-11T06:51:25.473 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012054279999999999 2024-09-11T06:51:25.473 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.473 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.473 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:25.473 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.032167+0000", 2024-09-11T06:51:25.473 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.0177000000000002e-05 2024-09-11T06:51:25.473 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.473 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.473 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:25.473 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.032227+0000", 2024-09-11T06:51:25.473 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.9551e-05 2024-09-11T06:51:25.473 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.473 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:25.473 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.474 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.474 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5542.0:23 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.tag_create in=17b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-11T06:51:25.474 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:22.033580+0000", 2024-09-11T06:51:25.474 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 3.33439402, 2024-09-11T06:51:25.474 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002804384, 2024-09-11T06:51:25.474 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:25.474 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:25.474 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:25.474 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:25.474 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5542", 2024-09-11T06:51:25.474 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1546942093", 2024-09-11T06:51:25.474 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 23 2024-09-11T06:51:25.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.474 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:25.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.475 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:25.475 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033580+0000", 2024-09-11T06:51:25.475 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.475 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:25.475 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033580+0000", 2024-09-11T06:51:25.475 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.475 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:25.475 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033581+0000", 2024-09-11T06:51:25.476 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.217e-06 2024-09-11T06:51:25.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.476 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:25.476 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033591+0000", 2024-09-11T06:51:25.476 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2660000000000007e-06 2024-09-11T06:51:25.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.476 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:25.476 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033611+0000", 2024-09-11T06:51:25.476 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0418999999999999e-05 2024-09-11T06:51:25.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.476 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.476 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033628+0000", 2024-09-11T06:51:25.477 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7316000000000002e-05 2024-09-11T06:51:25.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.477 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.477 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033803+0000", 2024-09-11T06:51:25.477 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017436200000000001 2024-09-11T06:51:25.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.477 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:25.477 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033944+0000", 2024-09-11T06:51:25.477 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000141688 2024-09-11T06:51:25.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.477 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:25.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.034475+0000", 2024-09-11T06:51:25.478 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00053032399999999999 2024-09-11T06:51:25.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.478 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:25.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.035684+0000", 2024-09-11T06:51:25.478 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001209452 2024-09-11T06:51:25.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.478 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:25.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.036254+0000", 2024-09-11T06:51:25.478 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00056982599999999995 2024-09-11T06:51:25.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.478 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:25.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.036328+0000", 2024-09-11T06:51:25.479 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4499999999999995e-05 2024-09-11T06:51:25.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.479 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.479 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:25.479 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.036384+0000", 2024-09-11T06:51:25.479 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.6014000000000003e-05 2024-09-11T06:51:25.479 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.479 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:25.479 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.479 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.479 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5542.0:26 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.client_register in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-11T06:51:25.479 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:22.038666+0000", 2024-09-11T06:51:25.479 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 3.3293080989999999, 2024-09-11T06:51:25.479 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027244439999999999, 2024-09-11T06:51:25.479 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:25.480 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:25.480 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:25.480 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:25.480 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5542", 2024-09-11T06:51:25.480 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1546942093", 2024-09-11T06:51:25.480 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 26 2024-09-11T06:51:25.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.480 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:25.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.480 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:25.480 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.038666+0000", 2024-09-11T06:51:25.480 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.481 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:25.481 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.038666+0000", 2024-09-11T06:51:25.481 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:25.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.481 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:25.481 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.038668+0000", 2024-09-11T06:51:25.481 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.516e-06 2024-09-11T06:51:25.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.481 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:25.481 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.038675+0000", 2024-09-11T06:51:25.481 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2030000000000003e-06 2024-09-11T06:51:25.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.482 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:25.482 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.038694+0000", 2024-09-11T06:51:25.482 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9276e-05 2024-09-11T06:51:25.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.482 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.482 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.038711+0000", 2024-09-11T06:51:25.482 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7119e-05 2024-09-11T06:51:25.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.482 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:25.482 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.038831+0000", 2024-09-11T06:51:25.482 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000119929 2024-09-11T06:51:25.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.483 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.483 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:25.483 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.038954+0000", 2024-09-11T06:51:25.483 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012327099999999999 2024-09-11T06:51:25.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.483 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.483 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:25.483 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.039374+0000", 2024-09-11T06:51:25.483 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000419372 2024-09-11T06:51:25.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.483 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.483 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:25.484 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.040347+0000", 2024-09-11T06:51:25.484 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00097351800000000004 2024-09-11T06:51:25.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.484 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.484 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:25.484 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.041254+0000", 2024-09-11T06:51:25.484 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00090697200000000003 2024-09-11T06:51:25.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.484 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.484 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:25.484 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.041334+0000", 2024-09-11T06:51:25.484 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.0302999999999993e-05 2024-09-11T06:51:25.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:25.484 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:25.485 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:25.485 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.041390+0000", 2024-09-11T06:51:25.485 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5964999999999997e-05 2024-09-11T06:51:25.485 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.485 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:25.485 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.485 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:25.485 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:25.485 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:25.486 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:51:25.487 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:25.481+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:25.588 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:25.585+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:25.618 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:25.618 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:25.619 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:51:25.619 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:25.632 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:51:25.688 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:25.682+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:25.749 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:25.749 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:25.749 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:51:25.749 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:51:25.749 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:25.762 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:51:25.796 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo blkdiscard --offset 0 --length 4096 /dev/nbd2 2024-09-11T06:51:25.796 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:25.797 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:25.798 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:25.798 INFO:tasks.workunit.client.0.smithi176.stderr:++ which blkdiscard 2024-09-11T06:51:25.800 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/sbin/blkdiscard 2024-09-11T06:51:25.801 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:25.801 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/sbin/blkdiscard --offset 0 --length 4096 /dev/nbd2 2024-09-11T06:51:25.805 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:25.802+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:25.837 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd feature disable rbd/testrbdnbd21607 journaling 2024-09-11T06:51:25.890 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:25.885+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:25.932 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:25.932 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:51:25.932 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:51:25.932 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:51:25.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.932 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:55 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'33, mlcod=13'29)", 2024-09-11T06:51:25.932 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:51:25.932 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 139.716757884, 2024-09-11T06:51:25.933 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.005086091, 2024-09-11T06:51:25.933 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:25.933 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:25.933 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:25.933 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:25.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.933 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:25.933 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:51:25.933 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:25.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.933 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:25.933 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211379+0000", 2024-09-11T06:51:25.933 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:25.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.934 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:25.934 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211380+0000", 2024-09-11T06:51:25.934 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.097e-06 2024-09-11T06:51:25.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.934 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:25.934 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211537+0000", 2024-09-11T06:51:25.934 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000157369 2024-09-11T06:51:25.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.935 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.935 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:25.935 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211559+0000", 2024-09-11T06:51:25.935 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000001e-05 2024-09-11T06:51:25.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.935 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.935 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.935 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211573+0000", 2024-09-11T06:51:25.935 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4946000000000001e-05 2024-09-11T06:51:25.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.935 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.935 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:25.935 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211676+0000", 2024-09-11T06:51:25.936 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0001026 2024-09-11T06:51:25.936 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.936 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:25.936 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.211709+0000", 2024-09-11T06:51:25.936 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3071999999999998e-05 2024-09-11T06:51:25.936 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.936 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:25.936 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216379+0000", 2024-09-11T06:51:25.936 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0046693780000000001 2024-09-11T06:51:25.936 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.216465+0000", 2024-09-11T06:51:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6469000000000002e-05 2024-09-11T06:51:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:56 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'34, mlcod=13'29)", 2024-09-11T06:51:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:51:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 139.715324025, 2024-09-11T06:51:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0060310629999999997, 2024-09-11T06:51:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:25.938 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:25.938 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:25.938 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:25.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.938 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:25.938 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:51:25.938 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:25.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.938 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:25.938 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:51:25.938 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:25.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.939 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.939 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:25.939 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212814+0000", 2024-09-11T06:51:25.939 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0869999999999999e-06 2024-09-11T06:51:25.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.939 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.939 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:25.939 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212822+0000", 2024-09-11T06:51:25.939 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0870000000000007e-06 2024-09-11T06:51:25.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.940 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:25.940 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212840+0000", 2024-09-11T06:51:25.940 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:51:25.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.940 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.940 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212854+0000", 2024-09-11T06:51:25.940 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4003e-05 2024-09-11T06:51:25.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.941 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.941 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:25.941 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213026+0000", 2024-09-11T06:51:25.941 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017207499999999999 2024-09-11T06:51:25.941 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.941 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.941 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:25.941 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213063+0000", 2024-09-11T06:51:25.941 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7187000000000002e-05 2024-09-11T06:51:25.941 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.941 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.941 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:25.941 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218762+0000", 2024-09-11T06:51:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0056993290000000004 2024-09-11T06:51:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218844+0000", 2024-09-11T06:51:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1701999999999998e-05 2024-09-11T06:51:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:60 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'38, mlcod=13'36)", 2024-09-11T06:51:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:51:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 139.70138267600001, 2024-09-11T06:51:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039254270000000004, 2024-09-11T06:51:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:25.944 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:51:25.944 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:25.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.944 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:25.944 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226754+0000", 2024-09-11T06:51:25.944 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:25.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.944 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:25.944 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226756+0000", 2024-09-11T06:51:25.945 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-11T06:51:25.945 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.945 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.945 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:25.945 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226768+0000", 2024-09-11T06:51:25.945 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1919999999999999e-05 2024-09-11T06:51:25.945 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.945 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.945 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:25.946 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226797+0000", 2024-09-11T06:51:25.946 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9573e-05 2024-09-11T06:51:25.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.946 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.946 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226818+0000", 2024-09-11T06:51:25.946 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0432999999999999e-05 2024-09-11T06:51:25.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.946 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:25.946 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226909+0000", 2024-09-11T06:51:25.946 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0920000000000004e-05 2024-09-11T06:51:25.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.226936+0000", 2024-09-11T06:51:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7472999999999999e-05 2024-09-11T06:51:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230596+0000", 2024-09-11T06:51:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0036602520000000001 2024-09-11T06:51:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.230680+0000", 2024-09-11T06:51:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3106e-05 2024-09-11T06:51:25.947 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:25.948 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:25.948 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:25.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.948 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:107 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'69, mlcod=13'64)", 2024-09-11T06:51:25.948 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:51:25.948 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 139.48493521899999, 2024-09-11T06:51:25.948 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040728830000000002, 2024-09-11T06:51:25.948 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:25.948 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:25.948 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:25.948 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:25.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.949 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:25.949 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:51:25.949 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:25.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.949 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:25.949 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443202+0000", 2024-09-11T06:51:25.949 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:25.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.950 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:25.950 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443203+0000", 2024-09-11T06:51:25.950 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1480000000000001e-06 2024-09-11T06:51:25.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.950 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:25.950 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443222+0000", 2024-09-11T06:51:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9040000000000001e-05 2024-09-11T06:51:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443246+0000", 2024-09-11T06:51:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4606000000000001e-05 2024-09-11T06:51:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443277+0000", 2024-09-11T06:51:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0207999999999999e-05 2024-09-11T06:51:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.951 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:25.952 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443340+0000", 2024-09-11T06:51:25.952 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3602000000000005e-05 2024-09-11T06:51:25.952 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.952 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.952 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:25.952 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.443363+0000", 2024-09-11T06:51:25.952 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2331e-05 2024-09-11T06:51:25.952 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.952 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.952 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:25.952 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447216+0000", 2024-09-11T06:51:25.952 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0038539690000000001 2024-09-11T06:51:25.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.953 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:25.953 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.447274+0000", 2024-09-11T06:51:25.953 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7979000000000002e-05 2024-09-11T06:51:25.953 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:25.953 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:25.953 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:25.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.954 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:114 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'76, mlcod=13'71)", 2024-09-11T06:51:25.954 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:51:25.954 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 139.47374551799999, 2024-09-11T06:51:25.954 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0042576660000000002, 2024-09-11T06:51:25.954 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:25.954 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:25.954 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:25.954 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:25.954 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.954 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:25.954 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:51:25.954 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:25.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.955 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:25.955 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454391+0000", 2024-09-11T06:51:25.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:25.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.955 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:25.955 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454392+0000", 2024-09-11T06:51:25.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5999999999999991e-07 2024-09-11T06:51:25.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.955 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:25.955 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454399+0000", 2024-09-11T06:51:25.956 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.878e-06 2024-09-11T06:51:25.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.956 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:25.956 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454415+0000", 2024-09-11T06:51:25.956 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5659999999999999e-05 2024-09-11T06:51:25.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.956 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.956 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454422+0000", 2024-09-11T06:51:25.956 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:51:25.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.957 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:25.957 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454474+0000", 2024-09-11T06:51:25.957 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1860999999999997e-05 2024-09-11T06:51:25.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.957 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:25.957 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.454497+0000", 2024-09-11T06:51:25.957 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3686e-05 2024-09-11T06:51:25.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.957 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:25.957 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458552+0000", 2024-09-11T06:51:25.957 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040544550000000002 2024-09-11T06:51:25.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.958 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:25.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.458649+0000", 2024-09-11T06:51:25.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7005999999999999e-05 2024-09-11T06:51:25.958 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:25.958 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:25.958 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:25.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.958 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:120 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'82, mlcod=13'78)", 2024-09-11T06:51:25.958 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:51:25.959 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 139.46172762099999, 2024-09-11T06:51:25.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0064612549999999999, 2024-09-11T06:51:25.959 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:25.959 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:25.959 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:25.959 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:25.959 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.959 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:25.959 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:51:25.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:25.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.959 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.959 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:25.960 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:51:25.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:25.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.960 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:25.960 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466410+0000", 2024-09-11T06:51:25.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0389999999999999e-06 2024-09-11T06:51:25.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.960 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:25.961 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466496+0000", 2024-09-11T06:51:25.961 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5519000000000006e-05 2024-09-11T06:51:25.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.961 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.961 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:25.961 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466510+0000", 2024-09-11T06:51:25.961 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3842999999999999e-05 2024-09-11T06:51:25.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.961 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.961 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.961 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466517+0000", 2024-09-11T06:51:25.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5229999999999998e-06 2024-09-11T06:51:25.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:25.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466595+0000", 2024-09-11T06:51:25.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8019e-05 2024-09-11T06:51:25.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:25.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466612+0000", 2024-09-11T06:51:25.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7013000000000001e-05 2024-09-11T06:51:25.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472787+0000", 2024-09-11T06:51:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0061750269999999996 2024-09-11T06:51:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472870+0000", 2024-09-11T06:51:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3271999999999998e-05 2024-09-11T06:51:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:121 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'83, mlcod=13'78)", 2024-09-11T06:51:25.964 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:51:25.964 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 139.46050282499999, 2024-09-11T06:51:25.964 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055613069999999997, 2024-09-11T06:51:25.964 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:25.964 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:25.964 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:25.964 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:25.964 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.964 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:25.964 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:51:25.964 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:25.964 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.964 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.964 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:25.965 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:51:25.965 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:25.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.965 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.965 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:25.965 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467635+0000", 2024-09-11T06:51:25.965 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6899999999999996e-07 2024-09-11T06:51:25.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.965 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.965 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:25.965 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467727+0000", 2024-09-11T06:51:25.965 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1800999999999997e-05 2024-09-11T06:51:25.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467740+0000", 2024-09-11T06:51:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3747999999999999e-05 2024-09-11T06:51:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467753+0000", 2024-09-11T06:51:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2807e-05 2024-09-11T06:51:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467812+0000", 2024-09-11T06:51:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8381999999999998e-05 2024-09-11T06:51:25.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.967 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:25.967 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467828+0000", 2024-09-11T06:51:25.967 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6728e-05 2024-09-11T06:51:25.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.967 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:25.967 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473112+0000", 2024-09-11T06:51:25.967 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052833619999999998 2024-09-11T06:51:25.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.967 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:25.967 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473195+0000", 2024-09-11T06:51:25.968 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3609999999999994e-05 2024-09-11T06:51:25.968 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:25.968 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:25.968 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:25.968 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.968 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.968 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:131 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'89, mlcod=13'85)", 2024-09-11T06:51:25.968 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:51:25.968 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 139.10909710999999, 2024-09-11T06:51:25.968 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052492260000000001, 2024-09-11T06:51:25.968 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:25.969 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:25.969 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:25.969 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:25.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.969 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:25.969 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:51:25.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:25.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.969 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:25.969 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:51:25.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:25.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.970 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:25.970 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819041+0000", 2024-09-11T06:51:25.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4360000000000001e-06 2024-09-11T06:51:25.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.970 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:25.970 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819100+0000", 2024-09-11T06:51:25.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9225999999999998e-05 2024-09-11T06:51:25.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.970 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:25.970 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819128+0000", 2024-09-11T06:51:25.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.779e-05 2024-09-11T06:51:25.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819142+0000", 2024-09-11T06:51:25.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3573e-05 2024-09-11T06:51:25.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:25.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819263+0000", 2024-09-11T06:51:25.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000121086 2024-09-11T06:51:25.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:25.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819292+0000", 2024-09-11T06:51:25.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9363e-05 2024-09-11T06:51:25.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:25.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824185+0000", 2024-09-11T06:51:25.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0048924620000000002 2024-09-11T06:51:25.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:25.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824289+0000", 2024-09-11T06:51:25.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010429000000000001 2024-09-11T06:51:25.973 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:25.973 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:25.973 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:25.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.973 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:132 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'90, mlcod=13'85)", 2024-09-11T06:51:25.973 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:51:25.974 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 139.10832242000001, 2024-09-11T06:51:25.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.004532342, 2024-09-11T06:51:25.974 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:25.974 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:25.974 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:25.974 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:25.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:25.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:51:25.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:25.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:25.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819814+0000", 2024-09-11T06:51:25.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:25.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:25.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819816+0000", 2024-09-11T06:51:25.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.61e-06 2024-09-11T06:51:25.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:25.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819827+0000", 2024-09-11T06:51:25.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1e-05 2024-09-11T06:51:25.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.976 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:25.976 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819851+0000", 2024-09-11T06:51:25.976 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4267e-05 2024-09-11T06:51:25.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.976 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.976 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.976 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819864+0000", 2024-09-11T06:51:25.976 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2767e-05 2024-09-11T06:51:25.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.976 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.976 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:25.976 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820071+0000", 2024-09-11T06:51:25.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000207061 2024-09-11T06:51:25.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:25.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820103+0000", 2024-09-11T06:51:25.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2326999999999997e-05 2024-09-11T06:51:25.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:25.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824302+0000", 2024-09-11T06:51:25.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0041986000000000002 2024-09-11T06:51:25.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.978 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:25.978 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824347+0000", 2024-09-11T06:51:25.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4709999999999997e-05 2024-09-11T06:51:25.978 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:25.978 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:25.978 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:25.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.978 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:133 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'91, mlcod=13'86)", 2024-09-11T06:51:25.978 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:51:25.978 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 139.10769076099999, 2024-09-11T06:51:25.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039421459999999997, 2024-09-11T06:51:25.978 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:25.978 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:25.979 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:25.979 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:25.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:25.979 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:51:25.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:25.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:25.979 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820446+0000", 2024-09-11T06:51:25.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:25.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.980 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:25.980 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820448+0000", 2024-09-11T06:51:25.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.84e-06 2024-09-11T06:51:25.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.980 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.980 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:25.980 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820463+0000", 2024-09-11T06:51:25.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5356e-05 2024-09-11T06:51:25.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.981 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:25.981 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820499+0000", 2024-09-11T06:51:25.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.625e-05 2024-09-11T06:51:25.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.981 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.981 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820546+0000", 2024-09-11T06:51:25.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7017000000000001e-05 2024-09-11T06:51:25.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.982 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:25.982 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820882+0000", 2024-09-11T06:51:25.982 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033512 2024-09-11T06:51:25.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.982 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:25.982 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.820941+0000", 2024-09-11T06:51:25.982 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8922999999999998e-05 2024-09-11T06:51:25.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.982 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:25.983 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824351+0000", 2024-09-11T06:51:25.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034101940000000001 2024-09-11T06:51:25.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:25.983 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824388+0000", 2024-09-11T06:51:25.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7446e-05 2024-09-11T06:51:25.983 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:25.983 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:25.983 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:25.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.983 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:139 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'97, mlcod=13'86)", 2024-09-11T06:51:25.984 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:51:25.984 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 139.102280783, 2024-09-11T06:51:25.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003732221, 2024-09-11T06:51:25.984 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:25.984 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:25.984 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:25.984 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:25.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.984 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:25.984 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:51:25.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:25.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:25.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825856+0000", 2024-09-11T06:51:25.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:25.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:25.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825858+0000", 2024-09-11T06:51:25.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5829999999999999e-06 2024-09-11T06:51:25.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:25.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825874+0000", 2024-09-11T06:51:25.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5967000000000001e-05 2024-09-11T06:51:25.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:25.986 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825909+0000", 2024-09-11T06:51:25.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5692999999999999e-05 2024-09-11T06:51:25.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.986 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.825923+0000", 2024-09-11T06:51:25.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4080000000000001e-05 2024-09-11T06:51:25.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:25.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826010+0000", 2024-09-11T06:51:25.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6248999999999994e-05 2024-09-11T06:51:25.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.987 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:25.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.826038+0000", 2024-09-11T06:51:25.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8010000000000001e-05 2024-09-11T06:51:25.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.987 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:25.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829499+0000", 2024-09-11T06:51:25.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003461783 2024-09-11T06:51:25.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.988 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:25.988 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829588+0000", 2024-09-11T06:51:25.988 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8856000000000004e-05 2024-09-11T06:51:25.988 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:25.988 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:25.988 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:25.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.988 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.988 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:140 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'98, mlcod=13'93)", 2024-09-11T06:51:25.988 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:51:25.988 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 139.10043005399999, 2024-09-11T06:51:25.988 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0057995929999999996, 2024-09-11T06:51:25.989 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:25.989 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:25.989 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:25.989 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:25.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:25.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:51:25.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:25.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:25.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:51:25.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:25.990 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.990 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:25.990 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827708+0000", 2024-09-11T06:51:25.990 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.547e-06 2024-09-11T06:51:25.990 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.990 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:25.990 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827765+0000", 2024-09-11T06:51:25.990 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6266e-05 2024-09-11T06:51:25.990 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.990 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:25.990 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827791+0000", 2024-09-11T06:51:25.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6098999999999999e-05 2024-09-11T06:51:25.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.991 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.991 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827806+0000", 2024-09-11T06:51:25.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5537e-05 2024-09-11T06:51:25.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.991 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:25.991 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827928+0000", 2024-09-11T06:51:25.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000122012 2024-09-11T06:51:25.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.991 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:25.992 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827961+0000", 2024-09-11T06:51:25.992 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2663999999999998e-05 2024-09-11T06:51:25.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.992 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.992 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:25.992 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833409+0000", 2024-09-11T06:51:25.992 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054479350000000001 2024-09-11T06:51:25.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.993 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:25.993 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833506+0000", 2024-09-11T06:51:25.993 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7533000000000003e-05 2024-09-11T06:51:25.993 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:25.993 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:25.993 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:25.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.993 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:141 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'99, mlcod=13'93)", 2024-09-11T06:51:25.993 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:51:25.993 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 139.09953667100001, 2024-09-11T06:51:25.993 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055394709999999998, 2024-09-11T06:51:25.994 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:25.994 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:25.994 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:25.994 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:25.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.994 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:25.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:51:25.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:25.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.994 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:25.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:51:25.995 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:25.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.995 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:25.995 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828602+0000", 2024-09-11T06:51:25.995 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.46e-06 2024-09-11T06:51:25.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.995 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:25.995 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828618+0000", 2024-09-11T06:51:25.995 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6592999999999999e-05 2024-09-11T06:51:25.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:25.996 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828646+0000", 2024-09-11T06:51:25.996 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7552999999999999e-05 2024-09-11T06:51:25.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:25.996 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828663+0000", 2024-09-11T06:51:25.996 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.719e-05 2024-09-11T06:51:25.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.997 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:25.997 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829023+0000", 2024-09-11T06:51:25.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036054799999999998 2024-09-11T06:51:25.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.997 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.997 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:25.997 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829062+0000", 2024-09-11T06:51:25.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8865999999999997e-05 2024-09-11T06:51:25.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.997 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.997 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:25.997 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834078+0000", 2024-09-11T06:51:25.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050157480000000004 2024-09-11T06:51:25.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:25.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834140+0000", 2024-09-11T06:51:25.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1513000000000005e-05 2024-09-11T06:51:25.998 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:25.998 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:25.998 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:25.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:25.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.999 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:145 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'103, mlcod=13'98)", 2024-09-11T06:51:25.999 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:51:25.999 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 139.08958417400001, 2024-09-11T06:51:25.999 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054455409999999999, 2024-09-11T06:51:25.999 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:25.999 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:25.999 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:25.999 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:25.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:25.999 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:26.000 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:51:26.000 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.000 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:26.000 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:51:26.000 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.000 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:26.000 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838554+0000", 2024-09-11T06:51:26.000 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3430000000000001e-06 2024-09-11T06:51:26.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.001 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:26.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838640+0000", 2024-09-11T06:51:26.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5569999999999996e-05 2024-09-11T06:51:26.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.001 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:26.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838660+0000", 2024-09-11T06:51:26.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0576000000000001e-05 2024-09-11T06:51:26.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:26.002 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838674+0000", 2024-09-11T06:51:26.002 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3529999999999999e-05 2024-09-11T06:51:26.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:26.002 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838838+0000", 2024-09-11T06:51:26.002 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016468799999999999 2024-09-11T06:51:26.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:26.002 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838885+0000", 2024-09-11T06:51:26.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6884e-05 2024-09-11T06:51:26.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:26.003 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843901+0000", 2024-09-11T06:51:26.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050162310000000003 2024-09-11T06:51:26.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:26.003 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843998+0000", 2024-09-11T06:51:26.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.6718999999999993e-05 2024-09-11T06:51:26.004 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.004 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.004 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.004 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4558.0:1 3.3 e56/55 3:c4c92e5a:::rbd_trash:head v 56'1, mlcod=0'0)", 2024-09-11T06:51:26.004 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:51:26.004 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 68.836781111999997, 2024-09-11T06:51:26.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0053791810000000002, 2024-09-11T06:51:26.004 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:26.004 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:26.004 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:26.005 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:26.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:26.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:51:26.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:26.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:51:26.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.006 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:26.006 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091357+0000", 2024-09-11T06:51:26.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.54e-07 2024-09-11T06:51:26.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.006 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:26.006 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091376+0000", 2024-09-11T06:51:26.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9769e-05 2024-09-11T06:51:26.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.006 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:26.006 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091396+0000", 2024-09-11T06:51:26.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0152999999999998e-05 2024-09-11T06:51:26.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.007 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:26.007 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091416+0000", 2024-09-11T06:51:26.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0006000000000001e-05 2024-09-11T06:51:26.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.007 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:26.007 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091506+0000", 2024-09-11T06:51:26.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9325000000000003e-05 2024-09-11T06:51:26.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.008 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:26.008 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091523+0000", 2024-09-11T06:51:26.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.77e-05 2024-09-11T06:51:26.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.008 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:26.008 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.096685+0000", 2024-09-11T06:51:26.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0051618569999999997 2024-09-11T06:51:26.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.008 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:26.008 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.096735+0000", 2024-09-11T06:51:26.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9517e-05 2024-09-11T06:51:26.009 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.009 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.009 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.009 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.5544.0:67 2.5 e82/69 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head v 82'77, mlcod=81'74)", 2024-09-11T06:51:26.009 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.595302+0000", 2024-09-11T06:51:26.010 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.332834695999999, 2024-09-11T06:51:26.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054511999999999998, 2024-09-11T06:51:26.010 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:26.010 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:26.010 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:26.010 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:26.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:26.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595302+0000", 2024-09-11T06:51:26.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.011 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:26.011 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595302+0000", 2024-09-11T06:51:26.011 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.011 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:26.011 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595303+0000", 2024-09-11T06:51:26.011 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2699999999999999e-07 2024-09-11T06:51:26.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.012 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:26.012 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595522+0000", 2024-09-11T06:51:26.012 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000218946 2024-09-11T06:51:26.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.012 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.012 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:26.012 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595544+0000", 2024-09-11T06:51:26.012 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2558999999999999e-05 2024-09-11T06:51:26.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.012 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.012 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:26.012 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595557+0000", 2024-09-11T06:51:26.013 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2653000000000001e-05 2024-09-11T06:51:26.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.013 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:26.013 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595662+0000", 2024-09-11T06:51:26.013 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000104862 2024-09-11T06:51:26.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.013 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:26.014 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595699+0000", 2024-09-11T06:51:26.014 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7376000000000003e-05 2024-09-11T06:51:26.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.014 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:26.014 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.600688+0000", 2024-09-11T06:51:26.014 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0049885279999999999 2024-09-11T06:51:26.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.014 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:26.015 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.600753+0000", 2024-09-11T06:51:26.015 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5549000000000005e-05 2024-09-11T06:51:26.015 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.015 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.015 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.015 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.5544.0:90 2.5 e82/69 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head v 82'86, mlcod=82'84)", 2024-09-11T06:51:26.015 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.725934+0000", 2024-09-11T06:51:26.015 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.202202935000003, 2024-09-11T06:51:26.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0044823220000000004, 2024-09-11T06:51:26.016 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:26.016 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:26.016 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:26.016 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:26.016 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.016 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:26.016 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.725934+0000", 2024-09-11T06:51:26.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.016 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.016 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:26.016 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.725934+0000", 2024-09-11T06:51:26.017 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.017 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:26.017 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.725936+0000", 2024-09-11T06:51:26.017 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9269999999999999e-06 2024-09-11T06:51:26.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.017 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:26.017 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.727188+0000", 2024-09-11T06:51:26.017 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012522779999999999 2024-09-11T06:51:26.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.017 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:26.018 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.727217+0000", 2024-09-11T06:51:26.018 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9373000000000002e-05 2024-09-11T06:51:26.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.018 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.018 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:26.018 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.727231+0000", 2024-09-11T06:51:26.018 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3499999999999999e-05 2024-09-11T06:51:26.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.018 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.018 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:26.018 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.727335+0000", 2024-09-11T06:51:26.018 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000104064 2024-09-11T06:51:26.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:26.019 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.727374+0000", 2024-09-11T06:51:26.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8967000000000001e-05 2024-09-11T06:51:26.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:26.019 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.730326+0000", 2024-09-11T06:51:26.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0029516519999999999 2024-09-11T06:51:26.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:26.019 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.730416+0000", 2024-09-11T06:51:26.020 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0561000000000002e-05 2024-09-11T06:51:26.020 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.020 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.020 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.020 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.6081.0:30 2.4 2:2d66d25d:::journal_data.2.1314f0a2bfa2.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+full_force+supports_pool_eio e101)", 2024-09-11T06:51:26.020 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:20.532261+0000", 2024-09-11T06:51:26.020 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.3958757009999996, 2024-09-11T06:51:26.020 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0081325019999999998, 2024-09-11T06:51:26.020 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:26.020 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:26.020 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:26.021 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:26.021 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.6081", 2024-09-11T06:51:26.021 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3800899964", 2024-09-11T06:51:26.021 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 30 2024-09-11T06:51:26.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.021 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:26.021 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.021 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:26.021 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.532261+0000", 2024-09-11T06:51:26.021 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.021 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.021 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:26.022 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.532261+0000", 2024-09-11T06:51:26.022 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.022 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:26.022 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.532263+0000", 2024-09-11T06:51:26.022 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.34e-06 2024-09-11T06:51:26.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.022 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:26.022 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.532274+0000", 2024-09-11T06:51:26.022 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.045e-05 2024-09-11T06:51:26.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.023 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:26.023 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.532305+0000", 2024-09-11T06:51:26.023 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0713000000000002e-05 2024-09-11T06:51:26.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.023 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.023 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:26.023 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.532330+0000", 2024-09-11T06:51:26.023 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5066999999999999e-05 2024-09-11T06:51:26.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.023 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.023 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:26.023 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.534090+0000", 2024-09-11T06:51:26.023 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0017608369999999999 2024-09-11T06:51:26.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.023 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.024 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:26.024 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.534258+0000", 2024-09-11T06:51:26.024 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016717300000000001 2024-09-11T06:51:26.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.024 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.024 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 4", 2024-09-11T06:51:26.024 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.534409+0000", 2024-09-11T06:51:26.024 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000150896 2024-09-11T06:51:26.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.025 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.025 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:26.025 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.535023+0000", 2024-09-11T06:51:26.025 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00061441600000000001 2024-09-11T06:51:26.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.025 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.025 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:26.025 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.540274+0000", 2024-09-11T06:51:26.025 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052512879999999998 2024-09-11T06:51:26.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.025 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.025 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:26.026 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.540317+0000", 2024-09-11T06:51:26.026 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.278e-05 2024-09-11T06:51:26.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.026 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:26.026 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.540394+0000", 2024-09-11T06:51:26.026 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6542e-05 2024-09-11T06:51:26.026 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.026 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.026 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.026 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.6081.0:37 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-11T06:51:26.027 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:20.548027+0000", 2024-09-11T06:51:26.027 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.3801101679999999, 2024-09-11T06:51:26.027 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.010236906, 2024-09-11T06:51:26.027 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:26.027 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:26.027 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:26.027 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:26.027 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.6081", 2024-09-11T06:51:26.027 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3800899964", 2024-09-11T06:51:26.027 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 37 2024-09-11T06:51:26.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.028 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:26.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.028 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:26.028 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.548027+0000", 2024-09-11T06:51:26.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.028 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:26.028 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.548027+0000", 2024-09-11T06:51:26.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.028 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:26.028 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.548028+0000", 2024-09-11T06:51:26.029 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7910000000000001e-06 2024-09-11T06:51:26.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.029 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:26.029 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.548035+0000", 2024-09-11T06:51:26.029 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6039999999999998e-06 2024-09-11T06:51:26.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.029 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:26.029 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.548049+0000", 2024-09-11T06:51:26.029 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3594000000000001e-05 2024-09-11T06:51:26.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.029 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:26.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.548057+0000", 2024-09-11T06:51:26.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4749999999999993e-06 2024-09-11T06:51:26.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:26.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.548120+0000", 2024-09-11T06:51:26.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2958000000000003e-05 2024-09-11T06:51:26.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:26.031 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.548197+0000", 2024-09-11T06:51:26.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7260000000000002e-05 2024-09-11T06:51:26.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.031 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 4", 2024-09-11T06:51:26.031 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.548337+0000", 2024-09-11T06:51:26.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013991200000000001 2024-09-11T06:51:26.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.031 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:26.031 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.548933+0000", 2024-09-11T06:51:26.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00059574499999999996 2024-09-11T06:51:26.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:26.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.558153+0000", 2024-09-11T06:51:26.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0092196959999999994 2024-09-11T06:51:26.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:26.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.558197+0000", 2024-09-11T06:51:26.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3856999999999997e-05 2024-09-11T06:51:26.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:26.033 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.558264+0000", 2024-09-11T06:51:26.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7014000000000006e-05 2024-09-11T06:51:26.033 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.033 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.033 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.033 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.6105.0:17 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-11T06:51:26.033 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:20.642981+0000", 2024-09-11T06:51:26.034 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.2851555010000002, 2024-09-11T06:51:26.034 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0043724539999999996, 2024-09-11T06:51:26.034 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:26.034 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:26.034 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:26.034 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:26.034 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.6105", 2024-09-11T06:51:26.034 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/722227396", 2024-09-11T06:51:26.034 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 17 2024-09-11T06:51:26.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.034 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:26.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.035 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:26.035 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.642981+0000", 2024-09-11T06:51:26.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.035 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:26.035 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.642981+0000", 2024-09-11T06:51:26.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:26.036 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.642984+0000", 2024-09-11T06:51:26.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7429999999999998e-06 2024-09-11T06:51:26.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:26.036 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.642994+0000", 2024-09-11T06:51:26.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0132999999999999e-05 2024-09-11T06:51:26.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:26.036 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.643016+0000", 2024-09-11T06:51:26.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1546999999999999e-05 2024-09-11T06:51:26.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.037 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:26.037 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.643031+0000", 2024-09-11T06:51:26.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4856e-05 2024-09-11T06:51:26.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.037 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:26.037 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.643143+0000", 2024-09-11T06:51:26.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000112539 2024-09-11T06:51:26.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.037 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:26.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.643302+0000", 2024-09-11T06:51:26.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015875500000000001 2024-09-11T06:51:26.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 4", 2024-09-11T06:51:26.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.643590+0000", 2024-09-11T06:51:26.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000288261 2024-09-11T06:51:26.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:26.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.644886+0000", 2024-09-11T06:51:26.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012958869999999999 2024-09-11T06:51:26.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.039 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:26.039 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.647117+0000", 2024-09-11T06:51:26.039 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002230662 2024-09-11T06:51:26.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.039 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.039 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:26.039 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.647216+0000", 2024-09-11T06:51:26.039 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8995999999999999e-05 2024-09-11T06:51:26.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.039 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.039 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:26.040 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.647354+0000", 2024-09-11T06:51:26.040 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013807500000000001 2024-09-11T06:51:26.040 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.040 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.040 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.040 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.040 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.040 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:26.041 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T06:51:26.042 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:26.037+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:26.043 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:51:26.044 INFO:tasks.workunit.client.0.smithi176.stderr:+ unmap_device /dev/nbd2 24979 2024-09-11T06:51:26.044 INFO:tasks.workunit.client.0.smithi176.stderr:+ local args=/dev/nbd2 2024-09-11T06:51:26.044 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pid=24979 2024-09-11T06:51:26.044 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd2 2024-09-11T06:51:26.044 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:26.044 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:26.044 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:26.044 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:51:26.045 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:51:26.045 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:26.045 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd2 2024-09-11T06:51:26.067 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:51:26.067 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^24979\b' 2024-09-11T06:51:26.067 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^24979\b' 2024-09-11T06:51:26.131 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:51:26.132 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -C rbd-nbd 2024-09-11T06:51:26.132 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^ *24979\b' 2024-09-11T06:51:26.132 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^ *24979\b' 2024-09-11T06:51:26.137 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:51:26.137 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 0.5 2024-09-11T06:51:26.142 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:26.137+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:26.211 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:26.211 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:26.212 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:51:26.212 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:26.227 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:51:26.243 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:26.237+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:26.344 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:26.344 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:26.344 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:51:26.344 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:51:26.344 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:26.346 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:26.342+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:26.356 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:51:26.446 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:26.445+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:26.523 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:51:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:51:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:51:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4558.0:5 3.2 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-11T06:51:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:51:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 68.432492326000002, 2024-09-11T06:51:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0062353959999999998, 2024-09-11T06:51:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4558", 2024-09-11T06:51:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1478388898", 2024-09-11T06:51:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 5 2024-09-11T06:51:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:51:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:51:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088422+0000", 2024-09-11T06:51:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7859999999999998e-06 2024-09-11T06:51:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088440+0000", 2024-09-11T06:51:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8185999999999998e-05 2024-09-11T06:51:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088479+0000", 2024-09-11T06:51:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8248999999999999e-05 2024-09-11T06:51:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088497+0000", 2024-09-11T06:51:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8889000000000002e-05 2024-09-11T06:51:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088658+0000", 2024-09-11T06:51:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000160271 2024-09-11T06:51:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op must wait for map", 2024-09-11T06:51:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088668+0000", 2024-09-11T06:51:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0409e-05 2024-09-11T06:51:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092347+0000", 2024-09-11T06:51:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00367915 2024-09-11T06:51:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092541+0000", 2024-09-11T06:51:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000193833 2024-09-11T06:51:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-11T06:51:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092732+0000", 2024-09-11T06:51:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000190772 2024-09-11T06:51:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094136+0000", 2024-09-11T06:51:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014040809999999999 2024-09-11T06:51:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094506+0000", 2024-09-11T06:51:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000369959 2024-09-11T06:51:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094590+0000", 2024-09-11T06:51:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4033e-05 2024-09-11T06:51:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094654+0000", 2024-09-11T06:51:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3777999999999998e-05 2024-09-11T06:51:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:587 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-11T06:51:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:51:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 64.291659671000005, 2024-09-11T06:51:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0023436289999999999, 2024-09-11T06:51:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:51:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:51:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 587 2024-09-11T06:51:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:51:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:51:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229252+0000", 2024-09-11T06:51:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3799999999999999e-06 2024-09-11T06:51:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229259+0000", 2024-09-11T06:51:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4760000000000003e-06 2024-09-11T06:51:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229271+0000", 2024-09-11T06:51:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2313000000000001e-05 2024-09-11T06:51:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229279+0000", 2024-09-11T06:51:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1610000000000003e-06 2024-09-11T06:51:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229344+0000", 2024-09-11T06:51:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4375000000000006e-05 2024-09-11T06:51:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229404+0000", 2024-09-11T06:51:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0517999999999999e-05 2024-09-11T06:51:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229516+0000", 2024-09-11T06:51:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011129900000000001 2024-09-11T06:51:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231502+0000", 2024-09-11T06:51:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001986238 2024-09-11T06:51:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231537+0000", 2024-09-11T06:51:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4968000000000001e-05 2024-09-11T06:51:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231569+0000", 2024-09-11T06:51:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2459000000000002e-05 2024-09-11T06:51:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231595+0000", 2024-09-11T06:51:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5442000000000001e-05 2024-09-11T06:51:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:15 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.709418+0000", 2024-09-11T06:51:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 55.811492518000001, 2024-09-11T06:51:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027504280000000001, 2024-09-11T06:51:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 15 2024-09-11T06:51:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709418+0000", 2024-09-11T06:51:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709418+0000", 2024-09-11T06:51:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709420+0000", 2024-09-11T06:51:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2299999999999998e-06 2024-09-11T06:51:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709432+0000", 2024-09-11T06:51:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1897e-05 2024-09-11T06:51:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709464+0000", 2024-09-11T06:51:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1819000000000001e-05 2024-09-11T06:51:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709479+0000", 2024-09-11T06:51:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4853e-05 2024-09-11T06:51:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709624+0000", 2024-09-11T06:51:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144912 2024-09-11T06:51:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709773+0000", 2024-09-11T06:51:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000149529 2024-09-11T06:51:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.710055+0000", 2024-09-11T06:51:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000281692 2024-09-11T06:51:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.711412+0000", 2024-09-11T06:51:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001356931 2024-09-11T06:51:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.711890+0000", 2024-09-11T06:51:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00047768700000000002 2024-09-11T06:51:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.712050+0000", 2024-09-11T06:51:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016041499999999999 2024-09-11T06:51:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.712169+0000", 2024-09-11T06:51:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011846299999999999 2024-09-11T06:51:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:18 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.713502+0000", 2024-09-11T06:51:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 55.807409092, 2024-09-11T06:51:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032323069999999998, 2024-09-11T06:51:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18 2024-09-11T06:51:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713502+0000", 2024-09-11T06:51:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713502+0000", 2024-09-11T06:51:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713503+0000", 2024-09-11T06:51:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8199999999999999e-06 2024-09-11T06:51:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713513+0000", 2024-09-11T06:51:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8539999999999992e-06 2024-09-11T06:51:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713540+0000", 2024-09-11T06:51:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6594999999999999e-05 2024-09-11T06:51:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713553+0000", 2024-09-11T06:51:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2843999999999999e-05 2024-09-11T06:51:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713686+0000", 2024-09-11T06:51:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013294899999999999 2024-09-11T06:51:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713937+0000", 2024-09-11T06:51:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025110600000000002 2024-09-11T06:51:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.714412+0000", 2024-09-11T06:51:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00047562 2024-09-11T06:51:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.715867+0000", 2024-09-11T06:51:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001455004 2024-09-11T06:51:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.716569+0000", 2024-09-11T06:51:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070159400000000005 2024-09-11T06:51:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.716660+0000", 2024-09-11T06:51:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1403000000000006e-05 2024-09-11T06:51:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.716734+0000", 2024-09-11T06:51:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3517999999999996e-05 2024-09-11T06:51:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:23 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.723392+0000", 2024-09-11T06:51:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 55.797518496000002, 2024-09-11T06:51:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027098489999999999, 2024-09-11T06:51:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23 2024-09-11T06:51:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723392+0000", 2024-09-11T06:51:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723392+0000", 2024-09-11T06:51:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723394+0000", 2024-09-11T06:51:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0669999999999999e-06 2024-09-11T06:51:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723406+0000", 2024-09-11T06:51:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1367e-05 2024-09-11T06:51:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723436+0000", 2024-09-11T06:51:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0874999999999998e-05 2024-09-11T06:51:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723451+0000", 2024-09-11T06:51:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4192999999999999e-05 2024-09-11T06:51:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723577+0000", 2024-09-11T06:51:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000126267 2024-09-11T06:51:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723718+0000", 2024-09-11T06:51:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000141206 2024-09-11T06:51:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723982+0000", 2024-09-11T06:51:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026338499999999999 2024-09-11T06:51:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.725377+0000", 2024-09-11T06:51:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013956999999999999 2024-09-11T06:51:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.725908+0000", 2024-09-11T06:51:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00053036200000000004 2024-09-11T06:51:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.725999+0000", 2024-09-11T06:51:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1016999999999999e-05 2024-09-11T06:51:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726102+0000", 2024-09-11T06:51:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010341 2024-09-11T06:51:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:15 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.868921+0000", 2024-09-11T06:51:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 55.651989454999999, 2024-09-11T06:51:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0026879, 2024-09-11T06:51:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 15 2024-09-11T06:51:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868921+0000", 2024-09-11T06:51:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.560 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.560 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.560 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:26.560 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868921+0000", 2024-09-11T06:51:26.560 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.560 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.560 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.560 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:26.560 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868923+0000", 2024-09-11T06:51:26.560 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8959999999999999e-06 2024-09-11T06:51:26.560 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.560 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.560 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:26.560 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868934+0000", 2024-09-11T06:51:26.561 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1083e-05 2024-09-11T06:51:26.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.561 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.561 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:26.561 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868965+0000", 2024-09-11T06:51:26.561 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0833000000000001e-05 2024-09-11T06:51:26.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.561 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.561 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:26.561 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868990+0000", 2024-09-11T06:51:26.561 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4545999999999998e-05 2024-09-11T06:51:26.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.561 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.561 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.869124+0000", 2024-09-11T06:51:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013489599999999999 2024-09-11T06:51:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.869270+0000", 2024-09-11T06:51:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000145976 2024-09-11T06:51:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.869538+0000", 2024-09-11T06:51:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026780599999999999 2024-09-11T06:51:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.870691+0000", 2024-09-11T06:51:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00115277 2024-09-11T06:51:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.871456+0000", 2024-09-11T06:51:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00076473800000000005 2024-09-11T06:51:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.871543+0000", 2024-09-11T06:51:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7040999999999996e-05 2024-09-11T06:51:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.871609+0000", 2024-09-11T06:51:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6315000000000005e-05 2024-09-11T06:51:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:20 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.877429+0000", 2024-09-11T06:51:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 55.643481166000001, 2024-09-11T06:51:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032382629999999999, 2024-09-11T06:51:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:26.565 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:26.565 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:26.565 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:26.565 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:26.565 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:26.565 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 20 2024-09-11T06:51:26.565 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.565 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:26.565 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.565 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:26.565 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877429+0000", 2024-09-11T06:51:26.565 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.565 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877429+0000", 2024-09-11T06:51:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877431+0000", 2024-09-11T06:51:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9429999999999999e-06 2024-09-11T06:51:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877444+0000", 2024-09-11T06:51:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2396e-05 2024-09-11T06:51:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877474+0000", 2024-09-11T06:51:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0239999999999998e-05 2024-09-11T06:51:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877496+0000", 2024-09-11T06:51:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1914999999999999e-05 2024-09-11T06:51:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877616+0000", 2024-09-11T06:51:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000119814 2024-09-11T06:51:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:26.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877836+0000", 2024-09-11T06:51:26.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022029600000000001 2024-09-11T06:51:26.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:26.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.878482+0000", 2024-09-11T06:51:26.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00064605200000000004 2024-09-11T06:51:26.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:26.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.879804+0000", 2024-09-11T06:51:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013216460000000001 2024-09-11T06:51:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.880519+0000", 2024-09-11T06:51:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00071547600000000005 2024-09-11T06:51:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.880606+0000", 2024-09-11T06:51:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6907e-05 2024-09-11T06:51:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:26.570 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.880668+0000", 2024-09-11T06:51:26.570 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1577999999999999e-05 2024-09-11T06:51:26.570 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.570 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.570 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.570 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.570 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:23 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:26.570 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.885067+0000", 2024-09-11T06:51:26.570 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 55.635843512000001, 2024-09-11T06:51:26.570 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028127320000000001, 2024-09-11T06:51:26.570 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:26.570 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23 2024-09-11T06:51:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885067+0000", 2024-09-11T06:51:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885067+0000", 2024-09-11T06:51:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885069+0000", 2024-09-11T06:51:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2000000000000001e-06 2024-09-11T06:51:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885081+0000", 2024-09-11T06:51:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1209999999999999e-05 2024-09-11T06:51:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885111+0000", 2024-09-11T06:51:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0318999999999999e-05 2024-09-11T06:51:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885132+0000", 2024-09-11T06:51:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0649000000000002e-05 2024-09-11T06:51:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885243+0000", 2024-09-11T06:51:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000111284 2024-09-11T06:51:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885387+0000", 2024-09-11T06:51:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144222 2024-09-11T06:51:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885699+0000", 2024-09-11T06:51:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00031239800000000003 2024-09-11T06:51:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.886817+0000", 2024-09-11T06:51:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001117191 2024-09-11T06:51:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.887684+0000", 2024-09-11T06:51:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00086699499999999996 2024-09-11T06:51:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.887772+0000", 2024-09-11T06:51:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8224000000000001e-05 2024-09-11T06:51:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.887880+0000", 2024-09-11T06:51:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010804 2024-09-11T06:51:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "MOSDPGPull(2.0 e70/69 cost 1048576)", 2024-09-11T06:51:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:35.528435+0000", 2024-09-11T06:51:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 50.992475775999999, 2024-09-11T06:51:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027390520000000001, 2024-09-11T06:51:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:26.577 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:26.577 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-11T06:51:26.577 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:26.577 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.577 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:26.577 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528435+0000", 2024-09-11T06:51:26.577 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.577 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.577 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:26.577 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528435+0000", 2024-09-11T06:51:26.577 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528437+0000", 2024-09-11T06:51:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5959999999999998e-06 2024-09-11T06:51:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528465+0000", 2024-09-11T06:51:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7436e-05 2024-09-11T06:51:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528499+0000", 2024-09-11T06:51:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3599000000000001e-05 2024-09-11T06:51:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528515+0000", 2024-09-11T06:51:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.677e-05 2024-09-11T06:51:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528652+0000", 2024-09-11T06:51:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000136546 2024-09-11T06:51:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.531174+0000", 2024-09-11T06:51:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0025221050000000002 2024-09-11T06:51:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4828.0:15 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-11T06:51:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:36.253730+0000", 2024-09-11T06:51:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 50.267180433, 2024-09-11T06:51:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0025257589999999998, 2024-09-11T06:51:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4828", 2024-09-11T06:51:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2410288650", 2024-09-11T06:51:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 15 2024-09-11T06:51:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253730+0000", 2024-09-11T06:51:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253730+0000", 2024-09-11T06:51:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253732+0000", 2024-09-11T06:51:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2400000000000002e-06 2024-09-11T06:51:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253744+0000", 2024-09-11T06:51:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1316e-05 2024-09-11T06:51:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253771+0000", 2024-09-11T06:51:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.724e-05 2024-09-11T06:51:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253785+0000", 2024-09-11T06:51:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3786e-05 2024-09-11T06:51:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253886+0000", 2024-09-11T06:51:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000100981 2024-09-11T06:51:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.254029+0000", 2024-09-11T06:51:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014311900000000001 2024-09-11T06:51:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.254317+0000", 2024-09-11T06:51:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00028809300000000001 2024-09-11T06:51:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.255403+0000", 2024-09-11T06:51:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001085639 2024-09-11T06:51:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.256107+0000", 2024-09-11T06:51:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070468799999999997 2024-09-11T06:51:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.256191+0000", 2024-09-11T06:51:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3714999999999994e-05 2024-09-11T06:51:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.256256+0000", 2024-09-11T06:51:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4942000000000003e-05 2024-09-11T06:51:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5313.0:26 2.6 2:7b712441:ns::rbd_object_map.11adc335555f:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:51:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.260230+0000", 2024-09-11T06:51:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 47.260681024999997, 2024-09-11T06:51:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002457201, 2024-09-11T06:51:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5313", 2024-09-11T06:51:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/624314229", 2024-09-11T06:51:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:51:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260230+0000", 2024-09-11T06:51:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260230+0000", 2024-09-11T06:51:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260232+0000", 2024-09-11T06:51:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.956e-06 2024-09-11T06:51:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260243+0000", 2024-09-11T06:51:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1687e-05 2024-09-11T06:51:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260269+0000", 2024-09-11T06:51:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6211999999999999e-05 2024-09-11T06:51:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260284+0000", 2024-09-11T06:51:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4053000000000001e-05 2024-09-11T06:51:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260391+0000", 2024-09-11T06:51:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000107278 2024-09-11T06:51:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260541+0000", 2024-09-11T06:51:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015033000000000001 2024-09-11T06:51:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 3", 2024-09-11T06:51:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260951+0000", 2024-09-11T06:51:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040981299999999997 2024-09-11T06:51:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262399+0000", 2024-09-11T06:51:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014475930000000001 2024-09-11T06:51:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262531+0000", 2024-09-11T06:51:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013239 2024-09-11T06:51:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262623+0000", 2024-09-11T06:51:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2361000000000006e-05 2024-09-11T06:51:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262687+0000", 2024-09-11T06:51:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3528000000000006e-05 2024-09-11T06:51:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5313.0:31 2.6 2:7b712441:ns::rbd_object_map.11adc335555f:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:51:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.266495+0000", 2024-09-11T06:51:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 47.254415250999998, 2024-09-11T06:51:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0026763870000000001, 2024-09-11T06:51:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5313", 2024-09-11T06:51:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/624314229", 2024-09-11T06:51:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 31 2024-09-11T06:51:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266495+0000", 2024-09-11T06:51:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266495+0000", 2024-09-11T06:51:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.594 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.594 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.594 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:26.594 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266498+0000", 2024-09-11T06:51:26.594 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2369999999999999e-06 2024-09-11T06:51:26.594 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.594 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.594 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:26.594 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266509+0000", 2024-09-11T06:51:26.594 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1459e-05 2024-09-11T06:51:26.594 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.595 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.595 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:26.595 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266540+0000", 2024-09-11T06:51:26.595 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1189999999999998e-05 2024-09-11T06:51:26.595 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.595 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.595 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:26.595 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266563+0000", 2024-09-11T06:51:26.595 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2529000000000001e-05 2024-09-11T06:51:26.595 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.595 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.595 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:26.595 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266708+0000", 2024-09-11T06:51:26.595 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014511 2024-09-11T06:51:26.596 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.596 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.596 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:26.596 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266962+0000", 2024-09-11T06:51:26.596 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000253897 2024-09-11T06:51:26.596 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.596 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.596 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 3", 2024-09-11T06:51:26.596 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.267430+0000", 2024-09-11T06:51:26.596 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00046799899999999998 2024-09-11T06:51:26.596 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.596 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.596 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:26.597 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.268765+0000", 2024-09-11T06:51:26.597 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013350359999999999 2024-09-11T06:51:26.597 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.597 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.597 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:26.597 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.269019+0000", 2024-09-11T06:51:26.597 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025397000000000001 2024-09-11T06:51:26.597 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.597 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.597 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:26.597 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.269105+0000", 2024-09-11T06:51:26.597 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5952000000000006e-05 2024-09-11T06:51:26.597 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.598 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.598 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:26.598 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.269172+0000", 2024-09-11T06:51:26.598 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7008000000000006e-05 2024-09-11T06:51:26.598 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.598 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.598 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.598 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.598 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.598 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:5 3.4 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:26.598 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.588582+0000", 2024-09-11T06:51:26.598 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 40.932329097999997, 2024-09-11T06:51:26.598 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002889857, 2024-09-11T06:51:26.599 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:26.599 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:26.599 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:26.599 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:26.599 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:51:26.599 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:51:26.599 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 5 2024-09-11T06:51:26.599 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.599 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:26.599 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.599 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:26.599 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588582+0000", 2024-09-11T06:51:26.599 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.599 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.600 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.600 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:26.600 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588582+0000", 2024-09-11T06:51:26.600 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.600 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.600 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.600 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:26.600 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588583+0000", 2024-09-11T06:51:26.600 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.223e-06 2024-09-11T06:51:26.600 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.600 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.600 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:26.600 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588588+0000", 2024-09-11T06:51:26.600 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4299999999999997e-06 2024-09-11T06:51:26.601 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.601 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.601 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:26.601 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588606+0000", 2024-09-11T06:51:26.601 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7652999999999999e-05 2024-09-11T06:51:26.601 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.601 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.601 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:26.601 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588615+0000", 2024-09-11T06:51:26.601 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4399999999999994e-06 2024-09-11T06:51:26.601 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.601 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.601 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:26.601 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588725+0000", 2024-09-11T06:51:26.602 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010945799999999999 2024-09-11T06:51:26.602 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.602 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.602 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:26.602 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.589031+0000", 2024-09-11T06:51:26.602 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030590000000000001 2024-09-11T06:51:26.602 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.602 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.602 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:26.602 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.589521+0000", 2024-09-11T06:51:26.602 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00049061299999999999 2024-09-11T06:51:26.602 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.602 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.602 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:26.603 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.590792+0000", 2024-09-11T06:51:26.603 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012708039999999999 2024-09-11T06:51:26.603 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.603 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.603 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:26.603 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591339+0000", 2024-09-11T06:51:26.603 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00054710499999999999 2024-09-11T06:51:26.603 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.603 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.603 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:26.603 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591412+0000", 2024-09-11T06:51:26.603 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2811999999999993e-05 2024-09-11T06:51:26.603 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.604 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.604 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:26.604 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591471+0000", 2024-09-11T06:51:26.604 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9419000000000001e-05 2024-09-11T06:51:26.604 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.604 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.604 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.604 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.604 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.604 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:8 3.4 3:3f0de396:::rbd_header.1415bf940188:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:26.604 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.596120+0000", 2024-09-11T06:51:26.604 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 40.924790213999998, 2024-09-11T06:51:26.604 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027911020000000002, 2024-09-11T06:51:26.604 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:26.605 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:26.605 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:26.605 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:26.605 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:51:26.605 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:51:26.605 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 8 2024-09-11T06:51:26.605 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.605 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:26.605 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.605 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:26.605 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596120+0000", 2024-09-11T06:51:26.605 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.605 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.606 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.606 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:26.606 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596120+0000", 2024-09-11T06:51:26.606 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.606 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.606 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.606 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:26.606 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596122+0000", 2024-09-11T06:51:26.606 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.513e-06 2024-09-11T06:51:26.606 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.606 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.606 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:26.606 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596130+0000", 2024-09-11T06:51:26.606 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4999999999999999e-06 2024-09-11T06:51:26.607 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.607 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.607 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:26.607 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596152+0000", 2024-09-11T06:51:26.607 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1846000000000001e-05 2024-09-11T06:51:26.607 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.607 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.607 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:26.607 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596163+0000", 2024-09-11T06:51:26.607 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1029999999999999e-05 2024-09-11T06:51:26.607 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.607 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.607 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:26.607 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596242+0000", 2024-09-11T06:51:26.608 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8275000000000005e-05 2024-09-11T06:51:26.608 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.608 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.608 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:26.608 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596439+0000", 2024-09-11T06:51:26.608 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000196999 2024-09-11T06:51:26.608 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.608 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.608 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:26.608 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596755+0000", 2024-09-11T06:51:26.608 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00031626100000000003 2024-09-11T06:51:26.608 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.608 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.609 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:26.609 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.598043+0000", 2024-09-11T06:51:26.609 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001288389 2024-09-11T06:51:26.609 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.609 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.609 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:26.609 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.598774+0000", 2024-09-11T06:51:26.609 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00073080499999999995 2024-09-11T06:51:26.609 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.609 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.609 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:26.609 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.598855+0000", 2024-09-11T06:51:26.609 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0615e-05 2024-09-11T06:51:26.609 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.609 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.610 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:26.610 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.598912+0000", 2024-09-11T06:51:26.610 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6869e-05 2024-09-11T06:51:26.610 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.610 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.610 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.610 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.610 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.610 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:9 3.0 3:00834abc:::rbd_object_map.1415bf940188:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:26.610 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.599161+0000", 2024-09-11T06:51:26.610 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 40.921749196999997, 2024-09-11T06:51:26.610 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002826051, 2024-09-11T06:51:26.610 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:26.610 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:26.611 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:26.611 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:26.611 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:51:26.611 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:51:26.611 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 9 2024-09-11T06:51:26.611 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.611 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:26.611 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.611 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:26.612 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599161+0000", 2024-09-11T06:51:26.612 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.612 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.612 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.612 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:26.612 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599161+0000", 2024-09-11T06:51:26.612 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.612 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.612 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.612 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:26.612 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599163+0000", 2024-09-11T06:51:26.612 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8700000000000001e-06 2024-09-11T06:51:26.612 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.613 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.613 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:26.613 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599174+0000", 2024-09-11T06:51:26.613 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0416e-05 2024-09-11T06:51:26.613 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.613 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.613 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:26.613 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599202+0000", 2024-09-11T06:51:26.613 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7946e-05 2024-09-11T06:51:26.613 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.613 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.613 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:26.614 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599216+0000", 2024-09-11T06:51:26.614 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3934000000000001e-05 2024-09-11T06:51:26.614 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.614 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.614 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:26.614 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599345+0000", 2024-09-11T06:51:26.614 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012954000000000001 2024-09-11T06:51:26.614 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.614 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.614 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:26.614 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599540+0000", 2024-09-11T06:51:26.614 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019516299999999999 2024-09-11T06:51:26.614 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.615 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.615 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:26.615 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599796+0000", 2024-09-11T06:51:26.615 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025556500000000002 2024-09-11T06:51:26.615 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.615 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.615 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:26.615 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.601071+0000", 2024-09-11T06:51:26.615 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001275396 2024-09-11T06:51:26.615 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.615 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.615 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:26.615 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.601852+0000", 2024-09-11T06:51:26.615 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000780524 2024-09-11T06:51:26.615 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.616 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.616 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:26.616 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.601925+0000", 2024-09-11T06:51:26.616 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3574999999999999e-05 2024-09-11T06:51:26.616 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.616 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.616 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:26.616 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.601988+0000", 2024-09-11T06:51:26.616 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2121999999999999e-05 2024-09-11T06:51:26.616 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.616 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.616 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.616 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.617 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.617 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5038.0:12 3.4 3:3f0de396:::rbd_header.1415bf940188:head [watch watch cookie 140693532068064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:26.617 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.718024+0000", 2024-09-11T06:51:26.617 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 40.802886258999997, 2024-09-11T06:51:26.617 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002724583, 2024-09-11T06:51:26.617 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:26.617 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:26.617 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:26.617 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:26.617 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5038", 2024-09-11T06:51:26.617 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/945190275", 2024-09-11T06:51:26.617 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:51:26.617 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.617 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:26.618 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.618 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:26.618 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718024+0000", 2024-09-11T06:51:26.618 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.618 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.618 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.618 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:26.618 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718024+0000", 2024-09-11T06:51:26.618 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.618 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.618 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.618 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:26.618 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718027+0000", 2024-09-11T06:51:26.618 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1430000000000001e-06 2024-09-11T06:51:26.618 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.619 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.619 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:26.619 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718037+0000", 2024-09-11T06:51:26.619 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0253e-05 2024-09-11T06:51:26.619 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.619 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.619 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:26.619 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718068+0000", 2024-09-11T06:51:26.619 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1050000000000003e-05 2024-09-11T06:51:26.619 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.619 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.619 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:26.619 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718086+0000", 2024-09-11T06:51:26.619 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8629000000000001e-05 2024-09-11T06:51:26.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.620 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.620 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:26.620 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718206+0000", 2024-09-11T06:51:26.620 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000119358 2024-09-11T06:51:26.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.620 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.620 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:26.620 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718425+0000", 2024-09-11T06:51:26.620 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000218975 2024-09-11T06:51:26.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.620 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.620 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:26.620 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718830+0000", 2024-09-11T06:51:26.621 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040538199999999998 2024-09-11T06:51:26.621 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.621 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.621 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:26.621 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.719849+0000", 2024-09-11T06:51:26.621 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010187589999999999 2024-09-11T06:51:26.621 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.621 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.621 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:26.621 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.720548+0000", 2024-09-11T06:51:26.621 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000699112 2024-09-11T06:51:26.621 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.621 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.622 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:26.622 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.720632+0000", 2024-09-11T06:51:26.622 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4314999999999995e-05 2024-09-11T06:51:26.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.622 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.622 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:26.622 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.720749+0000", 2024-09-11T06:51:26.622 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116607 2024-09-11T06:51:26.622 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.622 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.622 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.622 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.622 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:47 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:26.623 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.553228+0000", 2024-09-11T06:51:26.623 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.967682169, 2024-09-11T06:51:26.623 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040683100000000003, 2024-09-11T06:51:26.623 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:26.623 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:26.623 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:26.623 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:26.623 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:26.623 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:26.623 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 47 2024-09-11T06:51:26.623 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.623 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:26.623 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.623 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:26.624 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553228+0000", 2024-09-11T06:51:26.624 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.624 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.624 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.624 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:26.624 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553228+0000", 2024-09-11T06:51:26.624 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.624 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.624 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.624 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:26.624 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553231+0000", 2024-09-11T06:51:26.624 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3130000000000001e-06 2024-09-11T06:51:26.624 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.624 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.625 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:26.625 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553242+0000", 2024-09-11T06:51:26.625 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1345999999999999e-05 2024-09-11T06:51:26.625 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.625 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.625 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:26.625 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553272+0000", 2024-09-11T06:51:26.625 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0179999999999999e-05 2024-09-11T06:51:26.625 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.625 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.625 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:26.625 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553287+0000", 2024-09-11T06:51:26.625 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4283e-05 2024-09-11T06:51:26.625 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.626 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.626 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:26.626 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553415+0000", 2024-09-11T06:51:26.626 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000128491 2024-09-11T06:51:26.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.626 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.626 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:26.626 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553651+0000", 2024-09-11T06:51:26.626 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000236328 2024-09-11T06:51:26.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.626 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.626 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:26.626 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.554356+0000", 2024-09-11T06:51:26.627 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070436299999999995 2024-09-11T06:51:26.627 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.627 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.627 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:26.627 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.556064+0000", 2024-09-11T06:51:26.627 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0017084439999999999 2024-09-11T06:51:26.627 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.627 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.627 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:26.627 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.556993+0000", 2024-09-11T06:51:26.627 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00092912400000000003 2024-09-11T06:51:26.627 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.627 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.627 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:26.628 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.557173+0000", 2024-09-11T06:51:26.628 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017940400000000001 2024-09-11T06:51:26.628 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.628 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.628 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:26.628 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.557297+0000", 2024-09-11T06:51:26.628 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000124034 2024-09-11T06:51:26.628 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.628 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.628 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.628 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.628 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.628 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:66 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:26.628 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.589465+0000", 2024-09-11T06:51:26.629 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.931445363999998, 2024-09-11T06:51:26.629 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039855489999999997, 2024-09-11T06:51:26.629 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:26.629 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:26.629 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:26.629 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:26.629 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:26.629 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:26.629 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 66 2024-09-11T06:51:26.630 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.630 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:26.630 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.630 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:26.630 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589465+0000", 2024-09-11T06:51:26.630 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.630 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.630 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.630 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:26.630 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589465+0000", 2024-09-11T06:51:26.630 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.630 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.630 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.631 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:26.631 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589468+0000", 2024-09-11T06:51:26.631 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3539999999999998e-06 2024-09-11T06:51:26.631 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.631 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.631 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:26.631 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589480+0000", 2024-09-11T06:51:26.631 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1946e-05 2024-09-11T06:51:26.631 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.631 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.631 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:26.631 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589512+0000", 2024-09-11T06:51:26.631 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2712999999999997e-05 2024-09-11T06:51:26.631 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.631 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.632 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:26.632 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589527+0000", 2024-09-11T06:51:26.632 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4686e-05 2024-09-11T06:51:26.632 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.632 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.632 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:26.632 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589673+0000", 2024-09-11T06:51:26.632 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014630700000000001 2024-09-11T06:51:26.632 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.632 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.632 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:26.632 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589936+0000", 2024-09-11T06:51:26.632 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026234499999999998 2024-09-11T06:51:26.633 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.633 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.633 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:26.633 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.591001+0000", 2024-09-11T06:51:26.633 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001065023 2024-09-11T06:51:26.633 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.633 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.633 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:26.633 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.592757+0000", 2024-09-11T06:51:26.633 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0017560849999999999 2024-09-11T06:51:26.633 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.633 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.633 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:26.634 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.593297+0000", 2024-09-11T06:51:26.634 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00054007600000000001 2024-09-11T06:51:26.634 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.634 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.634 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:26.634 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.593385+0000", 2024-09-11T06:51:26.634 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8327999999999999e-05 2024-09-11T06:51:26.634 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.634 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.634 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:26.634 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.593451+0000", 2024-09-11T06:51:26.634 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5685999999999994e-05 2024-09-11T06:51:26.634 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.634 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.635 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.635 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.635 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.635 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5315.0:79 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-11T06:51:26.635 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:07.271557+0000", 2024-09-11T06:51:26.635 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.249353564, 2024-09-11T06:51:26.635 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0023454019999999999, 2024-09-11T06:51:26.635 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:26.635 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:26.635 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:26.635 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:26.635 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5315", 2024-09-11T06:51:26.635 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2243844038", 2024-09-11T06:51:26.635 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 79 2024-09-11T06:51:26.636 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.636 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:26.636 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.636 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:26.636 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271557+0000", 2024-09-11T06:51:26.636 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.636 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.636 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.636 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:26.636 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271557+0000", 2024-09-11T06:51:26.636 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.636 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.636 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.636 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:26.637 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271560+0000", 2024-09-11T06:51:26.637 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3560000000000001e-06 2024-09-11T06:51:26.637 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.637 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.637 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:26.637 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271596+0000", 2024-09-11T06:51:26.637 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5593000000000003e-05 2024-09-11T06:51:26.637 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.637 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.637 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:26.637 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271640+0000", 2024-09-11T06:51:26.637 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4125999999999997e-05 2024-09-11T06:51:26.637 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.637 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.638 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:26.638 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271652+0000", 2024-09-11T06:51:26.638 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1853e-05 2024-09-11T06:51:26.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.638 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.638 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:26.638 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271785+0000", 2024-09-11T06:51:26.638 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013291299999999999 2024-09-11T06:51:26.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.638 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.638 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:26.638 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271948+0000", 2024-09-11T06:51:26.638 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000163191 2024-09-11T06:51:26.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.639 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.639 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:26.639 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.272260+0000", 2024-09-11T06:51:26.639 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00031196700000000002 2024-09-11T06:51:26.639 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.639 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.639 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:26.639 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.273293+0000", 2024-09-11T06:51:26.639 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010324139999999999 2024-09-11T06:51:26.639 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.639 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.639 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:26.639 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.273805+0000", 2024-09-11T06:51:26.639 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00051249399999999995 2024-09-11T06:51:26.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.640 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.640 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:26.640 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.273858+0000", 2024-09-11T06:51:26.640 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2975e-05 2024-09-11T06:51:26.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.640 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.640 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:26.640 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.273902+0000", 2024-09-11T06:51:26.640 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4520000000000001e-05 2024-09-11T06:51:26.640 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.640 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.640 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.641 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.641 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.641 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5239.0:52 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-11T06:51:26.641 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:08.176534+0000", 2024-09-11T06:51:26.641 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.344376639, 2024-09-11T06:51:26.641 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0022800960000000001, 2024-09-11T06:51:26.641 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:26.641 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:26.641 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:26.641 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:26.641 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5239", 2024-09-11T06:51:26.641 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3565361812", 2024-09-11T06:51:26.641 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 52 2024-09-11T06:51:26.641 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.642 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:26.642 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.642 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:26.642 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176534+0000", 2024-09-11T06:51:26.642 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.642 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.642 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:26.642 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176534+0000", 2024-09-11T06:51:26.642 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:26.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.642 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.642 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:26.642 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176537+0000", 2024-09-11T06:51:26.642 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5600000000000001e-06 2024-09-11T06:51:26.643 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.643 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.643 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:26.643 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176551+0000", 2024-09-11T06:51:26.643 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3926e-05 2024-09-11T06:51:26.643 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.643 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.643 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:26.643 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176584+0000", 2024-09-11T06:51:26.643 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3105999999999998e-05 2024-09-11T06:51:26.643 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.643 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.643 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:26.643 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176600+0000", 2024-09-11T06:51:26.643 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6327000000000001e-05 2024-09-11T06:51:26.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.644 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.644 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:26.644 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176753+0000", 2024-09-11T06:51:26.644 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015280900000000001 2024-09-11T06:51:26.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.644 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.644 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:26.644 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176911+0000", 2024-09-11T06:51:26.644 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000158145 2024-09-11T06:51:26.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.644 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.644 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:26.645 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.177216+0000", 2024-09-11T06:51:26.645 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030545000000000002 2024-09-11T06:51:26.645 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.645 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.645 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:26.645 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.178461+0000", 2024-09-11T06:51:26.645 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012451210000000001 2024-09-11T06:51:26.645 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.645 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.645 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:26.645 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.178706+0000", 2024-09-11T06:51:26.645 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024469999999999998 2024-09-11T06:51:26.645 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.645 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.646 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:26.646 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.178763+0000", 2024-09-11T06:51:26.646 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7339000000000001e-05 2024-09-11T06:51:26.646 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:26.646 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:26.646 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:26.646 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.178814+0000", 2024-09-11T06:51:26.646 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0612999999999998e-05 2024-09-11T06:51:26.646 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.646 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.646 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.646 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:26.646 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:26.646 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:26.648 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV= 2024-09-11T06:51:26.648 INFO:tasks.workunit.client.0.smithi176.stderr:++ ceph-conf --show-config-value librados_thread_count 2024-09-11T06:51:26.648 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:51:26.649 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:26.642+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:26.664 INFO:tasks.workunit.client.0.smithi176.stderr:+ EXPECTED=2 2024-09-11T06:51:26.664 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo rbd device --device-type nbd map rbd/testrbdnbd21607 2024-09-11T06:51:26.664 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:51:26.665 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:51:26.666 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:51:26.666 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which rbd 2024-09-11T06:51:26.667 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/rbd 2024-09-11T06:51:26.667 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:51:26.668 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map rbd/testrbdnbd21607 2024-09-11T06:51:26.749 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:26.745+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:26.817 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV=/dev/nbd2 2024-09-11T06:51:26.818 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd 2024-09-11T06:51:26.818 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:51:26.818 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:51:26.818 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:51:26.818 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd2'\'']/id' 2024-09-11T06:51:26.822 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:26.822 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:26.822 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:51:26.822 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:26.837 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:51:26.881 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:26.878+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:26.920 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=25090 2024-09-11T06:51:26.920 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 25090 2024-09-11T06:51:26.920 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 25090 -C rbd-nbd 2024-09-11T06:51:26.927 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:51:26.928 INFO:tasks.workunit.client.0.smithi176.stdout: 25090 ? 00:00:00 rbd-nbd 2024-09-11T06:51:26.929 INFO:tasks.workunit.client.0.smithi176.stderr:++ ps -p 25090 -T 2024-09-11T06:51:26.929 INFO:tasks.workunit.client.0.smithi176.stderr:++ grep -c io_context_pool 2024-09-11T06:51:26.937 INFO:tasks.workunit.client.0.smithi176.stderr:+ ACTUAL=2 2024-09-11T06:51:26.937 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 2 -eq 2 ']' 2024-09-11T06:51:26.937 INFO:tasks.workunit.client.0.smithi176.stderr:+ unmap_device /dev/nbd2 25090 2024-09-11T06:51:26.937 INFO:tasks.workunit.client.0.smithi176.stderr:+ local args=/dev/nbd2 2024-09-11T06:51:26.937 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pid=25090 2024-09-11T06:51:26.938 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd2 2024-09-11T06:51:26.938 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:26.938 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:26.938 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:26.938 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:51:26.940 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:51:26.940 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:26.940 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd2 2024-09-11T06:51:26.951 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:26.946+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:26.957 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:26.958 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:26.958 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:51:26.958 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:51:26.958 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:26.971 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:51:27.006 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:51:27.029 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is unset 2024-09-11T06:51:27.053 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:51:27.053 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^25090\b' 2024-09-11T06:51:27.053 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^25090\b' 2024-09-11T06:51:27.060 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:27.054+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:27.121 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:51:27.122 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -C rbd-nbd 2024-09-11T06:51:27.122 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^ *25090\b' 2024-09-11T06:51:27.122 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^ *25090\b' 2024-09-11T06:51:27.127 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:51:27.128 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 0.5 2024-09-11T06:51:27.144 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:27.144 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:51:27.144 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:51:27.145 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:51:27.145 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.145 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:26 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 140250278007648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:27.145 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:51:27.145 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 56.411900629999998, 2024-09-11T06:51:27.145 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034645140000000001, 2024-09-11T06:51:27.145 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:27.145 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:27.145 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:27.145 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:27.145 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:27.145 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:27.145 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:51:27.145 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.146 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:27.146 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.146 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:27.146 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:51:27.146 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.146 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.146 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.146 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:27.146 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:51:27.146 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.146 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.146 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.146 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:27.147 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729634+0000", 2024-09-11T06:51:27.147 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0470000000000001e-06 2024-09-11T06:51:27.147 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.147 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.147 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:27.147 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729640+0000", 2024-09-11T06:51:27.147 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8860000000000001e-06 2024-09-11T06:51:27.147 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.147 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.147 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:27.147 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729654+0000", 2024-09-11T06:51:27.147 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:51:27.147 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.147 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.148 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:27.148 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729664+0000", 2024-09-11T06:51:27.148 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.594e-06 2024-09-11T06:51:27.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.148 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.148 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:27.148 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730422+0000", 2024-09-11T06:51:27.148 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000758648 2024-09-11T06:51:27.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.148 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.148 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:27.148 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730571+0000", 2024-09-11T06:51:27.148 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000148932 2024-09-11T06:51:27.149 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.149 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.149 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:27.149 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730936+0000", 2024-09-11T06:51:27.149 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036510900000000002 2024-09-11T06:51:27.149 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.149 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.149 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:27.149 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732159+0000", 2024-09-11T06:51:27.149 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001222302 2024-09-11T06:51:27.149 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.149 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.149 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:27.150 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732763+0000", 2024-09-11T06:51:27.150 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00060421599999999998 2024-09-11T06:51:27.150 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.150 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.150 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:27.150 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732916+0000", 2024-09-11T06:51:27.150 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015264 2024-09-11T06:51:27.150 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.150 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.150 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:27.150 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733097+0000", 2024-09-11T06:51:27.150 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000181504 2024-09-11T06:51:27.150 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.150 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:27.151 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.151 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.151 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.151 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:73 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:27.151 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:51:27.151 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 56.410842443999996, 2024-09-11T06:51:27.151 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033454439999999999, 2024-09-11T06:51:27.151 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:27.151 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:27.151 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:27.151 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:27.151 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:51:27.151 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:51:27.151 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73 2024-09-11T06:51:27.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.152 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:27.152 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.152 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:27.152 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:51:27.152 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.152 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.152 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:27.152 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:51:27.152 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.152 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.152 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:27.153 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730694+0000", 2024-09-11T06:51:27.153 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5099999999999999e-06 2024-09-11T06:51:27.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.153 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.153 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:27.153 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730708+0000", 2024-09-11T06:51:27.153 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3237e-05 2024-09-11T06:51:27.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.153 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.153 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:27.153 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730738+0000", 2024-09-11T06:51:27.153 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0688999999999997e-05 2024-09-11T06:51:27.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.154 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.154 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:27.154 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730759+0000", 2024-09-11T06:51:27.154 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0818999999999998e-05 2024-09-11T06:51:27.154 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.154 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.154 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:27.154 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731841+0000", 2024-09-11T06:51:27.154 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010821819999999999 2024-09-11T06:51:27.154 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.154 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.154 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:27.154 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732092+0000", 2024-09-11T06:51:27.154 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025086299999999999 2024-09-11T06:51:27.154 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.155 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.155 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:27.155 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733730+0000", 2024-09-11T06:51:27.155 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0016374600000000001 2024-09-11T06:51:27.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.155 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.155 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:27.155 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733866+0000", 2024-09-11T06:51:27.155 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000136666 2024-09-11T06:51:27.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.155 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.155 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:27.155 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.734036+0000", 2024-09-11T06:51:27.155 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000170018 2024-09-11T06:51:27.155 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.156 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:27.156 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.156 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.156 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.156 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:26 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 139740787512064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:27.156 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:51:27.156 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 56.249216904999997, 2024-09-11T06:51:27.156 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003613733, 2024-09-11T06:51:27.156 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:27.156 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:27.156 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:27.156 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:27.156 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:27.156 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:27.157 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:51:27.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.157 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:27.157 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.157 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:27.157 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:51:27.157 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.157 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.157 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:27.157 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:51:27.157 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.157 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.157 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:27.158 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892318+0000", 2024-09-11T06:51:27.158 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.623e-06 2024-09-11T06:51:27.158 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.158 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.158 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:27.158 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892325+0000", 2024-09-11T06:51:27.158 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:51:27.158 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.158 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.158 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:27.158 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892346+0000", 2024-09-11T06:51:27.158 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1246e-05 2024-09-11T06:51:27.158 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.159 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:27.159 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892362+0000", 2024-09-11T06:51:27.159 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5409000000000001e-05 2024-09-11T06:51:27.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.159 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:27.159 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892883+0000", 2024-09-11T06:51:27.159 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00052087599999999998 2024-09-11T06:51:27.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.159 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:27.159 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893123+0000", 2024-09-11T06:51:27.159 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000240769 2024-09-11T06:51:27.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.160 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.160 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:27.160 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893526+0000", 2024-09-11T06:51:27.160 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040300200000000002 2024-09-11T06:51:27.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.160 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.160 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:27.160 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894820+0000", 2024-09-11T06:51:27.160 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012934159999999999 2024-09-11T06:51:27.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.160 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.160 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:27.160 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895608+0000", 2024-09-11T06:51:27.160 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00078788100000000004 2024-09-11T06:51:27.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.161 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.161 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:27.161 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895724+0000", 2024-09-11T06:51:27.161 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011595999999999999 2024-09-11T06:51:27.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.161 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.161 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:27.161 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895930+0000", 2024-09-11T06:51:27.161 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000206391 2024-09-11T06:51:27.161 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.161 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:27.161 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.162 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.162 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:133 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:27.162 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:51:27.162 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 56.248418888000003, 2024-09-11T06:51:27.162 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035017360000000001, 2024-09-11T06:51:27.162 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:27.162 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:27.162 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:27.162 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:27.162 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:51:27.162 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:51:27.162 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 133 2024-09-11T06:51:27.162 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.162 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:27.162 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.163 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:27.163 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:51:27.163 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.163 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.163 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.163 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:27.163 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:51:27.163 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.163 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.163 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.163 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:27.163 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893116+0000", 2024-09-11T06:51:27.163 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1629999999999999e-06 2024-09-11T06:51:27.164 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.164 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.164 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:27.164 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893128+0000", 2024-09-11T06:51:27.164 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.11e-05 2024-09-11T06:51:27.164 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.164 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.164 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:27.164 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893156+0000", 2024-09-11T06:51:27.164 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8266e-05 2024-09-11T06:51:27.164 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.164 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.164 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:27.164 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893171+0000", 2024-09-11T06:51:27.165 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5426000000000001e-05 2024-09-11T06:51:27.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.165 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.165 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:27.165 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894561+0000", 2024-09-11T06:51:27.165 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013899959999999999 2024-09-11T06:51:27.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.165 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.165 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:27.165 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894781+0000", 2024-09-11T06:51:27.165 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021987100000000001 2024-09-11T06:51:27.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.165 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.165 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:27.166 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896349+0000", 2024-09-11T06:51:27.166 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015681779999999999 2024-09-11T06:51:27.166 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.166 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.166 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:27.166 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896458+0000", 2024-09-11T06:51:27.166 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000108187 2024-09-11T06:51:27.166 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.168 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.168 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:27.168 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896616+0000", 2024-09-11T06:51:27.168 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015854899999999999 2024-09-11T06:51:27.168 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.168 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:27.168 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.168 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.168 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:176 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch reconnect cookie 139735284585312 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:27.168 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:51:27.169 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 56.217040689000001, 2024-09-11T06:51:27.169 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033620960000000002, 2024-09-11T06:51:27.169 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:27.169 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:27.169 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:27.169 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:27.169 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:51:27.169 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:51:27.169 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 176 2024-09-11T06:51:27.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.169 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:27.169 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.169 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:27.169 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:51:27.170 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.170 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.170 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:27.170 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:51:27.170 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.170 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.170 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:27.170 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924494+0000", 2024-09-11T06:51:27.170 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.967e-06 2024-09-11T06:51:27.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.170 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.170 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:27.171 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924501+0000", 2024-09-11T06:51:27.171 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2659999999999998e-06 2024-09-11T06:51:27.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.171 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.171 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:27.171 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924520+0000", 2024-09-11T06:51:27.171 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9038999999999998e-05 2024-09-11T06:51:27.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.171 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.172 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:27.172 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924535+0000", 2024-09-11T06:51:27.172 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5359999999999999e-05 2024-09-11T06:51:27.172 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.172 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.172 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:27.172 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.925544+0000", 2024-09-11T06:51:27.172 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010085109999999999 2024-09-11T06:51:27.172 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.172 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.173 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:27.173 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.925671+0000", 2024-09-11T06:51:27.173 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012756599999999999 2024-09-11T06:51:27.173 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.173 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.173 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:27.173 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.927855+0000", 2024-09-11T06:51:27.173 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0021833870000000002 2024-09-11T06:51:27.173 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.173 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:27.174 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.174 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.174 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4828.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch watch cookie 140587969811264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-11T06:51:27.174 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:51:27.174 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 50.895337730999998, 2024-09-11T06:51:27.174 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032778859999999998, 2024-09-11T06:51:27.174 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:27.174 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:27.175 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:27.175 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:27.175 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4828", 2024-09-11T06:51:27.175 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2410288650", 2024-09-11T06:51:27.175 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:51:27.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.175 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:27.175 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.175 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:27.176 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:51:27.176 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.176 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.176 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:27.176 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:51:27.176 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.177 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.177 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:27.177 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246198+0000", 2024-09-11T06:51:27.177 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.667e-06 2024-09-11T06:51:27.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.177 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.177 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:27.177 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246209+0000", 2024-09-11T06:51:27.177 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0409e-05 2024-09-11T06:51:27.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.178 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:27.178 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246241+0000", 2024-09-11T06:51:27.178 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2592999999999998e-05 2024-09-11T06:51:27.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.178 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:27.179 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246260+0000", 2024-09-11T06:51:27.179 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9343000000000001e-05 2024-09-11T06:51:27.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.179 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.179 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:27.179 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246466+0000", 2024-09-11T06:51:27.180 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000205701 2024-09-11T06:51:27.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.180 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.180 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:27.180 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246713+0000", 2024-09-11T06:51:27.180 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024646999999999999 2024-09-11T06:51:27.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.181 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.181 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:27.181 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.247133+0000", 2024-09-11T06:51:27.181 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000420536 2024-09-11T06:51:27.181 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.181 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.181 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:27.181 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.248541+0000", 2024-09-11T06:51:27.181 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001407801 2024-09-11T06:51:27.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.182 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:27.182 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249241+0000", 2024-09-11T06:51:27.182 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070043500000000001 2024-09-11T06:51:27.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.182 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:27.182 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249326+0000", 2024-09-11T06:51:27.182 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5037999999999995e-05 2024-09-11T06:51:27.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.183 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:27.183 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249473+0000", 2024-09-11T06:51:27.183 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014689299999999999 2024-09-11T06:51:27.183 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.183 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:27.183 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.183 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.183 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.183 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5328.0:14 2.1 2:967e1786:ns::rbd_object_map.11adc335555f.0000000000000004:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:51:27.183 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.396257+0000", 2024-09-11T06:51:27.183 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 47.745276283000003, 2024-09-11T06:51:27.184 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032125370000000001, 2024-09-11T06:51:27.184 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:27.184 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:27.184 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:27.184 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:27.184 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5328", 2024-09-11T06:51:27.184 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2386493698", 2024-09-11T06:51:27.184 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 14 2024-09-11T06:51:27.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.184 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:27.184 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.184 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:27.185 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396257+0000", 2024-09-11T06:51:27.185 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.185 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.185 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:27.185 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396257+0000", 2024-09-11T06:51:27.185 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.185 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.185 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:27.185 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396259+0000", 2024-09-11T06:51:27.185 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4830000000000002e-06 2024-09-11T06:51:27.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.186 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:27.186 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396271+0000", 2024-09-11T06:51:27.186 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.115e-05 2024-09-11T06:51:27.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.186 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:27.186 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396292+0000", 2024-09-11T06:51:27.186 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1761999999999999e-05 2024-09-11T06:51:27.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.186 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:27.186 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396314+0000", 2024-09-11T06:51:27.187 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1982999999999999e-05 2024-09-11T06:51:27.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.187 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.187 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:27.187 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396409+0000", 2024-09-11T06:51:27.187 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4250000000000001e-05 2024-09-11T06:51:27.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.187 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.187 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:27.187 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.396464+0000", 2024-09-11T06:51:27.187 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5884000000000002e-05 2024-09-11T06:51:27.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.188 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.188 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:27.188 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.399469+0000", 2024-09-11T06:51:27.188 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0030050250000000001 2024-09-11T06:51:27.188 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.188 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:27.188 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.188 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.188 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.188 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:6 3.5 3:b2cc3336:::rbd_id.testrbdnbd21607:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:27.188 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.591899+0000", 2024-09-11T06:51:27.188 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.549633782000001, 2024-09-11T06:51:27.189 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032382999999999999, 2024-09-11T06:51:27.189 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:27.189 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:27.189 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:27.189 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:27.189 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:51:27.189 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:51:27.189 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 6 2024-09-11T06:51:27.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.189 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:27.189 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.189 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:27.189 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591899+0000", 2024-09-11T06:51:27.190 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.190 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.190 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:27.190 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591899+0000", 2024-09-11T06:51:27.190 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.190 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.190 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:27.190 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591902+0000", 2024-09-11T06:51:27.190 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0770000000000002e-06 2024-09-11T06:51:27.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.191 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:27.191 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591913+0000", 2024-09-11T06:51:27.191 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1182999999999999e-05 2024-09-11T06:51:27.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.191 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:27.191 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591940+0000", 2024-09-11T06:51:27.191 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7233000000000002e-05 2024-09-11T06:51:27.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.192 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:27.192 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591954+0000", 2024-09-11T06:51:27.192 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4503e-05 2024-09-11T06:51:27.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.192 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:27.192 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.592071+0000", 2024-09-11T06:51:27.192 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116731 2024-09-11T06:51:27.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.192 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:27.192 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.592274+0000", 2024-09-11T06:51:27.193 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00020320900000000001 2024-09-11T06:51:27.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.193 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.193 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 3", 2024-09-11T06:51:27.193 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.592576+0000", 2024-09-11T06:51:27.193 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030179000000000002 2024-09-11T06:51:27.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.193 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.193 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:27.193 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.594072+0000", 2024-09-11T06:51:27.193 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001495496 2024-09-11T06:51:27.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.194 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.194 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:27.194 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.594945+0000", 2024-09-11T06:51:27.194 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00087379799999999996 2024-09-11T06:51:27.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.194 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.194 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:27.194 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.595023+0000", 2024-09-11T06:51:27.194 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7751999999999997e-05 2024-09-11T06:51:27.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.194 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.195 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:27.195 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.595138+0000", 2024-09-11T06:51:27.195 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000114528 2024-09-11T06:51:27.195 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.195 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:27.195 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.195 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.195 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.195 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:44 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:27.195 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.540846+0000", 2024-09-11T06:51:27.195 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 38.600687649999998, 2024-09-11T06:51:27.196 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032138639999999999, 2024-09-11T06:51:27.196 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:27.196 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:27.196 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:27.196 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:27.196 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:27.196 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:27.196 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 44 2024-09-11T06:51:27.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.196 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:27.196 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.196 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:27.197 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540846+0000", 2024-09-11T06:51:27.197 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.197 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:27.197 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540846+0000", 2024-09-11T06:51:27.197 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.197 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:27.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540848+0000", 2024-09-11T06:51:27.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5469999999999999e-06 2024-09-11T06:51:27.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:27.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540863+0000", 2024-09-11T06:51:27.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4613000000000001e-05 2024-09-11T06:51:27.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:27.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540895+0000", 2024-09-11T06:51:27.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2153000000000002e-05 2024-09-11T06:51:27.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.199 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:27.199 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540912+0000", 2024-09-11T06:51:27.199 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7405999999999999e-05 2024-09-11T06:51:27.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.199 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:27.199 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.541111+0000", 2024-09-11T06:51:27.199 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019849599999999999 2024-09-11T06:51:27.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.200 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:27.200 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.541269+0000", 2024-09-11T06:51:27.200 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015792399999999999 2024-09-11T06:51:27.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.200 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.200 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:27.200 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.541853+0000", 2024-09-11T06:51:27.200 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00058443799999999995 2024-09-11T06:51:27.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.200 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.200 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:27.201 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.543133+0000", 2024-09-11T06:51:27.201 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012797850000000001 2024-09-11T06:51:27.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.201 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.201 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:27.201 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.543905+0000", 2024-09-11T06:51:27.201 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000771655 2024-09-11T06:51:27.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.201 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.201 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:27.201 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.543993+0000", 2024-09-11T06:51:27.202 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8168e-05 2024-09-11T06:51:27.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.202 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.202 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:27.202 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.544059+0000", 2024-09-11T06:51:27.202 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6679000000000003e-05 2024-09-11T06:51:27.202 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.202 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:27.202 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.202 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.202 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5553.0:24 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 139650593197904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:27.203 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.566567+0000", 2024-09-11T06:51:27.203 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 38.574966379999999, 2024-09-11T06:51:27.203 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034350790000000002, 2024-09-11T06:51:27.203 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:27.203 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:27.203 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:27.203 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:27.203 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5553", 2024-09-11T06:51:27.203 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3590511604", 2024-09-11T06:51:27.203 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 24 2024-09-11T06:51:27.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.204 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:27.204 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.204 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:27.204 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566567+0000", 2024-09-11T06:51:27.204 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.204 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.204 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:27.204 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566567+0000", 2024-09-11T06:51:27.204 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.205 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:27.205 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566569+0000", 2024-09-11T06:51:27.205 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.973e-06 2024-09-11T06:51:27.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.205 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:27.205 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566578+0000", 2024-09-11T06:51:27.205 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8170000000000005e-06 2024-09-11T06:51:27.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.205 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:27.206 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566602+0000", 2024-09-11T06:51:27.206 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4596e-05 2024-09-11T06:51:27.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.206 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.206 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:27.206 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566619+0000", 2024-09-11T06:51:27.206 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7099999999999999e-05 2024-09-11T06:51:27.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.206 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.206 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:27.206 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.567321+0000", 2024-09-11T06:51:27.207 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070168200000000002 2024-09-11T06:51:27.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.207 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:27.207 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.567454+0000", 2024-09-11T06:51:27.207 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013286400000000001 2024-09-11T06:51:27.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.207 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:27.207 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.567674+0000", 2024-09-11T06:51:27.207 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000219679 2024-09-11T06:51:27.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.208 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.208 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:27.208 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.569014+0000", 2024-09-11T06:51:27.208 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013404129999999999 2024-09-11T06:51:27.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.208 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.208 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:27.208 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.569634+0000", 2024-09-11T06:51:27.208 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00062029200000000004 2024-09-11T06:51:27.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.208 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:27.209 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.569805+0000", 2024-09-11T06:51:27.209 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000170366 2024-09-11T06:51:27.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:27.209 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.570002+0000", 2024-09-11T06:51:27.209 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019729700000000001 2024-09-11T06:51:27.209 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.209 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:27.210 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.210 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.210 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:82 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch reconnect cookie 139888024456048 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:27.210 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.690185+0000", 2024-09-11T06:51:27.210 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 38.451348377999999, 2024-09-11T06:51:27.210 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0045884799999999998, 2024-09-11T06:51:27.210 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:27.210 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:27.211 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:27.211 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:27.211 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:27.211 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:27.211 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 82 2024-09-11T06:51:27.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.211 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:27.211 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.211 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:27.211 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690185+0000", 2024-09-11T06:51:27.211 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.212 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:27.212 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690185+0000", 2024-09-11T06:51:27.212 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.212 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:27.212 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690186+0000", 2024-09-11T06:51:27.212 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8400000000000002e-07 2024-09-11T06:51:27.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.212 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:27.213 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690190+0000", 2024-09-11T06:51:27.213 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5499999999999996e-06 2024-09-11T06:51:27.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.213 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:27.213 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690203+0000", 2024-09-11T06:51:27.213 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2228999999999999e-05 2024-09-11T06:51:27.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.213 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:27.213 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690228+0000", 2024-09-11T06:51:27.214 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5587000000000001e-05 2024-09-11T06:51:27.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.214 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.214 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:27.214 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.691298+0000", 2024-09-11T06:51:27.214 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001070218 2024-09-11T06:51:27.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.214 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.214 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:27.214 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.691511+0000", 2024-09-11T06:51:27.214 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021301299999999999 2024-09-11T06:51:27.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.215 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.215 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:27.215 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.694773+0000", 2024-09-11T06:51:27.215 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003261899 2024-09-11T06:51:27.215 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.215 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:27.215 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.215 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.215 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.215 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:84 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify cookie 139887823033168] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:27.215 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:51:27.216 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 38.451256602000001, 2024-09-11T06:51:27.216 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035622620000000001, 2024-09-11T06:51:27.216 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:27.216 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:27.216 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:27.216 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:27.216 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:27.216 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:27.216 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 84 2024-09-11T06:51:27.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.216 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:27.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.216 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:27.217 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:51:27.217 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.217 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.217 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.217 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:27.217 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:51:27.217 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.217 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.217 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.217 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:27.217 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:51:27.218 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5300000000000003e-07 2024-09-11T06:51:27.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.218 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:27.218 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690281+0000", 2024-09-11T06:51:27.218 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9330000000000003e-06 2024-09-11T06:51:27.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.218 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:27.218 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690291+0000", 2024-09-11T06:51:27.218 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7359999999999996e-06 2024-09-11T06:51:27.219 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.219 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.219 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:27.219 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690298+0000", 2024-09-11T06:51:27.219 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3699999999999997e-06 2024-09-11T06:51:27.219 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.219 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.219 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:27.219 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693380+0000", 2024-09-11T06:51:27.219 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0030811060000000001 2024-09-11T06:51:27.219 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.219 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.219 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:27.220 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693599+0000", 2024-09-11T06:51:27.220 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021947199999999999 2024-09-11T06:51:27.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.220 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.220 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:27.220 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693839+0000", 2024-09-11T06:51:27.220 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023989200000000001 2024-09-11T06:51:27.220 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.220 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:27.220 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.220 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.221 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5098.0:25 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 140617967560800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:27.221 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.697727+0000", 2024-09-11T06:51:27.221 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 38.443806025999997, 2024-09-11T06:51:27.221 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035341330000000001, 2024-09-11T06:51:27.221 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:27.221 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:27.221 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:27.221 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:27.221 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5098", 2024-09-11T06:51:27.221 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/844541331", 2024-09-11T06:51:27.221 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 25 2024-09-11T06:51:27.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.222 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:27.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.222 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:27.222 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697727+0000", 2024-09-11T06:51:27.222 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.222 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:27.222 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697727+0000", 2024-09-11T06:51:27.222 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.223 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.223 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:27.223 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697729+0000", 2024-09-11T06:51:27.223 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.33e-06 2024-09-11T06:51:27.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.223 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.223 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:27.223 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697734+0000", 2024-09-11T06:51:27.223 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6590000000000004e-06 2024-09-11T06:51:27.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.224 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.224 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:27.224 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697749+0000", 2024-09-11T06:51:27.224 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4867e-05 2024-09-11T06:51:27.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.224 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.224 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:27.224 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697759+0000", 2024-09-11T06:51:27.224 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0073e-05 2024-09-11T06:51:27.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.225 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:27.225 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.698715+0000", 2024-09-11T06:51:27.225 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00095545799999999996 2024-09-11T06:51:27.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.225 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:27.225 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.698850+0000", 2024-09-11T06:51:27.225 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000135814 2024-09-11T06:51:27.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.226 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:27.226 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.699085+0000", 2024-09-11T06:51:27.226 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023507199999999999 2024-09-11T06:51:27.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.226 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:27.226 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.700258+0000", 2024-09-11T06:51:27.226 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011726600000000001 2024-09-11T06:51:27.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.227 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.227 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:27.227 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.701061+0000", 2024-09-11T06:51:27.227 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00080256100000000003 2024-09-11T06:51:27.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.227 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.227 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:27.227 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.701144+0000", 2024-09-11T06:51:27.227 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3188000000000004e-05 2024-09-11T06:51:27.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.228 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.228 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:27.228 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.701261+0000", 2024-09-11T06:51:27.228 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000117451 2024-09-11T06:51:27.228 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.228 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:27.228 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.228 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.228 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.229 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5380.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b,call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b,call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b,call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:27.229 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:12.536478+0000", 2024-09-11T06:51:27.229 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.605054845, 2024-09-11T06:51:27.229 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032220090000000001, 2024-09-11T06:51:27.229 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:27.229 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:27.229 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:27.229 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:27.229 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5380", 2024-09-11T06:51:27.229 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/290149879", 2024-09-11T06:51:27.230 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:51:27.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.230 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:27.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.230 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:27.230 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536478+0000", 2024-09-11T06:51:27.230 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.230 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:27.231 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536478+0000", 2024-09-11T06:51:27.231 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.231 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.231 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:27.232 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536481+0000", 2024-09-11T06:51:27.232 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4059999999999999e-06 2024-09-11T06:51:27.232 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.232 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.232 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:27.232 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536493+0000", 2024-09-11T06:51:27.232 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2003e-05 2024-09-11T06:51:27.232 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.232 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.233 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:27.233 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536526+0000", 2024-09-11T06:51:27.233 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2839999999999997e-05 2024-09-11T06:51:27.233 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.233 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.233 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:27.233 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536542+0000", 2024-09-11T06:51:27.233 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6120000000000002e-05 2024-09-11T06:51:27.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.234 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:27.234 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536662+0000", 2024-09-11T06:51:27.234 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000120285 2024-09-11T06:51:27.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.234 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:27.234 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536946+0000", 2024-09-11T06:51:27.235 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00028412400000000002 2024-09-11T06:51:27.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.235 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:27.235 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.539700+0000", 2024-09-11T06:51:27.235 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027542310000000002 2024-09-11T06:51:27.235 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.235 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:27.236 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.236 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.236 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.6105.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b,call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b,call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b,call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-11T06:51:27.236 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:20.630296+0000", 2024-09-11T06:51:27.236 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.5112371009999999, 2024-09-11T06:51:27.236 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003874653, 2024-09-11T06:51:27.236 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:27.236 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:27.237 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:27.237 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:27.237 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.6105", 2024-09-11T06:51:27.237 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/722227396", 2024-09-11T06:51:27.237 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:51:27.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.237 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:27.237 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.237 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:27.237 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.630296+0000", 2024-09-11T06:51:27.237 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.238 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.238 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:27.238 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.630296+0000", 2024-09-11T06:51:27.238 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.238 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.238 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:27.238 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.630298+0000", 2024-09-11T06:51:27.238 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2500000000000001e-06 2024-09-11T06:51:27.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.239 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.239 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:27.239 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.630317+0000", 2024-09-11T06:51:27.239 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9043e-05 2024-09-11T06:51:27.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.239 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.239 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:27.239 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.630349+0000", 2024-09-11T06:51:27.239 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1890000000000001e-05 2024-09-11T06:51:27.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.239 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.240 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:27.240 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.630368+0000", 2024-09-11T06:51:27.240 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8533e-05 2024-09-11T06:51:27.240 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.240 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.240 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:27.240 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.630500+0000", 2024-09-11T06:51:27.240 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013175900000000001 2024-09-11T06:51:27.240 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.240 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.240 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:27.241 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.630806+0000", 2024-09-11T06:51:27.241 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030631700000000002 2024-09-11T06:51:27.241 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.241 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.241 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:27.241 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.634171+0000", 2024-09-11T06:51:27.241 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033648609999999998 2024-09-11T06:51:27.241 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.241 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:27.241 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.242 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.242 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.242 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.6105.0:20 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.metadata_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-11T06:51:27.242 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:20.648867+0000", 2024-09-11T06:51:27.242 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.4926660890000001, 2024-09-11T06:51:27.242 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035862189999999999, 2024-09-11T06:51:27.242 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:27.242 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:27.242 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:27.242 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:27.242 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.6105", 2024-09-11T06:51:27.243 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/722227396", 2024-09-11T06:51:27.243 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 20 2024-09-11T06:51:27.243 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.243 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:27.243 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.243 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:27.243 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.648867+0000", 2024-09-11T06:51:27.243 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.243 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.243 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.243 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:27.243 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.648867+0000", 2024-09-11T06:51:27.243 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.244 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.244 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.244 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:27.244 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.648868+0000", 2024-09-11T06:51:27.244 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5000000000000001e-07 2024-09-11T06:51:27.244 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.244 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.244 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:27.244 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.648873+0000", 2024-09-11T06:51:27.244 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2970000000000003e-06 2024-09-11T06:51:27.244 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.244 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.244 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:27.245 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.648889+0000", 2024-09-11T06:51:27.245 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5549999999999999e-05 2024-09-11T06:51:27.245 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.245 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.245 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:27.245 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.648899+0000", 2024-09-11T06:51:27.245 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0516000000000001e-05 2024-09-11T06:51:27.245 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.245 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.245 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:27.245 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.649514+0000", 2024-09-11T06:51:27.245 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00061461800000000004 2024-09-11T06:51:27.246 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.246 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.246 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:27.246 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.649816+0000", 2024-09-11T06:51:27.246 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030191700000000002 2024-09-11T06:51:27.246 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.246 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.246 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:27.246 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.650295+0000", 2024-09-11T06:51:27.246 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00047892799999999999 2024-09-11T06:51:27.246 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.247 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.247 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:27.247 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.651725+0000", 2024-09-11T06:51:27.247 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014304630000000001 2024-09-11T06:51:27.247 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.247 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.247 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:27.247 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.652187+0000", 2024-09-11T06:51:27.247 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00046161299999999999 2024-09-11T06:51:27.247 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.247 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.247 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:27.248 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.652279+0000", 2024-09-11T06:51:27.248 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2025000000000001e-05 2024-09-11T06:51:27.248 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.248 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.248 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:27.248 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.652453+0000", 2024-09-11T06:51:27.248 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017434199999999999 2024-09-11T06:51:27.248 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.248 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:27.248 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.248 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.248 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.249 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.6105.0:21 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-11T06:51:27.249 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:20.649631+0000", 2024-09-11T06:51:27.249 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.4919017459999999, 2024-09-11T06:51:27.249 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034058339999999999, 2024-09-11T06:51:27.249 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:27.249 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:27.249 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:27.249 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:27.249 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.6105", 2024-09-11T06:51:27.249 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/722227396", 2024-09-11T06:51:27.249 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 21 2024-09-11T06:51:27.249 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.250 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:27.250 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.250 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:27.250 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.649631+0000", 2024-09-11T06:51:27.250 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.250 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.250 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.250 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:27.250 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.649631+0000", 2024-09-11T06:51:27.250 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.250 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.250 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.251 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:27.251 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.649634+0000", 2024-09-11T06:51:27.251 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.04e-06 2024-09-11T06:51:27.251 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.251 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.251 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:27.251 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.649644+0000", 2024-09-11T06:51:27.251 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.061e-05 2024-09-11T06:51:27.251 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.251 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.251 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:27.251 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.649671+0000", 2024-09-11T06:51:27.252 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6679e-05 2024-09-11T06:51:27.252 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.252 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.252 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:27.252 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.649690+0000", 2024-09-11T06:51:27.252 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9573999999999999e-05 2024-09-11T06:51:27.252 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.252 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.252 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:27.252 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.651104+0000", 2024-09-11T06:51:27.253 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001413739 2024-09-11T06:51:27.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.253 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:27.253 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.651336+0000", 2024-09-11T06:51:27.253 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023153799999999999 2024-09-11T06:51:27.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.253 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:27.253 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.652555+0000", 2024-09-11T06:51:27.253 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001219765 2024-09-11T06:51:27.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.254 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.254 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:27.254 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.652705+0000", 2024-09-11T06:51:27.254 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000149445 2024-09-11T06:51:27.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.254 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.254 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:27.254 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.653037+0000", 2024-09-11T06:51:27.254 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033244399999999999 2024-09-11T06:51:27.254 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.254 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:27.254 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.255 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.255 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.255 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.6105.0:33 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 140610048726272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-11T06:51:27.255 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:20.671124+0000", 2024-09-11T06:51:27.255 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.4704096980000001, 2024-09-11T06:51:27.255 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035064290000000001, 2024-09-11T06:51:27.255 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:27.255 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:27.255 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:27.255 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:27.255 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.6105", 2024-09-11T06:51:27.255 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/722227396", 2024-09-11T06:51:27.255 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 33 2024-09-11T06:51:27.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.256 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:27.256 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.256 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:27.256 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.671124+0000", 2024-09-11T06:51:27.256 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.256 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.256 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:27.256 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.671124+0000", 2024-09-11T06:51:27.256 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.257 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.257 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:27.257 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.671124+0000", 2024-09-11T06:51:27.257 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7600000000000006e-07 2024-09-11T06:51:27.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.257 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.257 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:27.257 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.671129+0000", 2024-09-11T06:51:27.257 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7330000000000003e-06 2024-09-11T06:51:27.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.257 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.258 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:27.258 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.671145+0000", 2024-09-11T06:51:27.258 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5367e-05 2024-09-11T06:51:27.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.258 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:27.258 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.671154+0000", 2024-09-11T06:51:27.258 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.363e-06 2024-09-11T06:51:27.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.258 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:27.259 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.671797+0000", 2024-09-11T06:51:27.259 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00064351000000000002 2024-09-11T06:51:27.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.259 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.259 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:27.259 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.671943+0000", 2024-09-11T06:51:27.259 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000145699 2024-09-11T06:51:27.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.259 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.259 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:27.259 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.672240+0000", 2024-09-11T06:51:27.260 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00029651100000000001 2024-09-11T06:51:27.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.260 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.260 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:27.260 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.673194+0000", 2024-09-11T06:51:27.260 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00095448699999999996 2024-09-11T06:51:27.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.260 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.260 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:27.260 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.674221+0000", 2024-09-11T06:51:27.261 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001026714 2024-09-11T06:51:27.261 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.261 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.261 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:27.261 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.674327+0000", 2024-09-11T06:51:27.261 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000105732 2024-09-11T06:51:27.261 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.261 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.261 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:27.261 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.674630+0000", 2024-09-11T06:51:27.261 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030333699999999999 2024-09-11T06:51:27.262 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.262 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:27.262 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.262 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.262 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5542.0:3 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-11T06:51:27.262 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:20.994530+0000", 2024-09-11T06:51:27.262 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.14700325, 2024-09-11T06:51:27.262 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.99603794099999998, 2024-09-11T06:51:27.262 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:27.262 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:27.263 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:27.263 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:27.263 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5542", 2024-09-11T06:51:27.263 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1546942093", 2024-09-11T06:51:27.263 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 3 2024-09-11T06:51:27.263 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.263 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:27.263 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.263 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:27.263 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.994530+0000", 2024-09-11T06:51:27.263 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.263 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.263 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.264 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:27.264 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.994530+0000", 2024-09-11T06:51:27.264 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.264 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:27.264 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.994532+0000", 2024-09-11T06:51:27.264 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9740000000000001e-06 2024-09-11T06:51:27.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.264 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:27.264 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.994543+0000", 2024-09-11T06:51:27.265 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1066999999999999e-05 2024-09-11T06:51:27.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.265 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:27.265 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.994558+0000", 2024-09-11T06:51:27.265 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4987999999999999e-05 2024-09-11T06:51:27.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.265 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:27.265 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.994572+0000", 2024-09-11T06:51:27.265 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3974e-05 2024-09-11T06:51:27.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.266 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.266 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:27.266 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.994659+0000", 2024-09-11T06:51:27.266 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7491000000000006e-05 2024-09-11T06:51:27.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.266 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.266 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-11T06:51:27.266 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.994675+0000", 2024-09-11T06:51:27.266 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.506e-05 2024-09-11T06:51:27.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.266 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.266 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:27.267 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:21.989532+0000", 2024-09-11T06:51:27.267 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.99485760400000001 2024-09-11T06:51:27.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.267 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.267 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:27.267 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:21.989833+0000", 2024-09-11T06:51:27.267 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030123699999999999 2024-09-11T06:51:27.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.267 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.267 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:27.268 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:21.990568+0000", 2024-09-11T06:51:27.268 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00073454599999999998 2024-09-11T06:51:27.268 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.268 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:27.268 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.268 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5542.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b,call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b,call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b,call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-11T06:51:27.268 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:22.009725+0000", 2024-09-11T06:51:27.268 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.1318083090000002, 2024-09-11T06:51:27.269 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033114450000000001, 2024-09-11T06:51:27.269 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:27.269 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:27.269 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:27.269 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:27.269 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5542", 2024-09-11T06:51:27.269 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1546942093", 2024-09-11T06:51:27.269 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:51:27.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.269 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:27.269 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.269 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:27.270 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:22.009725+0000", 2024-09-11T06:51:27.270 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.270 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.270 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.270 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:27.270 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:22.009725+0000", 2024-09-11T06:51:27.270 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:27.270 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.270 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.270 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:27.270 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:22.009727+0000", 2024-09-11T06:51:27.271 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.413e-06 2024-09-11T06:51:27.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.271 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:27.271 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:22.009740+0000", 2024-09-11T06:51:27.271 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2313000000000001e-05 2024-09-11T06:51:27.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.271 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:27.271 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:22.009771+0000", 2024-09-11T06:51:27.271 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1529999999999998e-05 2024-09-11T06:51:27.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.272 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.272 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:27.272 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:22.009787+0000", 2024-09-11T06:51:27.272 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6303999999999998e-05 2024-09-11T06:51:27.272 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.272 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.272 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:27.272 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:22.009913+0000", 2024-09-11T06:51:27.272 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000125115 2024-09-11T06:51:27.272 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.272 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.273 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:27.273 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:22.010200+0000", 2024-09-11T06:51:27.273 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00028791400000000001 2024-09-11T06:51:27.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:27.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:27.273 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:27.273 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:22.013036+0000", 2024-09-11T06:51:27.273 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028358559999999999 2024-09-11T06:51:27.273 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.273 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:27.273 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.273 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:27.274 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:27.274 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:27.275 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:51:27.276 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:51:27.276 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:51:27.277 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:51:27.277 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:51:27.277 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:51:27.277 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:51:27.278 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:51:27.278 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:51:27.278 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:51:27.279 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:51:27.279 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:51:27.280 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:51:27.280 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:27.269+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:27.376 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:27.370+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:27.410 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:27.410 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:27.425 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:51:27.477 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:27.469+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:27.550 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:27.550 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:27.564 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:51:27.605 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:27.601+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:27.629 INFO:tasks.workunit.client.0.smithi176.stderr:+ EXPECTED=7 2024-09-11T06:51:27.630 INFO:tasks.workunit.client.0.smithi176.stderr:++ _sudo rbd device --device-type nbd --rbd-op-threads 7 map rbd/testrbdnbd21607 2024-09-11T06:51:27.630 INFO:tasks.workunit.client.0.smithi176.stderr:++ local cmd 2024-09-11T06:51:27.630 INFO:tasks.workunit.client.0.smithi176.stderr:+++ id -u 2024-09-11T06:51:27.633 INFO:tasks.workunit.client.0.smithi176.stderr:++ '[' 1000 -eq 0 ']' 2024-09-11T06:51:27.633 INFO:tasks.workunit.client.0.smithi176.stderr:+++ which rbd 2024-09-11T06:51:27.635 INFO:tasks.workunit.client.0.smithi176.stderr:++ cmd=/usr/bin/rbd 2024-09-11T06:51:27.635 INFO:tasks.workunit.client.0.smithi176.stderr:++ shift 2024-09-11T06:51:27.635 INFO:tasks.workunit.client.0.smithi176.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd --rbd-op-threads 7 map rbd/testrbdnbd21607 2024-09-11T06:51:27.678 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:27.674+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:27.726 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:27.726 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:27.741 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:51:27.780 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV=/dev/nbd2 2024-09-11T06:51:27.780 INFO:tasks.workunit.client.0.smithi176.stderr:+ get_pid rbd 2024-09-11T06:51:27.780 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pool=rbd 2024-09-11T06:51:27.781 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns= 2024-09-11T06:51:27.781 INFO:tasks.workunit.client.0.smithi176.stderr:++ rbd device --device-type nbd --format xml list 2024-09-11T06:51:27.781 INFO:tasks.workunit.client.0.smithi176.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21607'\''][device='\''/dev/nbd2'\'']/id' 2024-09-11T06:51:27.793 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:27.789+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:27.862 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:27.862 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:27.875 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:51:27.889 INFO:tasks.workunit.client.0.smithi176.stderr:+ PID=25155 2024-09-11T06:51:27.889 INFO:tasks.workunit.client.0.smithi176.stderr:+ test -n 25155 2024-09-11T06:51:27.889 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -p 25155 -C rbd-nbd 2024-09-11T06:51:27.895 INFO:tasks.workunit.client.0.smithi176.stdout: PID TTY TIME CMD 2024-09-11T06:51:27.895 INFO:tasks.workunit.client.0.smithi176.stdout: 25155 ? 00:00:00 rbd-nbd 2024-09-11T06:51:27.896 INFO:tasks.workunit.client.0.smithi176.stderr:++ ps -p 25155 -T 2024-09-11T06:51:27.896 INFO:tasks.workunit.client.0.smithi176.stderr:++ grep -c io_context_pool 2024-09-11T06:51:27.904 INFO:tasks.workunit.client.0.smithi176.stderr:+ ACTUAL=7 2024-09-11T06:51:27.905 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 7 -eq 7 ']' 2024-09-11T06:51:27.905 INFO:tasks.workunit.client.0.smithi176.stderr:+ unmap_device /dev/nbd2 25155 2024-09-11T06:51:27.905 INFO:tasks.workunit.client.0.smithi176.stderr:+ local args=/dev/nbd2 2024-09-11T06:51:27.905 INFO:tasks.workunit.client.0.smithi176.stderr:+ local pid=25155 2024-09-11T06:51:27.905 INFO:tasks.workunit.client.0.smithi176.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd2 2024-09-11T06:51:27.905 INFO:tasks.workunit.client.0.smithi176.stderr:+ local cmd 2024-09-11T06:51:27.905 INFO:tasks.workunit.client.0.smithi176.stderr:++ id -u 2024-09-11T06:51:27.905 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' 1000 -eq 0 ']' 2024-09-11T06:51:27.906 INFO:tasks.workunit.client.0.smithi176.stderr:++ which rbd 2024-09-11T06:51:27.907 INFO:tasks.workunit.client.0.smithi176.stderr:+ cmd=/usr/bin/rbd 2024-09-11T06:51:27.907 INFO:tasks.workunit.client.0.smithi176.stderr:+ shift 2024-09-11T06:51:27.907 INFO:tasks.workunit.client.0.smithi176.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd2 2024-09-11T06:51:27.917 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:27.913+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:27.981 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:27.977+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:28.024 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^25155\b' 2024-09-11T06:51:28.024 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd device --device-type nbd list 2024-09-11T06:51:28.024 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^25155\b' 2024-09-11T06:51:28.042 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:28.042 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:28.057 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:51:28.082 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:28.078+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:28.089 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:51:28.090 INFO:tasks.workunit.client.0.smithi176.stderr:+ ps -C rbd-nbd 2024-09-11T06:51:28.090 INFO:tasks.workunit.client.0.smithi176.stderr:+ expect_false grep '^ *25155\b' 2024-09-11T06:51:28.090 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep '^ *25155\b' 2024-09-11T06:51:28.096 INFO:tasks.workunit.client.0.smithi176.stderr:+ return 0 2024-09-11T06:51:28.096 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 0.5 2024-09-11T06:51:28.183 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:28.182+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:28.225 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:28.226 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:28.242 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:51:28.284 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:28.278+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:28.363 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:28.364 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:28.378 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:51:28.385 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:28.382+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:28.486 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:28.481+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:28.496 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:28.497 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:28.510 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:51:28.587 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:28.582+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:28.597 INFO:tasks.workunit.client.0.smithi176.stderr:+ DEV= 2024-09-11T06:51:28.597 INFO:tasks.workunit.client.0.smithi176.stderr:+ echo OK 2024-09-11T06:51:28.597 INFO:tasks.workunit.client.0.smithi176.stdout:OK 2024-09-11T06:51:28.597 INFO:tasks.workunit.client.0.smithi176.stderr:+ cleanup 2024-09-11T06:51:28.598 INFO:tasks.workunit.client.0.smithi176.stderr:+ local ns s 2024-09-11T06:51:28.598 INFO:tasks.workunit.client.0.smithi176.stderr:+ set +e 2024-09-11T06:51:28.598 INFO:tasks.workunit.client.0.smithi176.stderr:+ fgrep /tmp/tmp.IGWYJYR7sz/mnt 2024-09-11T06:51:28.598 INFO:tasks.workunit.client.0.smithi176.stderr:+ mount 2024-09-11T06:51:28.609 INFO:tasks.workunit.client.0.smithi176.stderr:+ rm -Rf /tmp/tmp.IGWYJYR7sz 2024-09-11T06:51:28.622 INFO:tasks.workunit.client.0.smithi176.stderr:+ '[' -n '' ']' 2024-09-11T06:51:28.622 INFO:tasks.workunit.client.0.smithi176.stderr:+ for ns in '' ${NS} 2024-09-11T06:51:28.622 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd -p rbd --namespace '' status testrbdnbd21607 2024-09-11T06:51:28.677 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:28.677 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:28.688 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:28.681+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:28.691 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:51:28.692 INFO:tasks.workunit.client.0.smithi176.stdout:Watchers: none 2024-09-11T06:51:28.696 INFO:tasks.workunit.client.0.smithi176.stderr:+ for s in 0.5 1 2 4 8 16 32 2024-09-11T06:51:28.696 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 0.5 2024-09-11T06:51:28.793 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:28.790+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:28.822 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:28.822 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:28.836 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:51:28.890 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:28.885+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:28.991 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:28.986+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:29.003 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:29.003 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:29.017 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:51:29.061 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T06:51:29.109 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:29.106+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:29.184 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:29.184 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:29.193 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:29.190+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:29.198 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd -p rbd --namespace '' status testrbdnbd21607 2024-09-11T06:51:29.199 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep 'Watchers: none' 2024-09-11T06:51:29.199 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:51:29.263 INFO:tasks.workunit.client.0.smithi176.stdout:Watchers: none 2024-09-11T06:51:29.264 INFO:tasks.workunit.client.0.smithi176.stderr:+ break 2024-09-11T06:51:29.264 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd -p rbd --namespace '' snap purge testrbdnbd21607 2024-09-11T06:51:29.280 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:51:29.333 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:29.330+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:29.358 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:29.358 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:29.373 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:51:29.395 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:29.393+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:29.436 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:51:29.437 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:51:29.451 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:51:29.451 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:51:29.467 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:51:29.468 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:51:29.488 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:51:29.488 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:51:29.496 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:29.489+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:29.509 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:51:29.509 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:51:29.530 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:51:29.530 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:51:29.531 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:29.531 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:29.531 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:51:29.531 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:51:29.531 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:29.548 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:51:29.597 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:29.593+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:29.698 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:29.694+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:29.708 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:29.708 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:51:29.708 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:51:29.708 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:51:29.708 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.708 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4558.0:5 3.2 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-11T06:51:29.708 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:51:29.708 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 71.615682507000002, 2024-09-11T06:51:29.709 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0062353959999999998, 2024-09-11T06:51:29.709 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:29.709 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:29.709 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:29.709 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:29.709 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4558", 2024-09-11T06:51:29.709 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1478388898", 2024-09-11T06:51:29.709 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 5 2024-09-11T06:51:29.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.709 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:29.709 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.709 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:29.709 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:51:29.710 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.710 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.710 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.710 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:29.710 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:51:29.710 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.710 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.710 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.710 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:29.710 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088422+0000", 2024-09-11T06:51:29.710 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7859999999999998e-06 2024-09-11T06:51:29.710 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.710 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.711 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:29.711 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088440+0000", 2024-09-11T06:51:29.711 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8185999999999998e-05 2024-09-11T06:51:29.711 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.711 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.711 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:29.711 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088479+0000", 2024-09-11T06:51:29.711 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8248999999999999e-05 2024-09-11T06:51:29.711 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.711 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.711 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:29.711 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088497+0000", 2024-09-11T06:51:29.711 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8889000000000002e-05 2024-09-11T06:51:29.711 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.712 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.712 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:29.712 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088658+0000", 2024-09-11T06:51:29.712 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000160271 2024-09-11T06:51:29.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.712 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.712 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op must wait for map", 2024-09-11T06:51:29.712 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088668+0000", 2024-09-11T06:51:29.712 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0409e-05 2024-09-11T06:51:29.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.712 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.712 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:29.712 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092347+0000", 2024-09-11T06:51:29.713 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00367915 2024-09-11T06:51:29.713 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.713 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.713 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:29.713 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092541+0000", 2024-09-11T06:51:29.713 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000193833 2024-09-11T06:51:29.713 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.713 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.713 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-11T06:51:29.713 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092732+0000", 2024-09-11T06:51:29.713 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000190772 2024-09-11T06:51:29.713 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.714 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.714 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:29.714 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094136+0000", 2024-09-11T06:51:29.714 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014040809999999999 2024-09-11T06:51:29.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.714 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.714 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:29.714 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094506+0000", 2024-09-11T06:51:29.715 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000369959 2024-09-11T06:51:29.715 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.715 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.715 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:29.715 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094590+0000", 2024-09-11T06:51:29.715 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4033e-05 2024-09-11T06:51:29.715 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.715 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.715 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:29.715 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094654+0000", 2024-09-11T06:51:29.715 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3777999999999998e-05 2024-09-11T06:51:29.715 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.715 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:29.716 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.716 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.716 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.716 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:587 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-11T06:51:29.716 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:51:29.716 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 67.474849852000006, 2024-09-11T06:51:29.716 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0023436289999999999, 2024-09-11T06:51:29.717 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:29.717 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:29.717 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:29.717 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:29.717 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:51:29.717 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:51:29.717 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 587 2024-09-11T06:51:29.717 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.717 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:29.717 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.717 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:29.717 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:51:29.717 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.718 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.718 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.718 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:29.718 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:51:29.718 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.718 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.718 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.718 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:29.718 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229252+0000", 2024-09-11T06:51:29.718 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3799999999999999e-06 2024-09-11T06:51:29.718 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.718 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.718 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:29.718 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229259+0000", 2024-09-11T06:51:29.719 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4760000000000003e-06 2024-09-11T06:51:29.719 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.719 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.719 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:29.719 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229271+0000", 2024-09-11T06:51:29.719 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2313000000000001e-05 2024-09-11T06:51:29.719 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.719 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.719 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:29.719 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229279+0000", 2024-09-11T06:51:29.719 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1610000000000003e-06 2024-09-11T06:51:29.719 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.719 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.720 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:29.720 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229344+0000", 2024-09-11T06:51:29.720 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4375000000000006e-05 2024-09-11T06:51:29.720 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.720 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.720 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:29.720 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229404+0000", 2024-09-11T06:51:29.720 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0517999999999999e-05 2024-09-11T06:51:29.720 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.720 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.720 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:29.720 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229516+0000", 2024-09-11T06:51:29.720 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011129900000000001 2024-09-11T06:51:29.720 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.721 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.721 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:29.721 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231502+0000", 2024-09-11T06:51:29.721 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001986238 2024-09-11T06:51:29.721 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.721 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.721 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:29.721 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231537+0000", 2024-09-11T06:51:29.721 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4968000000000001e-05 2024-09-11T06:51:29.721 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.721 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.721 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:29.721 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231569+0000", 2024-09-11T06:51:29.722 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2459000000000002e-05 2024-09-11T06:51:29.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.722 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.722 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:29.722 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231595+0000", 2024-09-11T06:51:29.722 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5442000000000001e-05 2024-09-11T06:51:29.722 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.722 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:29.722 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.722 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.722 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:15 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:29.722 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.709418+0000", 2024-09-11T06:51:29.722 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 58.994682699000002, 2024-09-11T06:51:29.723 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027504280000000001, 2024-09-11T06:51:29.723 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:29.723 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:29.723 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:29.723 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:29.723 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:29.723 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:29.724 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 15 2024-09-11T06:51:29.724 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.724 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:29.724 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.724 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:29.724 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709418+0000", 2024-09-11T06:51:29.724 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.725 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.725 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.725 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:29.725 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709418+0000", 2024-09-11T06:51:29.725 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.725 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.725 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.725 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:29.725 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709420+0000", 2024-09-11T06:51:29.725 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2299999999999998e-06 2024-09-11T06:51:29.726 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.726 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.726 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:29.726 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709432+0000", 2024-09-11T06:51:29.726 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1897e-05 2024-09-11T06:51:29.726 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.726 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.726 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:29.726 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709464+0000", 2024-09-11T06:51:29.726 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1819000000000001e-05 2024-09-11T06:51:29.726 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.726 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.726 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:29.727 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709479+0000", 2024-09-11T06:51:29.727 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4853e-05 2024-09-11T06:51:29.727 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.727 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.727 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:29.727 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709624+0000", 2024-09-11T06:51:29.727 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144912 2024-09-11T06:51:29.727 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.727 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.727 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:29.727 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709773+0000", 2024-09-11T06:51:29.727 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000149529 2024-09-11T06:51:29.727 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.727 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.728 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:29.728 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.710055+0000", 2024-09-11T06:51:29.728 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000281692 2024-09-11T06:51:29.728 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.728 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.728 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:29.728 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.711412+0000", 2024-09-11T06:51:29.728 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001356931 2024-09-11T06:51:29.728 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.729 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.729 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:29.729 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.711890+0000", 2024-09-11T06:51:29.729 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00047768700000000002 2024-09-11T06:51:29.729 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.730 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.730 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:29.730 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.712050+0000", 2024-09-11T06:51:29.730 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016041499999999999 2024-09-11T06:51:29.730 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.730 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.731 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:29.731 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.712169+0000", 2024-09-11T06:51:29.731 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011846299999999999 2024-09-11T06:51:29.731 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.731 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:29.731 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.732 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.732 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:18 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:29.732 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.713502+0000", 2024-09-11T06:51:29.732 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 58.990599273000001, 2024-09-11T06:51:29.732 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032323069999999998, 2024-09-11T06:51:29.732 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:29.733 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:29.733 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:29.733 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:29.733 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:29.733 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:29.733 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18 2024-09-11T06:51:29.733 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.733 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:29.734 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.734 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:29.734 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713502+0000", 2024-09-11T06:51:29.734 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.734 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.734 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:29.735 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713502+0000", 2024-09-11T06:51:29.735 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.735 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:29.735 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713503+0000", 2024-09-11T06:51:29.736 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8199999999999999e-06 2024-09-11T06:51:29.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.736 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.736 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:29.736 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713513+0000", 2024-09-11T06:51:29.736 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8539999999999992e-06 2024-09-11T06:51:29.737 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.737 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.737 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:29.737 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713540+0000", 2024-09-11T06:51:29.737 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6594999999999999e-05 2024-09-11T06:51:29.737 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.737 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.738 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:29.738 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713553+0000", 2024-09-11T06:51:29.738 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2843999999999999e-05 2024-09-11T06:51:29.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.738 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.738 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:29.738 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713686+0000", 2024-09-11T06:51:29.738 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013294899999999999 2024-09-11T06:51:29.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.738 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.738 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:29.738 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713937+0000", 2024-09-11T06:51:29.738 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025110600000000002 2024-09-11T06:51:29.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.739 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:29.739 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.714412+0000", 2024-09-11T06:51:29.739 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00047562 2024-09-11T06:51:29.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.739 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:29.739 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.715867+0000", 2024-09-11T06:51:29.739 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001455004 2024-09-11T06:51:29.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.739 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:29.739 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.716569+0000", 2024-09-11T06:51:29.740 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070159400000000005 2024-09-11T06:51:29.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.740 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.740 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:29.740 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.716660+0000", 2024-09-11T06:51:29.740 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1403000000000006e-05 2024-09-11T06:51:29.741 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.741 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.741 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:29.741 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.716734+0000", 2024-09-11T06:51:29.741 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3517999999999996e-05 2024-09-11T06:51:29.741 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.741 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:29.741 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.741 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.741 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.741 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:23 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:29.741 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.723392+0000", 2024-09-11T06:51:29.741 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 58.980708677000003, 2024-09-11T06:51:29.741 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027098489999999999, 2024-09-11T06:51:29.742 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:29.742 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:29.742 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:29.742 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:29.742 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:29.742 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:29.742 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23 2024-09-11T06:51:29.742 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.742 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:29.742 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.742 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:29.742 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723392+0000", 2024-09-11T06:51:29.742 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.743 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.743 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.743 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:29.743 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723392+0000", 2024-09-11T06:51:29.743 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.743 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.743 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.743 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:29.743 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723394+0000", 2024-09-11T06:51:29.743 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0669999999999999e-06 2024-09-11T06:51:29.743 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.743 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.743 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:29.743 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723406+0000", 2024-09-11T06:51:29.744 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1367e-05 2024-09-11T06:51:29.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.744 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.744 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:29.744 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723436+0000", 2024-09-11T06:51:29.744 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0874999999999998e-05 2024-09-11T06:51:29.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.744 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.744 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:29.744 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723451+0000", 2024-09-11T06:51:29.744 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4192999999999999e-05 2024-09-11T06:51:29.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.745 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.745 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:29.745 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723577+0000", 2024-09-11T06:51:29.745 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000126267 2024-09-11T06:51:29.745 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.745 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.745 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:29.745 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723718+0000", 2024-09-11T06:51:29.745 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000141206 2024-09-11T06:51:29.745 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.745 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.745 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:29.745 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723982+0000", 2024-09-11T06:51:29.746 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026338499999999999 2024-09-11T06:51:29.746 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.746 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.746 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:29.746 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.725377+0000", 2024-09-11T06:51:29.746 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013956999999999999 2024-09-11T06:51:29.746 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.746 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.746 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:29.746 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.725908+0000", 2024-09-11T06:51:29.746 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00053036200000000004 2024-09-11T06:51:29.746 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.746 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.746 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:29.747 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.725999+0000", 2024-09-11T06:51:29.747 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1016999999999999e-05 2024-09-11T06:51:29.747 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.747 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.747 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:29.747 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726102+0000", 2024-09-11T06:51:29.747 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010341 2024-09-11T06:51:29.747 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.747 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:29.747 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.747 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.747 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.747 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:15 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:29.747 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.868921+0000", 2024-09-11T06:51:29.748 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 58.835179635999999, 2024-09-11T06:51:29.748 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0026879, 2024-09-11T06:51:29.748 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:29.748 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:29.748 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:29.748 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:29.748 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:29.748 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:29.748 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 15 2024-09-11T06:51:29.748 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.748 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:29.748 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.748 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:29.749 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868921+0000", 2024-09-11T06:51:29.749 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.749 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.749 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:29.749 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868921+0000", 2024-09-11T06:51:29.749 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.749 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.749 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:29.749 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868923+0000", 2024-09-11T06:51:29.749 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8959999999999999e-06 2024-09-11T06:51:29.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.750 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.750 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:29.750 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868934+0000", 2024-09-11T06:51:29.750 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1083e-05 2024-09-11T06:51:29.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.750 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.750 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:29.750 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868965+0000", 2024-09-11T06:51:29.750 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0833000000000001e-05 2024-09-11T06:51:29.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.751 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.751 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:29.751 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868990+0000", 2024-09-11T06:51:29.751 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4545999999999998e-05 2024-09-11T06:51:29.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.751 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.751 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:29.751 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.869124+0000", 2024-09-11T06:51:29.751 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013489599999999999 2024-09-11T06:51:29.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.751 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.751 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:29.752 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.869270+0000", 2024-09-11T06:51:29.752 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000145976 2024-09-11T06:51:29.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.752 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.752 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:29.752 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.869538+0000", 2024-09-11T06:51:29.752 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026780599999999999 2024-09-11T06:51:29.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.752 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.752 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:29.752 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.870691+0000", 2024-09-11T06:51:29.752 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00115277 2024-09-11T06:51:29.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.753 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:29.753 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.871456+0000", 2024-09-11T06:51:29.753 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00076473800000000005 2024-09-11T06:51:29.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.753 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:29.753 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.871543+0000", 2024-09-11T06:51:29.753 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7040999999999996e-05 2024-09-11T06:51:29.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.753 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:29.754 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.871609+0000", 2024-09-11T06:51:29.754 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6315000000000005e-05 2024-09-11T06:51:29.754 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.754 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:29.754 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.754 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.754 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.754 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:20 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:29.754 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.877429+0000", 2024-09-11T06:51:29.754 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 58.826671347000001, 2024-09-11T06:51:29.754 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032382629999999999, 2024-09-11T06:51:29.754 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:29.755 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:29.755 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:29.755 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:29.755 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:29.755 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:29.755 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 20 2024-09-11T06:51:29.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.755 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:29.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.755 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:29.755 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877429+0000", 2024-09-11T06:51:29.755 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.756 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.756 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:29.756 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877429+0000", 2024-09-11T06:51:29.756 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.756 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.756 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.756 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:29.756 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877431+0000", 2024-09-11T06:51:29.756 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9429999999999999e-06 2024-09-11T06:51:29.756 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.757 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.757 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:29.757 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877444+0000", 2024-09-11T06:51:29.757 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2396e-05 2024-09-11T06:51:29.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.757 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.757 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:29.757 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877474+0000", 2024-09-11T06:51:29.757 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0239999999999998e-05 2024-09-11T06:51:29.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.757 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.758 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:29.758 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877496+0000", 2024-09-11T06:51:29.758 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1914999999999999e-05 2024-09-11T06:51:29.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.758 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.758 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:29.758 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877616+0000", 2024-09-11T06:51:29.758 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000119814 2024-09-11T06:51:29.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.758 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.758 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:29.758 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877836+0000", 2024-09-11T06:51:29.758 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022029600000000001 2024-09-11T06:51:29.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.759 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:29.759 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.878482+0000", 2024-09-11T06:51:29.759 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00064605200000000004 2024-09-11T06:51:29.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.759 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:29.759 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.879804+0000", 2024-09-11T06:51:29.759 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013216460000000001 2024-09-11T06:51:29.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.759 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:29.759 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.880519+0000", 2024-09-11T06:51:29.759 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00071547600000000005 2024-09-11T06:51:29.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.760 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.760 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:29.760 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.880606+0000", 2024-09-11T06:51:29.760 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6907e-05 2024-09-11T06:51:29.760 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.760 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.760 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:29.760 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.880668+0000", 2024-09-11T06:51:29.760 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1577999999999999e-05 2024-09-11T06:51:29.761 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.761 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:29.761 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.761 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.761 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:23 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:29.761 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.885067+0000", 2024-09-11T06:51:29.761 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 58.819033693000002, 2024-09-11T06:51:29.761 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028127320000000001, 2024-09-11T06:51:29.761 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:29.761 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:29.761 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:29.762 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:29.762 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:29.762 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:29.762 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23 2024-09-11T06:51:29.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.762 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:29.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.762 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:29.762 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885067+0000", 2024-09-11T06:51:29.762 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.762 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:29.762 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885067+0000", 2024-09-11T06:51:29.763 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.763 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.763 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:29.763 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885069+0000", 2024-09-11T06:51:29.763 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2000000000000001e-06 2024-09-11T06:51:29.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.763 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.763 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:29.763 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885081+0000", 2024-09-11T06:51:29.763 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1209999999999999e-05 2024-09-11T06:51:29.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.763 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.763 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:29.764 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885111+0000", 2024-09-11T06:51:29.764 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0318999999999999e-05 2024-09-11T06:51:29.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.764 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:29.764 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885132+0000", 2024-09-11T06:51:29.764 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0649000000000002e-05 2024-09-11T06:51:29.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.764 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:29.764 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885243+0000", 2024-09-11T06:51:29.764 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000111284 2024-09-11T06:51:29.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.764 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:29.765 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885387+0000", 2024-09-11T06:51:29.765 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144222 2024-09-11T06:51:29.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.765 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.765 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:29.765 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885699+0000", 2024-09-11T06:51:29.765 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00031239800000000003 2024-09-11T06:51:29.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.765 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.765 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:29.765 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.886817+0000", 2024-09-11T06:51:29.765 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001117191 2024-09-11T06:51:29.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.766 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:29.766 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.887684+0000", 2024-09-11T06:51:29.766 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00086699499999999996 2024-09-11T06:51:29.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.766 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:29.766 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.887772+0000", 2024-09-11T06:51:29.766 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8224000000000001e-05 2024-09-11T06:51:29.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.766 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:29.767 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.887880+0000", 2024-09-11T06:51:29.767 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010804 2024-09-11T06:51:29.767 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.767 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:29.767 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.767 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "MOSDPGPull(2.0 e70/69 cost 1048576)", 2024-09-11T06:51:29.767 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:35.528435+0000", 2024-09-11T06:51:29.767 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 54.175665957, 2024-09-11T06:51:29.767 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027390520000000001, 2024-09-11T06:51:29.767 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:29.768 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:29.768 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-11T06:51:29.768 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:29.768 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.768 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:29.768 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528435+0000", 2024-09-11T06:51:29.768 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.768 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.768 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:29.768 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528435+0000", 2024-09-11T06:51:29.768 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.769 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.769 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:29.769 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528437+0000", 2024-09-11T06:51:29.769 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5959999999999998e-06 2024-09-11T06:51:29.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.769 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.769 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:29.769 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528465+0000", 2024-09-11T06:51:29.769 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7436e-05 2024-09-11T06:51:29.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.769 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.769 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:29.769 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528499+0000", 2024-09-11T06:51:29.769 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3599000000000001e-05 2024-09-11T06:51:29.770 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.770 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.770 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:29.770 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528515+0000", 2024-09-11T06:51:29.770 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.677e-05 2024-09-11T06:51:29.770 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.770 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.770 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:29.770 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528652+0000", 2024-09-11T06:51:29.771 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000136546 2024-09-11T06:51:29.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.771 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:29.771 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.531174+0000", 2024-09-11T06:51:29.771 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0025221050000000002 2024-09-11T06:51:29.771 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.771 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:29.771 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.771 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4828.0:15 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-11T06:51:29.771 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:36.253730+0000", 2024-09-11T06:51:29.771 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 53.450370614000001, 2024-09-11T06:51:29.771 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0025257589999999998, 2024-09-11T06:51:29.772 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:29.772 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:29.772 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:29.772 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:29.772 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4828", 2024-09-11T06:51:29.772 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2410288650", 2024-09-11T06:51:29.772 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 15 2024-09-11T06:51:29.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.772 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:29.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.772 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:29.772 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253730+0000", 2024-09-11T06:51:29.772 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.773 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:29.773 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253730+0000", 2024-09-11T06:51:29.773 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.773 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:29.773 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253732+0000", 2024-09-11T06:51:29.773 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2400000000000002e-06 2024-09-11T06:51:29.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.773 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:29.773 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253744+0000", 2024-09-11T06:51:29.773 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1316e-05 2024-09-11T06:51:29.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.774 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:29.774 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253771+0000", 2024-09-11T06:51:29.774 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.724e-05 2024-09-11T06:51:29.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.774 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:29.774 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253785+0000", 2024-09-11T06:51:29.774 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3786e-05 2024-09-11T06:51:29.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.774 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:29.774 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253886+0000", 2024-09-11T06:51:29.774 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000100981 2024-09-11T06:51:29.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.775 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:29.775 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.254029+0000", 2024-09-11T06:51:29.775 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014311900000000001 2024-09-11T06:51:29.775 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.775 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:29.775 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.254317+0000", 2024-09-11T06:51:29.775 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00028809300000000001 2024-09-11T06:51:29.775 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.775 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:29.775 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.255403+0000", 2024-09-11T06:51:29.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001085639 2024-09-11T06:51:29.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:29.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.256107+0000", 2024-09-11T06:51:29.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070468799999999997 2024-09-11T06:51:29.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:29.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.256191+0000", 2024-09-11T06:51:29.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3714999999999994e-05 2024-09-11T06:51:29.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:29.777 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.256256+0000", 2024-09-11T06:51:29.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4942000000000003e-05 2024-09-11T06:51:29.777 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.777 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:29.777 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.777 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5313.0:26 2.6 2:7b712441:ns::rbd_object_map.11adc335555f:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:51:29.777 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.260230+0000", 2024-09-11T06:51:29.777 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 50.443871205999997, 2024-09-11T06:51:29.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002457201, 2024-09-11T06:51:29.777 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:29.777 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:29.778 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:29.778 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:29.778 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5313", 2024-09-11T06:51:29.778 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/624314229", 2024-09-11T06:51:29.778 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:51:29.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.778 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:29.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.778 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:29.778 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260230+0000", 2024-09-11T06:51:29.778 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.778 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:29.778 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260230+0000", 2024-09-11T06:51:29.778 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.779 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.779 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.779 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:29.779 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260232+0000", 2024-09-11T06:51:29.779 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.956e-06 2024-09-11T06:51:29.779 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.779 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.779 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:29.779 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260243+0000", 2024-09-11T06:51:29.779 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1687e-05 2024-09-11T06:51:29.779 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.779 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.779 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:29.779 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260269+0000", 2024-09-11T06:51:29.780 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6211999999999999e-05 2024-09-11T06:51:29.780 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.780 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.780 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:29.780 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260284+0000", 2024-09-11T06:51:29.780 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4053000000000001e-05 2024-09-11T06:51:29.780 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.780 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.780 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:29.780 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260391+0000", 2024-09-11T06:51:29.780 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000107278 2024-09-11T06:51:29.780 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.780 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:29.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260541+0000", 2024-09-11T06:51:29.781 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015033000000000001 2024-09-11T06:51:29.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 3", 2024-09-11T06:51:29.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260951+0000", 2024-09-11T06:51:29.781 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040981299999999997 2024-09-11T06:51:29.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:29.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262399+0000", 2024-09-11T06:51:29.781 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014475930000000001 2024-09-11T06:51:29.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.782 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:29.782 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262531+0000", 2024-09-11T06:51:29.782 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013239 2024-09-11T06:51:29.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.782 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:29.782 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262623+0000", 2024-09-11T06:51:29.782 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2361000000000006e-05 2024-09-11T06:51:29.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.782 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:29.782 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262687+0000", 2024-09-11T06:51:29.783 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3528000000000006e-05 2024-09-11T06:51:29.783 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.783 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:29.783 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.783 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5313.0:31 2.6 2:7b712441:ns::rbd_object_map.11adc335555f:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:51:29.783 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.266495+0000", 2024-09-11T06:51:29.783 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 50.437605431999998, 2024-09-11T06:51:29.783 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0026763870000000001, 2024-09-11T06:51:29.783 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:29.783 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:29.783 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:29.784 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:29.784 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5313", 2024-09-11T06:51:29.784 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/624314229", 2024-09-11T06:51:29.784 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 31 2024-09-11T06:51:29.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.784 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:29.784 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.784 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:29.784 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266495+0000", 2024-09-11T06:51:29.784 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.784 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.784 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:29.784 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266495+0000", 2024-09-11T06:51:29.785 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.785 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.785 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:29.785 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266498+0000", 2024-09-11T06:51:29.785 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2369999999999999e-06 2024-09-11T06:51:29.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.785 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.785 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:29.785 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266509+0000", 2024-09-11T06:51:29.785 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1459e-05 2024-09-11T06:51:29.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.785 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.785 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:29.786 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266540+0000", 2024-09-11T06:51:29.786 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1189999999999998e-05 2024-09-11T06:51:29.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.786 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:29.786 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266563+0000", 2024-09-11T06:51:29.786 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2529000000000001e-05 2024-09-11T06:51:29.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.786 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:29.786 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266708+0000", 2024-09-11T06:51:29.786 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014511 2024-09-11T06:51:29.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.786 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:29.787 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266962+0000", 2024-09-11T06:51:29.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000253897 2024-09-11T06:51:29.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.787 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 3", 2024-09-11T06:51:29.787 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.267430+0000", 2024-09-11T06:51:29.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00046799899999999998 2024-09-11T06:51:29.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.787 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:29.787 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.268765+0000", 2024-09-11T06:51:29.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013350359999999999 2024-09-11T06:51:29.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.788 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:29.788 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.269019+0000", 2024-09-11T06:51:29.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025397000000000001 2024-09-11T06:51:29.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.788 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:29.788 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.269105+0000", 2024-09-11T06:51:29.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5952000000000006e-05 2024-09-11T06:51:29.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.788 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:29.788 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.269172+0000", 2024-09-11T06:51:29.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7008000000000006e-05 2024-09-11T06:51:29.789 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.789 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:29.789 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.789 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.789 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:5 3.4 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:29.789 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.588582+0000", 2024-09-11T06:51:29.789 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 44.115519278999997, 2024-09-11T06:51:29.789 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002889857, 2024-09-11T06:51:29.789 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:29.789 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:29.789 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:29.790 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:29.790 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:51:29.790 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:51:29.790 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 5 2024-09-11T06:51:29.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.790 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:29.790 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.790 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:29.790 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588582+0000", 2024-09-11T06:51:29.790 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.790 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.790 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:29.790 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588582+0000", 2024-09-11T06:51:29.791 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.791 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.791 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:29.791 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588583+0000", 2024-09-11T06:51:29.791 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.223e-06 2024-09-11T06:51:29.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.791 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.791 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:29.791 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588588+0000", 2024-09-11T06:51:29.791 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4299999999999997e-06 2024-09-11T06:51:29.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.791 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.791 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:29.792 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588606+0000", 2024-09-11T06:51:29.792 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7652999999999999e-05 2024-09-11T06:51:29.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.792 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:29.792 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588615+0000", 2024-09-11T06:51:29.792 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4399999999999994e-06 2024-09-11T06:51:29.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.792 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:29.792 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588725+0000", 2024-09-11T06:51:29.792 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010945799999999999 2024-09-11T06:51:29.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.793 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.793 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:29.793 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.589031+0000", 2024-09-11T06:51:29.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030590000000000001 2024-09-11T06:51:29.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.793 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.793 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:29.793 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.589521+0000", 2024-09-11T06:51:29.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00049061299999999999 2024-09-11T06:51:29.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.793 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.793 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:29.793 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.590792+0000", 2024-09-11T06:51:29.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012708039999999999 2024-09-11T06:51:29.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.794 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:29.794 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591339+0000", 2024-09-11T06:51:29.794 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00054710499999999999 2024-09-11T06:51:29.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.794 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:29.794 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591412+0000", 2024-09-11T06:51:29.794 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2811999999999993e-05 2024-09-11T06:51:29.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.795 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:29.795 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591471+0000", 2024-09-11T06:51:29.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9419000000000001e-05 2024-09-11T06:51:29.795 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.795 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:29.795 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.795 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:8 3.4 3:3f0de396:::rbd_header.1415bf940188:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:29.795 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.596120+0000", 2024-09-11T06:51:29.795 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 44.107980394999998, 2024-09-11T06:51:29.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027911020000000002, 2024-09-11T06:51:29.795 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:29.795 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:29.796 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:29.796 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:29.796 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:51:29.796 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:51:29.796 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 8 2024-09-11T06:51:29.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.796 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:29.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:29.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596120+0000", 2024-09-11T06:51:29.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:29.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596120+0000", 2024-09-11T06:51:29.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:29.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596122+0000", 2024-09-11T06:51:29.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.513e-06 2024-09-11T06:51:29.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:29.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596130+0000", 2024-09-11T06:51:29.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4999999999999999e-06 2024-09-11T06:51:29.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:29.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596152+0000", 2024-09-11T06:51:29.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1846000000000001e-05 2024-09-11T06:51:29.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:29.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596163+0000", 2024-09-11T06:51:29.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1029999999999999e-05 2024-09-11T06:51:29.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:29.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596242+0000", 2024-09-11T06:51:29.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8275000000000005e-05 2024-09-11T06:51:29.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.799 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:29.799 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596439+0000", 2024-09-11T06:51:29.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000196999 2024-09-11T06:51:29.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.799 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:29.799 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596755+0000", 2024-09-11T06:51:29.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00031626100000000003 2024-09-11T06:51:29.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.799 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:29.799 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.598043+0000", 2024-09-11T06:51:29.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001288389 2024-09-11T06:51:29.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.800 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.800 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:29.800 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.598774+0000", 2024-09-11T06:51:29.800 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00073080499999999995 2024-09-11T06:51:29.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.800 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.800 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:29.800 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.598855+0000", 2024-09-11T06:51:29.800 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0615e-05 2024-09-11T06:51:29.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.800 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.800 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:29.800 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.598912+0000", 2024-09-11T06:51:29.800 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6869e-05 2024-09-11T06:51:29.801 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.801 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:29.801 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.801 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:9 3.0 3:00834abc:::rbd_object_map.1415bf940188:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:29.801 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.599161+0000", 2024-09-11T06:51:29.801 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 44.104939377999997, 2024-09-11T06:51:29.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002826051, 2024-09-11T06:51:29.801 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:29.801 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:29.801 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:29.801 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:29.801 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:51:29.801 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:51:29.802 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 9 2024-09-11T06:51:29.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.802 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:29.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:29.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599161+0000", 2024-09-11T06:51:29.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:29.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599161+0000", 2024-09-11T06:51:29.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.803 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:29.803 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599163+0000", 2024-09-11T06:51:29.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8700000000000001e-06 2024-09-11T06:51:29.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.803 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:29.803 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599174+0000", 2024-09-11T06:51:29.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0416e-05 2024-09-11T06:51:29.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.803 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:29.804 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599202+0000", 2024-09-11T06:51:29.804 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7946e-05 2024-09-11T06:51:29.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.804 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:29.804 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599216+0000", 2024-09-11T06:51:29.804 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3934000000000001e-05 2024-09-11T06:51:29.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.804 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:29.804 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599345+0000", 2024-09-11T06:51:29.804 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012954000000000001 2024-09-11T06:51:29.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.805 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:29.805 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599540+0000", 2024-09-11T06:51:29.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019516299999999999 2024-09-11T06:51:29.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.805 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:29.805 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599796+0000", 2024-09-11T06:51:29.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025556500000000002 2024-09-11T06:51:29.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:29.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.601071+0000", 2024-09-11T06:51:29.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001275396 2024-09-11T06:51:29.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:29.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.601852+0000", 2024-09-11T06:51:29.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000780524 2024-09-11T06:51:29.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:29.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.601925+0000", 2024-09-11T06:51:29.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3574999999999999e-05 2024-09-11T06:51:29.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.807 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:29.807 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.601988+0000", 2024-09-11T06:51:29.807 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2121999999999999e-05 2024-09-11T06:51:29.807 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.807 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:29.807 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.807 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5038.0:12 3.4 3:3f0de396:::rbd_header.1415bf940188:head [watch watch cookie 140693532068064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:29.807 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.718024+0000", 2024-09-11T06:51:29.808 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 43.986076439999998, 2024-09-11T06:51:29.808 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002724583, 2024-09-11T06:51:29.808 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:29.808 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:29.808 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:29.808 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:29.808 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5038", 2024-09-11T06:51:29.808 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/945190275", 2024-09-11T06:51:29.808 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:51:29.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.808 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:29.808 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.808 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:29.809 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718024+0000", 2024-09-11T06:51:29.809 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.809 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:29.809 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718024+0000", 2024-09-11T06:51:29.809 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.809 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:29.809 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718027+0000", 2024-09-11T06:51:29.809 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1430000000000001e-06 2024-09-11T06:51:29.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:29.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718037+0000", 2024-09-11T06:51:29.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0253e-05 2024-09-11T06:51:29.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:29.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718068+0000", 2024-09-11T06:51:29.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1050000000000003e-05 2024-09-11T06:51:29.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:29.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718086+0000", 2024-09-11T06:51:29.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8629000000000001e-05 2024-09-11T06:51:29.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.811 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:29.811 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718206+0000", 2024-09-11T06:51:29.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000119358 2024-09-11T06:51:29.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.811 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:29.811 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718425+0000", 2024-09-11T06:51:29.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000218975 2024-09-11T06:51:29.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.811 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:29.811 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718830+0000", 2024-09-11T06:51:29.812 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040538199999999998 2024-09-11T06:51:29.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.812 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:29.812 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.719849+0000", 2024-09-11T06:51:29.812 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010187589999999999 2024-09-11T06:51:29.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.812 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:29.812 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.720548+0000", 2024-09-11T06:51:29.812 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000699112 2024-09-11T06:51:29.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.813 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:29.813 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.720632+0000", 2024-09-11T06:51:29.813 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4314999999999995e-05 2024-09-11T06:51:29.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.813 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:29.813 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.720749+0000", 2024-09-11T06:51:29.813 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116607 2024-09-11T06:51:29.813 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.813 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:29.813 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.814 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:47 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:29.814 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.553228+0000", 2024-09-11T06:51:29.814 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.15087235, 2024-09-11T06:51:29.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040683100000000003, 2024-09-11T06:51:29.814 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:29.814 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:29.814 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:29.814 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:29.814 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:29.814 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:29.814 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 47 2024-09-11T06:51:29.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.814 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:29.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:29.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553228+0000", 2024-09-11T06:51:29.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:29.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553228+0000", 2024-09-11T06:51:29.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:29.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553231+0000", 2024-09-11T06:51:29.816 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3130000000000001e-06 2024-09-11T06:51:29.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.816 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:29.816 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553242+0000", 2024-09-11T06:51:29.816 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1345999999999999e-05 2024-09-11T06:51:29.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.816 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:29.817 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553272+0000", 2024-09-11T06:51:29.817 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0179999999999999e-05 2024-09-11T06:51:29.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.817 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.817 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:29.817 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553287+0000", 2024-09-11T06:51:29.817 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4283e-05 2024-09-11T06:51:29.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.817 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.817 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:29.817 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553415+0000", 2024-09-11T06:51:29.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000128491 2024-09-11T06:51:29.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.818 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:29.818 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553651+0000", 2024-09-11T06:51:29.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000236328 2024-09-11T06:51:29.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.818 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:29.818 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.554356+0000", 2024-09-11T06:51:29.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070436299999999995 2024-09-11T06:51:29.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:29.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.556064+0000", 2024-09-11T06:51:29.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0017084439999999999 2024-09-11T06:51:29.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:29.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.556993+0000", 2024-09-11T06:51:29.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00092912400000000003 2024-09-11T06:51:29.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:29.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.557173+0000", 2024-09-11T06:51:29.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017940400000000001 2024-09-11T06:51:29.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:29.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.557297+0000", 2024-09-11T06:51:29.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000124034 2024-09-11T06:51:29.820 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.820 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:29.820 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.820 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:66 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:29.820 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.589465+0000", 2024-09-11T06:51:29.820 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.114635544999999, 2024-09-11T06:51:29.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039855489999999997, 2024-09-11T06:51:29.821 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:29.821 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:29.821 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:29.821 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:29.821 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:29.821 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:29.821 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 66 2024-09-11T06:51:29.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.821 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:29.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.821 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:29.821 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589465+0000", 2024-09-11T06:51:29.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.822 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.822 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:29.822 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589465+0000", 2024-09-11T06:51:29.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.822 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.822 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:29.822 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589468+0000", 2024-09-11T06:51:29.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3539999999999998e-06 2024-09-11T06:51:29.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.822 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.822 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:29.822 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589480+0000", 2024-09-11T06:51:29.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1946e-05 2024-09-11T06:51:29.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.823 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:29.823 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589512+0000", 2024-09-11T06:51:29.823 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2712999999999997e-05 2024-09-11T06:51:29.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.823 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:29.823 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589527+0000", 2024-09-11T06:51:29.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4686e-05 2024-09-11T06:51:29.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.824 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:29.824 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589673+0000", 2024-09-11T06:51:29.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014630700000000001 2024-09-11T06:51:29.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:29.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589936+0000", 2024-09-11T06:51:29.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026234499999999998 2024-09-11T06:51:29.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:29.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.591001+0000", 2024-09-11T06:51:29.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001065023 2024-09-11T06:51:29.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:29.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.592757+0000", 2024-09-11T06:51:29.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0017560849999999999 2024-09-11T06:51:29.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.826 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:29.826 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.593297+0000", 2024-09-11T06:51:29.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00054007600000000001 2024-09-11T06:51:29.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.826 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:29.826 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.593385+0000", 2024-09-11T06:51:29.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8327999999999999e-05 2024-09-11T06:51:29.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.826 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:29.826 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.593451+0000", 2024-09-11T06:51:29.827 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5685999999999994e-05 2024-09-11T06:51:29.827 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.827 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:29.827 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.827 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.827 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.827 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5315.0:79 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-11T06:51:29.827 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:07.271557+0000", 2024-09-11T06:51:29.827 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.432543745, 2024-09-11T06:51:29.827 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0023454019999999999, 2024-09-11T06:51:29.827 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:29.827 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:29.827 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:29.827 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:29.827 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5315", 2024-09-11T06:51:29.828 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2243844038", 2024-09-11T06:51:29.828 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 79 2024-09-11T06:51:29.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.828 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:29.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:29.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271557+0000", 2024-09-11T06:51:29.828 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:29.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271557+0000", 2024-09-11T06:51:29.828 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:29.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271560+0000", 2024-09-11T06:51:29.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3560000000000001e-06 2024-09-11T06:51:29.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:29.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271596+0000", 2024-09-11T06:51:29.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5593000000000003e-05 2024-09-11T06:51:29.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:29.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271640+0000", 2024-09-11T06:51:29.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4125999999999997e-05 2024-09-11T06:51:29.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.830 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:29.830 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271652+0000", 2024-09-11T06:51:29.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1853e-05 2024-09-11T06:51:29.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.830 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:29.830 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271785+0000", 2024-09-11T06:51:29.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013291299999999999 2024-09-11T06:51:29.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:29.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271948+0000", 2024-09-11T06:51:29.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000163191 2024-09-11T06:51:29.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:29.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.272260+0000", 2024-09-11T06:51:29.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00031196700000000002 2024-09-11T06:51:29.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:29.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.273293+0000", 2024-09-11T06:51:29.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010324139999999999 2024-09-11T06:51:29.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:29.832 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.273805+0000", 2024-09-11T06:51:29.832 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00051249399999999995 2024-09-11T06:51:29.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:29.832 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.273858+0000", 2024-09-11T06:51:29.832 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2975e-05 2024-09-11T06:51:29.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:29.832 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.273902+0000", 2024-09-11T06:51:29.832 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4520000000000001e-05 2024-09-11T06:51:29.833 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.833 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:29.833 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.833 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5239.0:52 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-11T06:51:29.833 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:08.176534+0000", 2024-09-11T06:51:29.833 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.527566820000001, 2024-09-11T06:51:29.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0022800960000000001, 2024-09-11T06:51:29.833 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:29.833 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:29.833 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:29.833 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:29.834 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5239", 2024-09-11T06:51:29.834 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3565361812", 2024-09-11T06:51:29.834 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 52 2024-09-11T06:51:29.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.834 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:29.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.834 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:29.834 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176534+0000", 2024-09-11T06:51:29.834 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.834 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:29.834 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176534+0000", 2024-09-11T06:51:29.834 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:29.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:29.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176537+0000", 2024-09-11T06:51:29.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5600000000000001e-06 2024-09-11T06:51:29.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:29.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176551+0000", 2024-09-11T06:51:29.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3926e-05 2024-09-11T06:51:29.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:29.836 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176584+0000", 2024-09-11T06:51:29.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3105999999999998e-05 2024-09-11T06:51:29.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:29.836 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176600+0000", 2024-09-11T06:51:29.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6327000000000001e-05 2024-09-11T06:51:29.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:29.836 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176753+0000", 2024-09-11T06:51:29.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015280900000000001 2024-09-11T06:51:29.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:29.837 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176911+0000", 2024-09-11T06:51:29.837 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000158145 2024-09-11T06:51:29.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.837 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.837 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:29.837 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.177216+0000", 2024-09-11T06:51:29.837 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030545000000000002 2024-09-11T06:51:29.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.837 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.837 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:29.837 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.178461+0000", 2024-09-11T06:51:29.837 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012451210000000001 2024-09-11T06:51:29.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.837 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:29.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.178706+0000", 2024-09-11T06:51:29.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024469999999999998 2024-09-11T06:51:29.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:29.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.178763+0000", 2024-09-11T06:51:29.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7339000000000001e-05 2024-09-11T06:51:29.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:29.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:29.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:29.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.178814+0000", 2024-09-11T06:51:29.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0612999999999998e-05 2024-09-11T06:51:29.838 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.838 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:29.838 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.839 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:29.839 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:29.839 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:29.840 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:51:29.841 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:29.838+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:29.941 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:29.938+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:30.007 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:30.008 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:30.008 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:51:30.008 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:30.021 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:51:30.034 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 3, 5] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T06:51:30.034 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-11T06:51:30.042 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:30.041+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:30.104 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is unset 2024-09-11T06:51:30.173 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:30.170+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:30.191 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:30.191 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:30.191 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:51:30.191 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:51:30.191 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:30.207 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:51:30.243 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:30.237+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:30.344 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:30.337+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:30.369 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:30.370 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:51:30.370 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:51:30.370 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:51:30.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.370 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:26 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 140250278007648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:30.370 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:51:30.370 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 59.637236254999998, 2024-09-11T06:51:30.370 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034645140000000001, 2024-09-11T06:51:30.370 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:30.371 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:30.371 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:30.371 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:30.371 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:30.371 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:30.371 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:51:30.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.371 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:30.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.371 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:30.371 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:51:30.371 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.372 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:30.372 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:51:30.372 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.372 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:30.372 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729634+0000", 2024-09-11T06:51:30.372 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0470000000000001e-06 2024-09-11T06:51:30.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:30.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729640+0000", 2024-09-11T06:51:30.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8860000000000001e-06 2024-09-11T06:51:30.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:30.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729654+0000", 2024-09-11T06:51:30.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:51:30.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729664+0000", 2024-09-11T06:51:30.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.594e-06 2024-09-11T06:51:30.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:30.374 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730422+0000", 2024-09-11T06:51:30.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000758648 2024-09-11T06:51:30.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:30.374 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730571+0000", 2024-09-11T06:51:30.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000148932 2024-09-11T06:51:30.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:30.374 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730936+0000", 2024-09-11T06:51:30.375 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036510900000000002 2024-09-11T06:51:30.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.375 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:30.375 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732159+0000", 2024-09-11T06:51:30.375 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001222302 2024-09-11T06:51:30.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.375 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:30.375 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732763+0000", 2024-09-11T06:51:30.375 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00060421599999999998 2024-09-11T06:51:30.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.375 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:30.376 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732916+0000", 2024-09-11T06:51:30.376 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015264 2024-09-11T06:51:30.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.376 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:30.376 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733097+0000", 2024-09-11T06:51:30.376 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000181504 2024-09-11T06:51:30.376 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.376 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:30.376 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.376 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:73 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:30.376 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:51:30.377 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 59.636178069000003, 2024-09-11T06:51:30.377 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033454439999999999, 2024-09-11T06:51:30.377 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:30.377 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:30.377 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:30.377 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:30.377 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:51:30.377 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:51:30.377 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73 2024-09-11T06:51:30.377 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.377 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:30.377 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.377 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:30.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:51:30.378 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.378 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:30.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730691+0000", 2024-09-11T06:51:30.378 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.378 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:30.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730694+0000", 2024-09-11T06:51:30.378 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5099999999999999e-06 2024-09-11T06:51:30.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.379 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:30.379 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730708+0000", 2024-09-11T06:51:30.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3237e-05 2024-09-11T06:51:30.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.379 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:30.379 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730738+0000", 2024-09-11T06:51:30.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0688999999999997e-05 2024-09-11T06:51:30.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.379 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.379 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730759+0000", 2024-09-11T06:51:30.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0818999999999998e-05 2024-09-11T06:51:30.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:30.380 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.731841+0000", 2024-09-11T06:51:30.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010821819999999999 2024-09-11T06:51:30.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:30.380 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732092+0000", 2024-09-11T06:51:30.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025086299999999999 2024-09-11T06:51:30.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:30.380 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733730+0000", 2024-09-11T06:51:30.381 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0016374600000000001 2024-09-11T06:51:30.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.381 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:30.381 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733866+0000", 2024-09-11T06:51:30.381 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000136666 2024-09-11T06:51:30.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.381 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:30.381 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.734036+0000", 2024-09-11T06:51:30.381 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000170018 2024-09-11T06:51:30.382 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.382 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:30.382 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.382 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:26 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 139740787512064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:30.382 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:51:30.382 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 59.474552529999997, 2024-09-11T06:51:30.382 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003613733, 2024-09-11T06:51:30.382 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:30.382 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:30.382 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:30.382 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:30.382 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:30.382 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:30.383 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:51:30.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.383 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:30.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.383 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:30.383 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:51:30.383 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.383 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:30.383 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:51:30.383 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.384 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:30.384 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892318+0000", 2024-09-11T06:51:30.384 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.623e-06 2024-09-11T06:51:30.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.384 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:30.384 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892325+0000", 2024-09-11T06:51:30.384 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:51:30.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.384 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:30.384 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892346+0000", 2024-09-11T06:51:30.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1246e-05 2024-09-11T06:51:30.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.385 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.385 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892362+0000", 2024-09-11T06:51:30.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5409000000000001e-05 2024-09-11T06:51:30.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.385 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:30.385 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892883+0000", 2024-09-11T06:51:30.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00052087599999999998 2024-09-11T06:51:30.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.386 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:30.386 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893123+0000", 2024-09-11T06:51:30.386 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000240769 2024-09-11T06:51:30.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.386 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:30.386 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893526+0000", 2024-09-11T06:51:30.386 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040300200000000002 2024-09-11T06:51:30.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.386 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:30.386 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894820+0000", 2024-09-11T06:51:30.386 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012934159999999999 2024-09-11T06:51:30.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.387 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:30.387 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895608+0000", 2024-09-11T06:51:30.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00078788100000000004 2024-09-11T06:51:30.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.387 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:30.387 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895724+0000", 2024-09-11T06:51:30.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011595999999999999 2024-09-11T06:51:30.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.387 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:30.387 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895930+0000", 2024-09-11T06:51:30.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000206391 2024-09-11T06:51:30.388 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.388 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:30.388 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.388 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:133 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:30.388 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:51:30.388 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 59.473754513000003, 2024-09-11T06:51:30.388 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035017360000000001, 2024-09-11T06:51:30.388 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:30.388 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:30.388 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:30.388 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:30.388 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:51:30.389 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:51:30.389 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 133 2024-09-11T06:51:30.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.389 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:30.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.389 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:30.389 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:51:30.389 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.389 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:30.389 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:51:30.389 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.390 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:30.390 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893116+0000", 2024-09-11T06:51:30.390 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1629999999999999e-06 2024-09-11T06:51:30.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.390 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:30.390 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893128+0000", 2024-09-11T06:51:30.390 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.11e-05 2024-09-11T06:51:30.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.390 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:30.390 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893156+0000", 2024-09-11T06:51:30.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8266e-05 2024-09-11T06:51:30.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.391 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.391 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893171+0000", 2024-09-11T06:51:30.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5426000000000001e-05 2024-09-11T06:51:30.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.391 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:30.391 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894561+0000", 2024-09-11T06:51:30.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013899959999999999 2024-09-11T06:51:30.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.392 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:30.392 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894781+0000", 2024-09-11T06:51:30.392 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021987100000000001 2024-09-11T06:51:30.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.392 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:30.392 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896349+0000", 2024-09-11T06:51:30.392 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015681779999999999 2024-09-11T06:51:30.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.392 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:30.392 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896458+0000", 2024-09-11T06:51:30.392 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000108187 2024-09-11T06:51:30.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.393 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:30.393 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896616+0000", 2024-09-11T06:51:30.393 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015854899999999999 2024-09-11T06:51:30.393 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.393 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:30.393 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.393 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:176 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch reconnect cookie 139735284585312 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:30.393 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:51:30.393 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 59.442376314000001, 2024-09-11T06:51:30.394 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033620960000000002, 2024-09-11T06:51:30.394 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:30.394 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:30.394 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:30.394 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:30.394 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:51:30.394 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:51:30.394 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 176 2024-09-11T06:51:30.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.394 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:30.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.394 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:30.394 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:51:30.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:30.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924493+0000", 2024-09-11T06:51:30.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:30.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924494+0000", 2024-09-11T06:51:30.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.967e-06 2024-09-11T06:51:30.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:30.396 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924501+0000", 2024-09-11T06:51:30.396 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2659999999999998e-06 2024-09-11T06:51:30.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.396 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:30.396 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924520+0000", 2024-09-11T06:51:30.396 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9038999999999998e-05 2024-09-11T06:51:30.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.396 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.396 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.924535+0000", 2024-09-11T06:51:30.396 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5359999999999999e-05 2024-09-11T06:51:30.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.397 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:30.397 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.925544+0000", 2024-09-11T06:51:30.397 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010085109999999999 2024-09-11T06:51:30.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.397 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.397 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:30.397 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.925671+0000", 2024-09-11T06:51:30.397 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012756599999999999 2024-09-11T06:51:30.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.397 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.397 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:30.397 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.927855+0000", 2024-09-11T06:51:30.397 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0021833870000000002 2024-09-11T06:51:30.397 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.398 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:30.398 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.398 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.398 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.398 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4828.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch watch cookie 140587969811264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-11T06:51:30.398 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:51:30.398 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 54.120673355999998, 2024-09-11T06:51:30.398 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032778859999999998, 2024-09-11T06:51:30.398 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:30.398 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:30.398 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:30.398 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:30.398 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4828", 2024-09-11T06:51:30.398 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2410288650", 2024-09-11T06:51:30.399 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:51:30.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.399 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:30.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.399 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:30.399 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:51:30.399 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.399 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:30.399 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246195+0000", 2024-09-11T06:51:30.399 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.400 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:30.400 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246198+0000", 2024-09-11T06:51:30.400 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.667e-06 2024-09-11T06:51:30.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.400 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.400 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:30.400 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246209+0000", 2024-09-11T06:51:30.400 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0409e-05 2024-09-11T06:51:30.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.400 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.400 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:30.400 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246241+0000", 2024-09-11T06:51:30.400 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2592999999999998e-05 2024-09-11T06:51:30.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.401 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.401 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246260+0000", 2024-09-11T06:51:30.401 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9343000000000001e-05 2024-09-11T06:51:30.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.401 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:30.401 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246466+0000", 2024-09-11T06:51:30.401 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000205701 2024-09-11T06:51:30.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.401 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:30.401 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.246713+0000", 2024-09-11T06:51:30.402 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024646999999999999 2024-09-11T06:51:30.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.402 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.402 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:30.402 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.247133+0000", 2024-09-11T06:51:30.402 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000420536 2024-09-11T06:51:30.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.402 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.402 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:30.402 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.248541+0000", 2024-09-11T06:51:30.402 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001407801 2024-09-11T06:51:30.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.402 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.403 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:30.403 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249241+0000", 2024-09-11T06:51:30.403 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070043500000000001 2024-09-11T06:51:30.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.403 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.403 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:30.403 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249326+0000", 2024-09-11T06:51:30.403 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5037999999999995e-05 2024-09-11T06:51:30.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.403 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.403 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:30.403 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.249473+0000", 2024-09-11T06:51:30.403 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014689299999999999 2024-09-11T06:51:30.403 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.404 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:30.404 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.404 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.404 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:6 3.5 3:b2cc3336:::rbd_id.testrbdnbd21607:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:30.404 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.591899+0000", 2024-09-11T06:51:30.404 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 44.774969407, 2024-09-11T06:51:30.404 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032382999999999999, 2024-09-11T06:51:30.404 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:30.404 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:30.404 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:30.404 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:30.404 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:51:30.404 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:51:30.405 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 6 2024-09-11T06:51:30.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.405 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:30.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.405 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:30.405 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591899+0000", 2024-09-11T06:51:30.405 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.405 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:30.405 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591899+0000", 2024-09-11T06:51:30.405 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.406 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:30.406 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591902+0000", 2024-09-11T06:51:30.406 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0770000000000002e-06 2024-09-11T06:51:30.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.406 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:30.406 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591913+0000", 2024-09-11T06:51:30.406 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1182999999999999e-05 2024-09-11T06:51:30.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.406 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:30.406 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591940+0000", 2024-09-11T06:51:30.407 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7233000000000002e-05 2024-09-11T06:51:30.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.407 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.407 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591954+0000", 2024-09-11T06:51:30.407 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4503e-05 2024-09-11T06:51:30.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.407 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:30.407 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.592071+0000", 2024-09-11T06:51:30.407 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116731 2024-09-11T06:51:30.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.408 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:30.408 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.592274+0000", 2024-09-11T06:51:30.408 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00020320900000000001 2024-09-11T06:51:30.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.408 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.408 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 3", 2024-09-11T06:51:30.408 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.592576+0000", 2024-09-11T06:51:30.408 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030179000000000002 2024-09-11T06:51:30.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.408 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.408 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:30.408 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.594072+0000", 2024-09-11T06:51:30.408 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001495496 2024-09-11T06:51:30.409 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.409 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.409 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:30.409 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.594945+0000", 2024-09-11T06:51:30.409 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00087379799999999996 2024-09-11T06:51:30.409 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.409 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.409 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:30.409 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.595023+0000", 2024-09-11T06:51:30.409 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7751999999999997e-05 2024-09-11T06:51:30.409 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.409 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.409 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:30.409 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.595138+0000", 2024-09-11T06:51:30.410 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000114528 2024-09-11T06:51:30.410 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.410 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:30.410 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.410 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.410 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:44 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:30.410 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.540846+0000", 2024-09-11T06:51:30.410 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.826023274999997, 2024-09-11T06:51:30.410 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032138639999999999, 2024-09-11T06:51:30.410 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:30.410 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:30.410 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:30.411 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:30.411 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:30.411 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:30.411 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 44 2024-09-11T06:51:30.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.411 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:30.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.411 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:30.411 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540846+0000", 2024-09-11T06:51:30.411 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.411 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:30.411 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540846+0000", 2024-09-11T06:51:30.412 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.412 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:30.412 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540848+0000", 2024-09-11T06:51:30.412 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5469999999999999e-06 2024-09-11T06:51:30.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.412 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:30.412 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540863+0000", 2024-09-11T06:51:30.412 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4613000000000001e-05 2024-09-11T06:51:30.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.413 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:30.413 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540895+0000", 2024-09-11T06:51:30.413 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2153000000000002e-05 2024-09-11T06:51:30.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.413 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.413 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.540912+0000", 2024-09-11T06:51:30.413 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7405999999999999e-05 2024-09-11T06:51:30.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.413 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:30.413 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.541111+0000", 2024-09-11T06:51:30.413 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019849599999999999 2024-09-11T06:51:30.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.414 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:30.414 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.541269+0000", 2024-09-11T06:51:30.414 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015792399999999999 2024-09-11T06:51:30.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.414 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:30.414 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.541853+0000", 2024-09-11T06:51:30.414 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00058443799999999995 2024-09-11T06:51:30.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.414 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:30.414 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.543133+0000", 2024-09-11T06:51:30.415 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012797850000000001 2024-09-11T06:51:30.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.415 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.415 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:30.415 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.543905+0000", 2024-09-11T06:51:30.415 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000771655 2024-09-11T06:51:30.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.415 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.415 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:30.415 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.543993+0000", 2024-09-11T06:51:30.415 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8168e-05 2024-09-11T06:51:30.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.415 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.415 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:30.416 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.544059+0000", 2024-09-11T06:51:30.416 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6679000000000003e-05 2024-09-11T06:51:30.416 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.416 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:30.416 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.416 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5553.0:24 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 139650593197904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:30.416 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.566567+0000", 2024-09-11T06:51:30.416 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.800302004999999, 2024-09-11T06:51:30.416 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034350790000000002, 2024-09-11T06:51:30.416 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:30.416 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:30.416 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:30.417 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:30.417 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5553", 2024-09-11T06:51:30.417 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3590511604", 2024-09-11T06:51:30.417 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 24 2024-09-11T06:51:30.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.417 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:30.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.417 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:30.417 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566567+0000", 2024-09-11T06:51:30.417 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.417 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:30.418 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566567+0000", 2024-09-11T06:51:30.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.418 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:30.418 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566569+0000", 2024-09-11T06:51:30.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.973e-06 2024-09-11T06:51:30.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.418 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:30.418 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566578+0000", 2024-09-11T06:51:30.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8170000000000005e-06 2024-09-11T06:51:30.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.419 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:30.419 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566602+0000", 2024-09-11T06:51:30.419 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4596e-05 2024-09-11T06:51:30.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.419 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.419 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566619+0000", 2024-09-11T06:51:30.419 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7099999999999999e-05 2024-09-11T06:51:30.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.419 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:30.419 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.567321+0000", 2024-09-11T06:51:30.419 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070168200000000002 2024-09-11T06:51:30.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.420 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:30.420 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.567454+0000", 2024-09-11T06:51:30.420 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013286400000000001 2024-09-11T06:51:30.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.420 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:30.420 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.567674+0000", 2024-09-11T06:51:30.420 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000219679 2024-09-11T06:51:30.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.420 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:30.420 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.569014+0000", 2024-09-11T06:51:30.421 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013404129999999999 2024-09-11T06:51:30.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.421 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.421 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:30.421 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.569634+0000", 2024-09-11T06:51:30.421 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00062029200000000004 2024-09-11T06:51:30.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.421 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.421 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:30.421 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.569805+0000", 2024-09-11T06:51:30.421 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000170366 2024-09-11T06:51:30.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.421 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.422 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:30.422 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.570002+0000", 2024-09-11T06:51:30.422 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019729700000000001 2024-09-11T06:51:30.422 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.422 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:30.422 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.422 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:82 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch reconnect cookie 139888024456048 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:30.422 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.690185+0000", 2024-09-11T06:51:30.422 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.676684002999998, 2024-09-11T06:51:30.422 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0045884799999999998, 2024-09-11T06:51:30.422 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:30.422 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:30.423 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:30.423 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:30.423 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:30.423 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:30.423 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 82 2024-09-11T06:51:30.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.423 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:30.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.423 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:30.423 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690185+0000", 2024-09-11T06:51:30.423 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.424 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:30.424 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690185+0000", 2024-09-11T06:51:30.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.424 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:30.424 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690186+0000", 2024-09-11T06:51:30.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8400000000000002e-07 2024-09-11T06:51:30.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.424 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:30.424 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690190+0000", 2024-09-11T06:51:30.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5499999999999996e-06 2024-09-11T06:51:30.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.425 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:30.425 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690203+0000", 2024-09-11T06:51:30.425 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2228999999999999e-05 2024-09-11T06:51:30.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.425 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.425 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690228+0000", 2024-09-11T06:51:30.425 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5587000000000001e-05 2024-09-11T06:51:30.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.426 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:30.426 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.691298+0000", 2024-09-11T06:51:30.426 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001070218 2024-09-11T06:51:30.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.426 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.426 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:30.426 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.691511+0000", 2024-09-11T06:51:30.426 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021301299999999999 2024-09-11T06:51:30.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.426 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.426 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:30.427 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.694773+0000", 2024-09-11T06:51:30.427 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003261899 2024-09-11T06:51:30.427 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.427 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:30.427 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.427 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:84 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify cookie 139887823033168] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:30.427 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:51:30.427 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.676592227, 2024-09-11T06:51:30.427 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035622620000000001, 2024-09-11T06:51:30.427 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:30.427 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:30.427 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:30.428 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:30.428 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:30.428 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:30.428 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 84 2024-09-11T06:51:30.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.428 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:30.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.428 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:30.428 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:51:30.428 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.428 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:30.429 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:51:30.429 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.429 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:30.429 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:51:30.429 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5300000000000003e-07 2024-09-11T06:51:30.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.429 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:30.429 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690281+0000", 2024-09-11T06:51:30.429 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9330000000000003e-06 2024-09-11T06:51:30.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.430 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:30.430 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690291+0000", 2024-09-11T06:51:30.430 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7359999999999996e-06 2024-09-11T06:51:30.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.430 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.430 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690298+0000", 2024-09-11T06:51:30.430 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3699999999999997e-06 2024-09-11T06:51:30.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.430 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:30.430 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693380+0000", 2024-09-11T06:51:30.430 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0030811060000000001 2024-09-11T06:51:30.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.431 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:30.431 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693599+0000", 2024-09-11T06:51:30.431 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021947199999999999 2024-09-11T06:51:30.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.431 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:30.431 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693839+0000", 2024-09-11T06:51:30.431 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023989200000000001 2024-09-11T06:51:30.431 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.431 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:30.431 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.432 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5098.0:25 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 140617967560800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:30.432 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.697727+0000", 2024-09-11T06:51:30.432 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.669141650999997, 2024-09-11T06:51:30.432 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035341330000000001, 2024-09-11T06:51:30.432 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:30.432 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:30.432 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:30.432 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:30.433 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5098", 2024-09-11T06:51:30.433 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/844541331", 2024-09-11T06:51:30.433 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 25 2024-09-11T06:51:30.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.433 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:30.433 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.433 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:30.433 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697727+0000", 2024-09-11T06:51:30.433 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.433 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.433 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:30.434 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697727+0000", 2024-09-11T06:51:30.434 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.434 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:30.434 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697729+0000", 2024-09-11T06:51:30.434 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.33e-06 2024-09-11T06:51:30.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.434 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:30.434 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697734+0000", 2024-09-11T06:51:30.434 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6590000000000004e-06 2024-09-11T06:51:30.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.435 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:30.435 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697749+0000", 2024-09-11T06:51:30.435 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4867e-05 2024-09-11T06:51:30.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.435 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.435 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.435 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697759+0000", 2024-09-11T06:51:30.435 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0073e-05 2024-09-11T06:51:30.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.435 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.435 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:30.435 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.698715+0000", 2024-09-11T06:51:30.435 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00095545799999999996 2024-09-11T06:51:30.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.436 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.436 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:30.436 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.698850+0000", 2024-09-11T06:51:30.436 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000135814 2024-09-11T06:51:30.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.436 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.436 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:30.436 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.699085+0000", 2024-09-11T06:51:30.436 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023507199999999999 2024-09-11T06:51:30.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.436 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.436 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:30.437 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.700258+0000", 2024-09-11T06:51:30.437 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011726600000000001 2024-09-11T06:51:30.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.437 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:30.437 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.701061+0000", 2024-09-11T06:51:30.437 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00080256100000000003 2024-09-11T06:51:30.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.437 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:30.437 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.701144+0000", 2024-09-11T06:51:30.437 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3188000000000004e-05 2024-09-11T06:51:30.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.438 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:30.438 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.701261+0000", 2024-09-11T06:51:30.438 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000117451 2024-09-11T06:51:30.438 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.438 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:30.438 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.438 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.438 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5380.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b,call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b,call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b,call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:30.438 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:12.536478+0000", 2024-09-11T06:51:30.438 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 17.830390470000001, 2024-09-11T06:51:30.438 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032220090000000001, 2024-09-11T06:51:30.438 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:30.439 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:30.439 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:30.439 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:30.439 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5380", 2024-09-11T06:51:30.439 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/290149879", 2024-09-11T06:51:30.439 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:51:30.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.439 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:30.439 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.439 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:30.439 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536478+0000", 2024-09-11T06:51:30.439 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.439 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.440 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:30.440 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536478+0000", 2024-09-11T06:51:30.440 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.440 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:30.440 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536481+0000", 2024-09-11T06:51:30.440 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4059999999999999e-06 2024-09-11T06:51:30.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.440 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:30.440 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536493+0000", 2024-09-11T06:51:30.440 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2003e-05 2024-09-11T06:51:30.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.441 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.441 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:30.441 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536526+0000", 2024-09-11T06:51:30.441 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2839999999999997e-05 2024-09-11T06:51:30.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.441 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.441 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.441 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536542+0000", 2024-09-11T06:51:30.441 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6120000000000002e-05 2024-09-11T06:51:30.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.441 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.441 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:30.442 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536662+0000", 2024-09-11T06:51:30.442 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000120285 2024-09-11T06:51:30.442 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.442 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.442 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:30.442 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.536946+0000", 2024-09-11T06:51:30.442 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00028412400000000002 2024-09-11T06:51:30.442 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.442 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.442 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:30.442 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:12.539700+0000", 2024-09-11T06:51:30.442 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027542310000000002 2024-09-11T06:51:30.442 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.442 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:30.443 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.443 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.443 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.6105.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b,call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b,call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b,call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-11T06:51:30.443 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:20.630296+0000", 2024-09-11T06:51:30.443 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.7365727260000003, 2024-09-11T06:51:30.443 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003874653, 2024-09-11T06:51:30.443 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:30.443 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:30.443 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:30.443 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:30.443 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.6105", 2024-09-11T06:51:30.443 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/722227396", 2024-09-11T06:51:30.444 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:51:30.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.444 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:30.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.444 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:30.444 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.630296+0000", 2024-09-11T06:51:30.444 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.444 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:30.444 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.630296+0000", 2024-09-11T06:51:30.444 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.445 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:30.445 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.630298+0000", 2024-09-11T06:51:30.445 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2500000000000001e-06 2024-09-11T06:51:30.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.445 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:30.445 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.630317+0000", 2024-09-11T06:51:30.445 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9043e-05 2024-09-11T06:51:30.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.445 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:30.445 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.630349+0000", 2024-09-11T06:51:30.445 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1890000000000001e-05 2024-09-11T06:51:30.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.446 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.446 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.446 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.630368+0000", 2024-09-11T06:51:30.446 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8533e-05 2024-09-11T06:51:30.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.446 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.446 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:30.446 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.630500+0000", 2024-09-11T06:51:30.446 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013175900000000001 2024-09-11T06:51:30.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.446 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.446 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:30.446 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.630806+0000", 2024-09-11T06:51:30.446 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030631700000000002 2024-09-11T06:51:30.447 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.447 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.447 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:30.447 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.634171+0000", 2024-09-11T06:51:30.447 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033648609999999998 2024-09-11T06:51:30.447 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.447 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:30.447 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.447 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.447 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.447 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.6105.0:20 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.metadata_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-11T06:51:30.447 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:20.648867+0000", 2024-09-11T06:51:30.447 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.7180017139999997, 2024-09-11T06:51:30.447 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035862189999999999, 2024-09-11T06:51:30.447 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:30.448 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:30.448 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:30.448 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:30.448 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.6105", 2024-09-11T06:51:30.448 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/722227396", 2024-09-11T06:51:30.448 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 20 2024-09-11T06:51:30.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.448 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:30.448 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.448 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:30.448 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.648867+0000", 2024-09-11T06:51:30.448 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.449 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:30.449 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.648867+0000", 2024-09-11T06:51:30.449 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.449 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:30.449 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.648868+0000", 2024-09-11T06:51:30.449 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5000000000000001e-07 2024-09-11T06:51:30.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.449 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:30.449 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.648873+0000", 2024-09-11T06:51:30.449 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2970000000000003e-06 2024-09-11T06:51:30.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.450 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:30.450 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.648889+0000", 2024-09-11T06:51:30.450 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5549999999999999e-05 2024-09-11T06:51:30.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.450 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.450 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.648899+0000", 2024-09-11T06:51:30.450 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0516000000000001e-05 2024-09-11T06:51:30.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.450 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:30.451 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.649514+0000", 2024-09-11T06:51:30.451 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00061461800000000004 2024-09-11T06:51:30.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.451 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.451 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:30.451 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.649816+0000", 2024-09-11T06:51:30.451 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030191700000000002 2024-09-11T06:51:30.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.451 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.451 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:30.451 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.650295+0000", 2024-09-11T06:51:30.451 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00047892799999999999 2024-09-11T06:51:30.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.452 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.452 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:30.452 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.651725+0000", 2024-09-11T06:51:30.452 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014304630000000001 2024-09-11T06:51:30.452 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.452 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.452 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:30.452 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.652187+0000", 2024-09-11T06:51:30.452 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00046161299999999999 2024-09-11T06:51:30.452 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.452 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.452 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:30.452 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.652279+0000", 2024-09-11T06:51:30.452 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2025000000000001e-05 2024-09-11T06:51:30.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.453 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.453 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:30.453 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.652453+0000", 2024-09-11T06:51:30.453 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017434199999999999 2024-09-11T06:51:30.453 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.453 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:30.453 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.453 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.453 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.6105.0:21 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-11T06:51:30.453 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:20.649631+0000", 2024-09-11T06:51:30.453 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.7172373709999995, 2024-09-11T06:51:30.453 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034058339999999999, 2024-09-11T06:51:30.454 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:30.454 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:30.454 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:30.454 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:30.454 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.6105", 2024-09-11T06:51:30.454 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/722227396", 2024-09-11T06:51:30.454 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 21 2024-09-11T06:51:30.454 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.454 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:30.454 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.454 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:30.454 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.649631+0000", 2024-09-11T06:51:30.454 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.455 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:30.455 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.649631+0000", 2024-09-11T06:51:30.455 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.455 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:30.455 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.649634+0000", 2024-09-11T06:51:30.455 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.04e-06 2024-09-11T06:51:30.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.455 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:30.456 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.649644+0000", 2024-09-11T06:51:30.456 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.061e-05 2024-09-11T06:51:30.456 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.456 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.456 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:30.456 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.649671+0000", 2024-09-11T06:51:30.456 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6679e-05 2024-09-11T06:51:30.456 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.456 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.456 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.456 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.649690+0000", 2024-09-11T06:51:30.456 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9573999999999999e-05 2024-09-11T06:51:30.456 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.456 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.457 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:30.457 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.651104+0000", 2024-09-11T06:51:30.457 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001413739 2024-09-11T06:51:30.457 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.457 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.457 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:30.457 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.651336+0000", 2024-09-11T06:51:30.457 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023153799999999999 2024-09-11T06:51:30.457 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.457 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.457 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:30.457 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.652555+0000", 2024-09-11T06:51:30.457 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001219765 2024-09-11T06:51:30.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.458 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.458 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:30.458 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.652705+0000", 2024-09-11T06:51:30.458 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000149445 2024-09-11T06:51:30.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.458 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.458 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:30.458 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.653037+0000", 2024-09-11T06:51:30.458 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033244399999999999 2024-09-11T06:51:30.458 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.458 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:30.458 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.459 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.459 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.6105.0:33 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 140610048726272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-11T06:51:30.459 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:20.671124+0000", 2024-09-11T06:51:30.459 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.6957453230000006, 2024-09-11T06:51:30.459 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035064290000000001, 2024-09-11T06:51:30.459 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:30.459 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:30.459 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:30.459 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:30.459 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.6105", 2024-09-11T06:51:30.459 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/722227396", 2024-09-11T06:51:30.459 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 33 2024-09-11T06:51:30.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.460 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:30.460 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.460 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:30.460 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.671124+0000", 2024-09-11T06:51:30.460 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.460 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.460 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:30.460 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.671124+0000", 2024-09-11T06:51:30.460 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.460 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.460 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:30.460 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.671124+0000", 2024-09-11T06:51:30.461 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7600000000000006e-07 2024-09-11T06:51:30.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.461 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:30.461 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.671129+0000", 2024-09-11T06:51:30.461 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7330000000000003e-06 2024-09-11T06:51:30.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.461 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:30.461 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.671145+0000", 2024-09-11T06:51:30.461 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5367e-05 2024-09-11T06:51:30.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.461 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.671154+0000", 2024-09-11T06:51:30.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.363e-06 2024-09-11T06:51:30.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:30.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.671797+0000", 2024-09-11T06:51:30.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00064351000000000002 2024-09-11T06:51:30.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:30.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.671943+0000", 2024-09-11T06:51:30.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000145699 2024-09-11T06:51:30.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:30.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.672240+0000", 2024-09-11T06:51:30.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00029651100000000001 2024-09-11T06:51:30.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:30.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.673194+0000", 2024-09-11T06:51:30.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00095448699999999996 2024-09-11T06:51:30.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:30.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.674221+0000", 2024-09-11T06:51:30.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001026714 2024-09-11T06:51:30.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:30.464 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.674327+0000", 2024-09-11T06:51:30.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000105732 2024-09-11T06:51:30.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:30.464 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.674630+0000", 2024-09-11T06:51:30.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030333699999999999 2024-09-11T06:51:30.464 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.464 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:30.465 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.465 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5542.0:3 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-11T06:51:30.465 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:20.994530+0000", 2024-09-11T06:51:30.465 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.3723388750000005, 2024-09-11T06:51:30.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.99603794099999998, 2024-09-11T06:51:30.465 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:30.465 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:30.465 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:30.465 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:30.465 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5542", 2024-09-11T06:51:30.465 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1546942093", 2024-09-11T06:51:30.465 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 3 2024-09-11T06:51:30.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.466 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:30.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.466 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:30.466 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.994530+0000", 2024-09-11T06:51:30.466 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.466 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:30.466 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.994530+0000", 2024-09-11T06:51:30.466 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.466 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:30.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.994532+0000", 2024-09-11T06:51:30.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9740000000000001e-06 2024-09-11T06:51:30.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:30.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.994543+0000", 2024-09-11T06:51:30.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1066999999999999e-05 2024-09-11T06:51:30.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:30.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.994558+0000", 2024-09-11T06:51:30.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4987999999999999e-05 2024-09-11T06:51:30.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.994572+0000", 2024-09-11T06:51:30.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3974e-05 2024-09-11T06:51:30.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:30.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.994659+0000", 2024-09-11T06:51:30.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7491000000000006e-05 2024-09-11T06:51:30.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-11T06:51:30.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.994675+0000", 2024-09-11T06:51:30.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.506e-05 2024-09-11T06:51:30.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.469 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:30.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:21.989532+0000", 2024-09-11T06:51:30.469 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.99485760400000001 2024-09-11T06:51:30.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.469 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:30.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:21.989833+0000", 2024-09-11T06:51:30.469 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030123699999999999 2024-09-11T06:51:30.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.469 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:30.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:21.990568+0000", 2024-09-11T06:51:30.470 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00073454599999999998 2024-09-11T06:51:30.470 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.470 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:30.470 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.470 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.470 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.470 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5542.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b,call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b,call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b,call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-11T06:51:30.470 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:22.009725+0000", 2024-09-11T06:51:30.470 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.3571439339999998, 2024-09-11T06:51:30.470 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033114450000000001, 2024-09-11T06:51:30.470 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:30.470 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:30.470 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:30.470 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:30.471 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5542", 2024-09-11T06:51:30.471 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1546942093", 2024-09-11T06:51:30.471 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:51:30.471 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.471 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:30.471 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.471 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:30.471 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:22.009725+0000", 2024-09-11T06:51:30.471 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.471 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.471 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.471 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:30.471 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:22.009725+0000", 2024-09-11T06:51:30.471 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.472 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.472 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.472 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:30.472 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:22.009727+0000", 2024-09-11T06:51:30.472 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.413e-06 2024-09-11T06:51:30.472 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.472 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.472 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:30.472 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:22.009740+0000", 2024-09-11T06:51:30.472 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2313000000000001e-05 2024-09-11T06:51:30.472 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.472 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.472 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:30.473 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:22.009771+0000", 2024-09-11T06:51:30.473 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1529999999999998e-05 2024-09-11T06:51:30.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.473 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.473 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.473 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:22.009787+0000", 2024-09-11T06:51:30.473 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6303999999999998e-05 2024-09-11T06:51:30.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.473 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.473 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:30.473 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:22.009913+0000", 2024-09-11T06:51:30.473 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000125115 2024-09-11T06:51:30.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.473 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.474 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:30.474 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:22.010200+0000", 2024-09-11T06:51:30.474 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00028791400000000001 2024-09-11T06:51:30.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.474 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.474 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:30.474 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:22.013036+0000", 2024-09-11T06:51:30.474 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028358559999999999 2024-09-11T06:51:30.474 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.474 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:30.474 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.474 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.474 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5660.0:26 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e107)", 2024-09-11T06:51:30.475 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:30.104530+0000", 2024-09-11T06:51:30.475 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.26233901100000001, 2024-09-11T06:51:30.475 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034986409999999998, 2024-09-11T06:51:30.475 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:30.475 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:30.475 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:30.475 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:30.475 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5660", 2024-09-11T06:51:30.475 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3214439167", 2024-09-11T06:51:30.475 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:51:30.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.475 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:30.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.476 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:30.476 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:30.104530+0000", 2024-09-11T06:51:30.476 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.476 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.476 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:30.476 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:30.104530+0000", 2024-09-11T06:51:30.476 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:30.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.476 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.476 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:30.476 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:30.104532+0000", 2024-09-11T06:51:30.476 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7990000000000001e-06 2024-09-11T06:51:30.477 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.477 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:30.477 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:30.104539+0000", 2024-09-11T06:51:30.477 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4109999999999999e-06 2024-09-11T06:51:30.477 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.477 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:30.477 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:30.104553+0000", 2024-09-11T06:51:30.477 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4228e-05 2024-09-11T06:51:30.477 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.477 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.477 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:30.104562+0000", 2024-09-11T06:51:30.478 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7830000000000004e-06 2024-09-11T06:51:30.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.478 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.478 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:30.478 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:30.105488+0000", 2024-09-11T06:51:30.478 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00092615299999999998 2024-09-11T06:51:30.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.478 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.478 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:30.478 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:30.105588+0000", 2024-09-11T06:51:30.478 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010027299999999999 2024-09-11T06:51:30.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.478 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.478 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:30.479 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:30.105962+0000", 2024-09-11T06:51:30.479 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00037397299999999999 2024-09-11T06:51:30.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.479 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.479 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:30.479 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:30.107613+0000", 2024-09-11T06:51:30.479 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0016503119999999999 2024-09-11T06:51:30.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.479 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.479 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:30.479 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:30.107903+0000", 2024-09-11T06:51:30.479 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00028990999999999999 2024-09-11T06:51:30.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.479 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.480 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:30.480 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:30.107975+0000", 2024-09-11T06:51:30.480 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2014999999999994e-05 2024-09-11T06:51:30.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:30.480 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:30.480 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:30.480 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:30.108028+0000", 2024-09-11T06:51:30.480 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3783999999999999e-05 2024-09-11T06:51:30.480 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.480 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:30.480 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.480 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:30.480 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:30.481 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:30.482 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:51:30.483 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:30.478+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:30.583 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:30.578+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:30.614 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:30.614 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:30.615 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:51:30.615 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:30.629 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:51:30.684 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:30.682+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:30.746 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:30.746 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:30.747 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:51:30.747 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:51:30.747 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:30.761 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:51:30.805 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:30.801+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:30.887 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:30.886+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:30.888 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:30.888 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:51:30.888 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:51:30.888 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:51:30.888 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.888 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:30.889 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:51:30.889 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 144.61195526, 2024-09-11T06:51:30.889 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283336, 2024-09-11T06:51:30.889 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:30.889 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:30.889 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:30.889 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:30.889 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:30.889 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:30.889 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 87 2024-09-11T06:51:30.889 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.889 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:30.890 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.890 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:30.890 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:51:30.890 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.890 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.890 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.890 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:30.890 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271767+0000", 2024-09-11T06:51:30.890 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.890 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.890 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.890 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:30.890 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271768+0000", 2024-09-11T06:51:30.890 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7599999999999996e-07 2024-09-11T06:51:30.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.891 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.891 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:30.891 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271860+0000", 2024-09-11T06:51:30.891 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2639999999999994e-05 2024-09-11T06:51:30.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.891 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.891 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:30.891 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271873+0000", 2024-09-11T06:51:30.891 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3139000000000001e-05 2024-09-11T06:51:30.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.891 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.891 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.891 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271882+0000", 2024-09-11T06:51:30.892 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2369999999999992e-06 2024-09-11T06:51:30.892 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.892 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.892 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:30.892 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277150+0000", 2024-09-11T06:51:30.892 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052684419999999999 2024-09-11T06:51:30.892 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.892 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.892 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:30.892 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277273+0000", 2024-09-11T06:51:30.892 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012278899999999999 2024-09-11T06:51:30.892 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.892 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.893 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:30.893 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.277466+0000", 2024-09-11T06:51:30.893 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000193578 2024-09-11T06:51:30.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.893 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.893 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:30.893 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.280808+0000", 2024-09-11T06:51:30.893 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003341971 2024-09-11T06:51:30.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.893 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.893 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:30.893 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284419+0000", 2024-09-11T06:51:30.893 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003610982 2024-09-11T06:51:30.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.894 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.894 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:30.894 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284531+0000", 2024-09-11T06:51:30.894 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011121299999999999 2024-09-11T06:51:30.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.894 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.894 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:30.894 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284600+0000", 2024-09-11T06:51:30.894 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.9592999999999996e-05 2024-09-11T06:51:30.894 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.894 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:30.894 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.894 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.895 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:30.895 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:51:30.895 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 144.61182500800001, 2024-09-11T06:51:30.895 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01283314, 2024-09-11T06:51:30.895 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:30.895 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:30.895 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:30.895 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:30.895 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:30.895 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:30.895 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 88 2024-09-11T06:51:30.895 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.895 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:30.895 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.896 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:30.896 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:51:30.896 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.896 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.896 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:30.896 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271897+0000", 2024-09-11T06:51:30.896 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.896 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.896 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:30.896 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271898+0000", 2024-09-11T06:51:30.896 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2999999999999999e-07 2024-09-11T06:51:30.897 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.897 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.897 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:30.897 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271904+0000", 2024-09-11T06:51:30.897 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.767e-06 2024-09-11T06:51:30.897 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.897 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.897 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:30.897 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271913+0000", 2024-09-11T06:51:30.897 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7200000000000001e-06 2024-09-11T06:51:30.897 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.897 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.897 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.898 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.271920+0000", 2024-09-11T06:51:30.898 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7100000000000001e-06 2024-09-11T06:51:30.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.898 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.898 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:30.898 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278684+0000", 2024-09-11T06:51:30.898 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0067642650000000002 2024-09-11T06:51:30.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.898 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.898 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:30.898 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.278838+0000", 2024-09-11T06:51:30.898 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015371499999999999 2024-09-11T06:51:30.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.898 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.898 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:30.899 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.279040+0000", 2024-09-11T06:51:30.899 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020207499999999999 2024-09-11T06:51:30.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.899 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:30.899 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.281448+0000", 2024-09-11T06:51:30.899 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002407686 2024-09-11T06:51:30.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.899 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:30.899 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284615+0000", 2024-09-11T06:51:30.899 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031675499999999999 2024-09-11T06:51:30.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.900 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.900 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:30.900 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284682+0000", 2024-09-11T06:51:30.900 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7032000000000005e-05 2024-09-11T06:51:30.900 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.900 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.900 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:30.900 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.284730+0000", 2024-09-11T06:51:30.900 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7790000000000002e-05 2024-09-11T06:51:30.900 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.900 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:30.900 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.900 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.900 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.901 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:30.901 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:51:30.901 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 144.442975808, 2024-09-11T06:51:30.901 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014944941, 2024-09-11T06:51:30.901 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:30.901 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:30.901 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:30.901 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:30.901 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:30.901 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:30.901 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 111 2024-09-11T06:51:30.901 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.901 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:30.902 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.902 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:30.902 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:51:30.902 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.902 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.902 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.902 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:30.902 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:51:30.902 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.902 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.902 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.902 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:30.902 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440748+0000", 2024-09-11T06:51:30.903 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:51:30.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.903 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.903 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:30.903 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440760+0000", 2024-09-11T06:51:30.903 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1897e-05 2024-09-11T06:51:30.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.903 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.903 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:30.903 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440782+0000", 2024-09-11T06:51:30.903 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2552999999999999e-05 2024-09-11T06:51:30.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.903 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.904 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.904 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440793+0000", 2024-09-11T06:51:30.904 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0643e-05 2024-09-11T06:51:30.904 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.904 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.904 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:30.904 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449755+0000", 2024-09-11T06:51:30.904 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089622620000000004 2024-09-11T06:51:30.904 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.904 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.904 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:30.904 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449918+0000", 2024-09-11T06:51:30.904 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163086 2024-09-11T06:51:30.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.905 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.905 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:30.905 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450150+0000", 2024-09-11T06:51:30.905 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023204500000000001 2024-09-11T06:51:30.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.905 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.905 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:30.905 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454533+0000", 2024-09-11T06:51:30.905 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0043831349999999998 2024-09-11T06:51:30.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.905 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.905 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:30.906 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455511+0000", 2024-09-11T06:51:30.906 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0009776050000000001 2024-09-11T06:51:30.906 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.906 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.906 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:30.906 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455597+0000", 2024-09-11T06:51:30.906 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6446000000000005e-05 2024-09-11T06:51:30.906 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.906 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.906 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:30.906 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455691+0000", 2024-09-11T06:51:30.906 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3832000000000006e-05 2024-09-11T06:51:30.906 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.907 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:30.907 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.907 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.907 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.907 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:30.907 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:51:30.907 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 144.442906045, 2024-09-11T06:51:30.907 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018885411000000001, 2024-09-11T06:51:30.907 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:30.907 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:30.907 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:30.907 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:30.907 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:30.907 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:30.908 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 112 2024-09-11T06:51:30.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.908 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:30.908 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.908 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:30.908 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:51:30.908 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.908 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.908 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:30.908 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:51:30.908 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.908 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.909 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:30.909 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440817+0000", 2024-09-11T06:51:30.909 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:51:30.909 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.909 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.909 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:30.909 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441003+0000", 2024-09-11T06:51:30.909 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018661200000000001 2024-09-11T06:51:30.909 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.909 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.909 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:30.909 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441027+0000", 2024-09-11T06:51:30.909 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3920000000000001e-05 2024-09-11T06:51:30.910 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.910 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.910 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.910 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441062+0000", 2024-09-11T06:51:30.910 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4663000000000003e-05 2024-09-11T06:51:30.910 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.910 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.910 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:30.910 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451981+0000", 2024-09-11T06:51:30.910 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010918926000000001 2024-09-11T06:51:30.910 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.910 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.910 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:30.911 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452111+0000", 2024-09-11T06:51:30.911 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129529 2024-09-11T06:51:30.911 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.911 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.911 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:30.911 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452311+0000", 2024-09-11T06:51:30.911 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020013799999999999 2024-09-11T06:51:30.911 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.911 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.911 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:30.911 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455809+0000", 2024-09-11T06:51:30.911 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003498443 2024-09-11T06:51:30.911 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.912 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.912 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:30.912 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459566+0000", 2024-09-11T06:51:30.912 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003757271 2024-09-11T06:51:30.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.912 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.912 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:30.912 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459640+0000", 2024-09-11T06:51:30.912 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3949000000000006e-05 2024-09-11T06:51:30.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.912 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.912 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:30.912 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459701+0000", 2024-09-11T06:51:30.913 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1117000000000005e-05 2024-09-11T06:51:30.913 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.913 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:30.913 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.913 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.913 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.913 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:30.913 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:51:30.913 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 144.44262719400001, 2024-09-11T06:51:30.913 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018760915, 2024-09-11T06:51:30.913 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:30.913 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:30.913 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:30.914 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:30.914 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:30.914 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:30.914 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 113 2024-09-11T06:51:30.914 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.914 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:30.914 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.914 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:30.914 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:51:30.914 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.914 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.914 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.914 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:30.914 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:51:30.914 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.915 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.915 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.915 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:30.915 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441096+0000", 2024-09-11T06:51:30.915 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4130000000000001e-06 2024-09-11T06:51:30.915 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.915 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.915 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:30.915 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441110+0000", 2024-09-11T06:51:30.915 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3900000000000001e-05 2024-09-11T06:51:30.915 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.915 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.915 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:30.915 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441131+0000", 2024-09-11T06:51:30.916 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0636999999999999e-05 2024-09-11T06:51:30.916 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.916 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.916 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.916 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441147+0000", 2024-09-11T06:51:30.916 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6512999999999999e-05 2024-09-11T06:51:30.916 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.916 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.916 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:30.916 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453274+0000", 2024-09-11T06:51:30.916 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012126764999999999 2024-09-11T06:51:30.916 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.916 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.916 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:30.917 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453498+0000", 2024-09-11T06:51:30.917 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000223568 2024-09-11T06:51:30.917 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.917 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.917 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:30.917 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453872+0000", 2024-09-11T06:51:30.917 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037467099999999998 2024-09-11T06:51:30.917 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.917 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.917 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:30.917 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458709+0000", 2024-09-11T06:51:30.917 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0048365680000000003 2024-09-11T06:51:30.918 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.918 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.918 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:30.918 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459716+0000", 2024-09-11T06:51:30.918 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010075240000000001 2024-09-11T06:51:30.918 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.918 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.918 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:30.918 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459779+0000", 2024-09-11T06:51:30.918 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2336999999999996e-05 2024-09-11T06:51:30.918 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.918 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.918 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:30.919 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459856+0000", 2024-09-11T06:51:30.919 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7019000000000002e-05 2024-09-11T06:51:30.919 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.919 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:30.919 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.919 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.919 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.919 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:30.919 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:51:30.919 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 144.442550455, 2024-09-11T06:51:30.919 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018963114, 2024-09-11T06:51:30.919 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:30.919 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:30.919 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:30.920 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:30.920 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:30.920 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:30.920 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 114 2024-09-11T06:51:30.920 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.920 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:30.920 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.920 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:30.920 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:51:30.920 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.920 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.920 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.920 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:30.921 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:51:30.921 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.921 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.921 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.921 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:30.921 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:51:30.921 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9299999999999996e-07 2024-09-11T06:51:30.921 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.921 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.921 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:30.921 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441180+0000", 2024-09-11T06:51:30.921 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7200000000000006e-06 2024-09-11T06:51:30.921 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.921 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.922 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:30.922 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441265+0000", 2024-09-11T06:51:30.922 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4753000000000007e-05 2024-09-11T06:51:30.922 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.922 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.922 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.922 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441288+0000", 2024-09-11T06:51:30.922 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2860000000000001e-05 2024-09-11T06:51:30.922 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.922 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.922 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:30.922 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454585+0000", 2024-09-11T06:51:30.922 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013297286 2024-09-11T06:51:30.923 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.923 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.923 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:30.923 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454714+0000", 2024-09-11T06:51:30.923 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000128465 2024-09-11T06:51:30.923 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.923 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.923 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:30.923 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454909+0000", 2024-09-11T06:51:30.923 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019522499999999999 2024-09-11T06:51:30.923 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.923 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.923 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:30.924 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459868+0000", 2024-09-11T06:51:30.924 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0049591239999999997 2024-09-11T06:51:30.924 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.924 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.924 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:30.924 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459988+0000", 2024-09-11T06:51:30.924 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000120532 2024-09-11T06:51:30.924 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.924 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.924 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:30.924 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460071+0000", 2024-09-11T06:51:30.924 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2256000000000007e-05 2024-09-11T06:51:30.924 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.925 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.925 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:30.925 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460135+0000", 2024-09-11T06:51:30.925 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3999999999999997e-05 2024-09-11T06:51:30.925 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.925 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:30.925 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.925 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.925 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.925 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:30.925 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:51:30.925 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 144.442392826, 2024-09-11T06:51:30.925 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.019036849000000002, 2024-09-11T06:51:30.925 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:30.926 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:30.926 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:30.926 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:30.926 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:30.926 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:30.926 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 115 2024-09-11T06:51:30.926 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.926 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:30.926 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.926 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:30.926 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:51:30.926 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.926 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.927 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.927 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:30.927 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:51:30.927 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.927 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.927 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.927 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:30.927 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441330+0000", 2024-09-11T06:51:30.927 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1930000000000001e-06 2024-09-11T06:51:30.927 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.928 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.928 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:30.928 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441544+0000", 2024-09-11T06:51:30.928 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021405799999999999 2024-09-11T06:51:30.928 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.928 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.928 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:30.928 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441572+0000", 2024-09-11T06:51:30.928 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7317e-05 2024-09-11T06:51:30.928 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.928 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.928 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.928 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441593+0000", 2024-09-11T06:51:30.929 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1702999999999999e-05 2024-09-11T06:51:30.929 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.929 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.929 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:30.929 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455891+0000", 2024-09-11T06:51:30.929 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01429708 2024-09-11T06:51:30.929 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.929 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.929 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:30.929 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456046+0000", 2024-09-11T06:51:30.929 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000155682 2024-09-11T06:51:30.929 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.929 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.930 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:30.930 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456242+0000", 2024-09-11T06:51:30.930 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000195692 2024-09-11T06:51:30.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.930 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.930 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:30.930 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459876+0000", 2024-09-11T06:51:30.930 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036336480000000002 2024-09-11T06:51:30.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.930 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.930 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:30.930 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460245+0000", 2024-09-11T06:51:30.930 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036901700000000002 2024-09-11T06:51:30.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.931 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.931 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:30.931 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460316+0000", 2024-09-11T06:51:30.931 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1846000000000003e-05 2024-09-11T06:51:30.931 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.931 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.931 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:30.931 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460366+0000", 2024-09-11T06:51:30.931 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9613000000000001e-05 2024-09-11T06:51:30.931 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.931 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:30.931 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.931 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.932 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.932 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:30.932 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:51:30.932 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 144.44207074600001, 2024-09-11T06:51:30.932 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01904432, 2024-09-11T06:51:30.932 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:30.932 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:30.932 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:30.932 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:30.932 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:30.932 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:30.932 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 116 2024-09-11T06:51:30.932 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.933 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:30.933 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.933 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:30.933 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:51:30.933 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.933 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.933 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.933 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:30.933 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:51:30.933 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.933 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.933 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.933 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:30.933 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441653+0000", 2024-09-11T06:51:30.934 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:51:30.934 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.934 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.934 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:30.934 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441665+0000", 2024-09-11T06:51:30.934 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1653e-05 2024-09-11T06:51:30.934 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.934 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.934 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:30.934 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441684+0000", 2024-09-11T06:51:30.934 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9490000000000001e-05 2024-09-11T06:51:30.934 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.934 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.935 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.935 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441699+0000", 2024-09-11T06:51:30.935 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5044e-05 2024-09-11T06:51:30.935 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.935 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.935 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:30.935 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457297+0000", 2024-09-11T06:51:30.935 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015598241000000001 2024-09-11T06:51:30.935 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.935 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.935 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:30.935 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457577+0000", 2024-09-11T06:51:30.935 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027930799999999998 2024-09-11T06:51:30.935 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.936 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.936 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:30.936 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457957+0000", 2024-09-11T06:51:30.936 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00038088 2024-09-11T06:51:30.936 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.936 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.936 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:30.936 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460432+0000", 2024-09-11T06:51:30.936 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024743349999999998 2024-09-11T06:51:30.936 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.936 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.936 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:30.936 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460584+0000", 2024-09-11T06:51:30.937 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015232900000000001 2024-09-11T06:51:30.937 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.937 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.937 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:30.937 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460646+0000", 2024-09-11T06:51:30.937 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1883000000000004e-05 2024-09-11T06:51:30.937 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.937 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.937 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:30.937 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460696+0000", 2024-09-11T06:51:30.937 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9589999999999998e-05 2024-09-11T06:51:30.937 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.937 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:30.937 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.938 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.938 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.938 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:30.938 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:51:30.938 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 144.44199868000001, 2024-09-11T06:51:30.938 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.021834473, 2024-09-11T06:51:30.938 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:30.938 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:30.938 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:30.938 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:30.938 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:30.938 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:30.938 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 117 2024-09-11T06:51:30.939 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.939 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:30.939 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.939 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:30.939 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:51:30.939 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.939 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.939 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.939 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:30.939 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:51:30.939 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.939 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.939 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.939 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:30.940 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441725+0000", 2024-09-11T06:51:30.940 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5069999999999999e-06 2024-09-11T06:51:30.940 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.940 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.940 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:30.940 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441738+0000", 2024-09-11T06:51:30.940 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2813000000000001e-05 2024-09-11T06:51:30.940 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.940 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.940 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:30.940 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441759+0000", 2024-09-11T06:51:30.940 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1246999999999998e-05 2024-09-11T06:51:30.941 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.941 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.941 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.941 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441779+0000", 2024-09-11T06:51:30.941 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0392999999999999e-05 2024-09-11T06:51:30.941 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.941 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.941 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:30.941 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458804+0000", 2024-09-11T06:51:30.941 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.017024182999999998 2024-09-11T06:51:30.941 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.941 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.941 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:30.942 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458977+0000", 2024-09-11T06:51:30.942 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000173926 2024-09-11T06:51:30.942 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.942 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.942 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:30.942 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459170+0000", 2024-09-11T06:51:30.942 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000192184 2024-09-11T06:51:30.942 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.942 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.942 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:30.942 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.462300+0000", 2024-09-11T06:51:30.942 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031304969999999999 2024-09-11T06:51:30.942 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.942 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.943 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:30.943 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463408+0000", 2024-09-11T06:51:30.943 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011077839999999999 2024-09-11T06:51:30.943 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.943 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.943 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:30.943 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463497+0000", 2024-09-11T06:51:30.943 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8828999999999999e-05 2024-09-11T06:51:30.943 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.943 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.943 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:30.943 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463558+0000", 2024-09-11T06:51:30.943 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1110000000000003e-05 2024-09-11T06:51:30.943 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.943 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:30.944 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.944 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.944 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.944 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:30.944 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:51:30.944 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 144.065357178, 2024-09-11T06:51:30.944 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013136297, 2024-09-11T06:51:30.944 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:30.944 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:30.944 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:30.944 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:30.944 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:30.944 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:30.945 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 138 2024-09-11T06:51:30.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.945 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:30.945 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.945 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:30.945 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:51:30.945 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.945 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.945 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:30.945 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:51:30.945 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.946 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.946 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:30.946 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:51:30.946 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.89e-07 2024-09-11T06:51:30.946 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.946 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.946 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:30.946 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818369+0000", 2024-09-11T06:51:30.946 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4170000000000001e-06 2024-09-11T06:51:30.946 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.946 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.946 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:30.946 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818376+0000", 2024-09-11T06:51:30.947 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4270000000000002e-06 2024-09-11T06:51:30.947 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.947 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.947 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.947 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818383+0000", 2024-09-11T06:51:30.947 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3060000000000003e-06 2024-09-11T06:51:30.947 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.947 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.947 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:30.947 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824783+0000", 2024-09-11T06:51:30.947 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064003020000000001 2024-09-11T06:51:30.947 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.947 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.948 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:30.948 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825013+0000", 2024-09-11T06:51:30.948 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023039499999999999 2024-09-11T06:51:30.948 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.948 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.948 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:30.948 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825484+0000", 2024-09-11T06:51:30.948 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000470562 2024-09-11T06:51:30.948 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.948 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.948 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:30.948 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828633+0000", 2024-09-11T06:51:30.949 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031496689999999999 2024-09-11T06:51:30.949 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.949 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.949 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:30.949 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831413+0000", 2024-09-11T06:51:30.949 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027793219999999999 2024-09-11T06:51:30.949 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.949 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.949 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:30.949 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831450+0000", 2024-09-11T06:51:30.949 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7370999999999998e-05 2024-09-11T06:51:30.949 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.949 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.949 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:30.950 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831501+0000", 2024-09-11T06:51:30.950 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1036999999999999e-05 2024-09-11T06:51:30.950 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.950 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:30.950 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.950 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.950 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.950 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:30.950 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:51:30.950 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 144.06533269799999, 2024-09-11T06:51:30.950 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013223011, 2024-09-11T06:51:30.950 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:30.950 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:30.951 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:30.951 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:30.951 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:30.951 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:30.951 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 139 2024-09-11T06:51:30.951 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.951 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:30.951 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.951 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:30.951 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:51:30.951 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.951 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.951 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.952 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:30.952 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:51:30.952 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.952 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.952 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.952 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:30.952 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818390+0000", 2024-09-11T06:51:30.952 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5600000000000001e-07 2024-09-11T06:51:30.952 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.952 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.952 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:30.952 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818394+0000", 2024-09-11T06:51:30.952 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5929999999999999e-06 2024-09-11T06:51:30.952 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.953 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.953 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:30.953 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818399+0000", 2024-09-11T06:51:30.953 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7740000000000004e-06 2024-09-11T06:51:30.953 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.953 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.953 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.953 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818412+0000", 2024-09-11T06:51:30.953 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2625e-05 2024-09-11T06:51:30.953 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.953 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.953 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:30.953 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826205+0000", 2024-09-11T06:51:30.954 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0077930730000000002 2024-09-11T06:51:30.954 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.954 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.954 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:30.954 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826281+0000", 2024-09-11T06:51:30.954 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5876000000000003e-05 2024-09-11T06:51:30.954 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.954 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.954 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:30.954 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826384+0000", 2024-09-11T06:51:30.954 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103838 2024-09-11T06:51:30.954 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.954 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.955 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:30.955 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830642+0000", 2024-09-11T06:51:30.955 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004257083 2024-09-11T06:51:30.955 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.955 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.955 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:30.955 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831509+0000", 2024-09-11T06:51:30.955 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086730900000000003 2024-09-11T06:51:30.955 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.955 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.955 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:30.955 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831547+0000", 2024-09-11T06:51:30.955 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7793000000000002e-05 2024-09-11T06:51:30.955 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.956 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.956 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:30.956 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831612+0000", 2024-09-11T06:51:30.956 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5691000000000005e-05 2024-09-11T06:51:30.956 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.956 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:30.956 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.956 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.956 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.956 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:30.956 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:51:30.956 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 144.06530400400001, 2024-09-11T06:51:30.956 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016098024999999998, 2024-09-11T06:51:30.957 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:30.957 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:30.957 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:30.957 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:30.957 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:30.957 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:30.957 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 140 2024-09-11T06:51:30.957 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.957 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:30.957 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.957 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:30.957 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:51:30.957 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.958 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.958 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.958 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:30.958 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:51:30.958 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.958 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.958 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.958 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:30.958 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:51:30.958 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5199999999999998e-07 2024-09-11T06:51:30.958 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.958 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.958 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:30.958 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818479+0000", 2024-09-11T06:51:30.959 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0368000000000002e-05 2024-09-11T06:51:30.959 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.959 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.959 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:30.959 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818487+0000", 2024-09-11T06:51:30.959 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2339999999999994e-06 2024-09-11T06:51:30.959 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.959 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.959 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.959 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818491+0000", 2024-09-11T06:51:30.959 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4329999999999997e-06 2024-09-11T06:51:30.959 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.959 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.960 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:30.960 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828080+0000", 2024-09-11T06:51:30.960 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0095890449999999992 2024-09-11T06:51:30.960 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.960 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.960 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:30.960 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828130+0000", 2024-09-11T06:51:30.960 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9239000000000001e-05 2024-09-11T06:51:30.960 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.960 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.960 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:30.960 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828208+0000", 2024-09-11T06:51:30.960 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.852e-05 2024-09-11T06:51:30.961 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.961 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.961 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:30.961 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831625+0000", 2024-09-11T06:51:30.961 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034169410000000002 2024-09-11T06:51:30.961 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.961 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.961 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:30.961 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834411+0000", 2024-09-11T06:51:30.961 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002786111 2024-09-11T06:51:30.961 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.961 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.961 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:30.962 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834457+0000", 2024-09-11T06:51:30.962 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6041000000000002e-05 2024-09-11T06:51:30.962 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.962 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.962 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:30.962 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834516+0000", 2024-09-11T06:51:30.962 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8740999999999999e-05 2024-09-11T06:51:30.962 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.962 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:30.962 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.962 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.962 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.962 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:30.963 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:51:30.963 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 144.06522293699999, 2024-09-11T06:51:30.963 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016588772000000002, 2024-09-11T06:51:30.963 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:30.963 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:30.963 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:30.963 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:30.963 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:30.963 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:30.963 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 141 2024-09-11T06:51:30.963 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.963 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:30.963 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.964 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:30.964 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:51:30.964 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.964 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.964 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.964 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:30.964 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:51:30.964 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.964 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.964 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.964 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:30.964 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818500+0000", 2024-09-11T06:51:30.964 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0800000000000005e-07 2024-09-11T06:51:30.965 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.965 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.965 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:30.965 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818505+0000", 2024-09-11T06:51:30.965 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2429999999999996e-06 2024-09-11T06:51:30.965 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.965 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.965 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:30.965 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818510+0000", 2024-09-11T06:51:30.965 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9339999999999997e-06 2024-09-11T06:51:30.965 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.965 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.965 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.965 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818514+0000", 2024-09-11T06:51:30.966 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7270000000000001e-06 2024-09-11T06:51:30.966 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.966 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.966 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:30.966 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828711+0000", 2024-09-11T06:51:30.966 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010197031 2024-09-11T06:51:30.966 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.966 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.966 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:30.966 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828853+0000", 2024-09-11T06:51:30.966 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014201200000000001 2024-09-11T06:51:30.966 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.966 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.967 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:30.967 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.829115+0000", 2024-09-11T06:51:30.967 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026257500000000003 2024-09-11T06:51:30.967 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.967 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.967 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:30.967 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831629+0000", 2024-09-11T06:51:30.967 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025138169999999998 2024-09-11T06:51:30.967 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.967 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.967 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:30.967 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834991+0000", 2024-09-11T06:51:30.967 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033624459999999998 2024-09-11T06:51:30.968 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.968 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.968 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:30.968 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835032+0000", 2024-09-11T06:51:30.968 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1001000000000003e-05 2024-09-11T06:51:30.968 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.968 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.968 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:30.968 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835088+0000", 2024-09-11T06:51:30.968 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5478e-05 2024-09-11T06:51:30.968 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.968 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:30.968 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.969 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.969 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.969 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5235.0:1 2.5 2:b2cc3336:::rbd_id.testrbdnbd21607:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-11T06:51:30.969 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:51:30.969 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 56.324420914999997, 2024-09-11T06:51:30.969 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.68353462499999995, 2024-09-11T06:51:30.969 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:30.969 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:30.969 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:30.969 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:30.969 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5235", 2024-09-11T06:51:30.969 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2008977164", 2024-09-11T06:51:30.969 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:51:30.970 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.970 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:30.970 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.970 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:30.970 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:51:30.970 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.970 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.970 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.970 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:30.970 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:51:30.970 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.970 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.970 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.970 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:30.971 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559303+0000", 2024-09-11T06:51:30.971 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1830000000000001e-06 2024-09-11T06:51:30.971 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.971 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.971 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:30.971 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559314+0000", 2024-09-11T06:51:30.971 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0396e-05 2024-09-11T06:51:30.971 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.971 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.971 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:30.971 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559335+0000", 2024-09-11T06:51:30.971 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1183e-05 2024-09-11T06:51:30.971 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.972 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.972 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.972 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559354+0000", 2024-09-11T06:51:30.972 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9225999999999999e-05 2024-09-11T06:51:30.972 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.972 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.972 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:30.972 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559522+0000", 2024-09-11T06:51:30.972 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000168021 2024-09-11T06:51:30.972 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.972 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.972 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:51:30.972 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559543+0000", 2024-09-11T06:51:30.972 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0616999999999999e-05 2024-09-11T06:51:30.973 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.973 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.973 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:30.973 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.238584+0000", 2024-09-11T06:51:30.973 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.67904107499999999 2024-09-11T06:51:30.973 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.973 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.973 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for missing object", 2024-09-11T06:51:30.973 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.238933+0000", 2024-09-11T06:51:30.973 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034879600000000001 2024-09-11T06:51:30.973 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.973 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.973 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:30.974 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.241968+0000", 2024-09-11T06:51:30.974 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030350830000000001 2024-09-11T06:51:30.974 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.974 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.974 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:30.974 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.242281+0000", 2024-09-11T06:51:30.974 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00031299499999999999 2024-09-11T06:51:30.974 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.974 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.974 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:30.974 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.242836+0000", 2024-09-11T06:51:30.974 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00055504999999999997 2024-09-11T06:51:30.974 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.974 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:30.975 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.975 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.975 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.975 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5235.0:18 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 0~1024 [fadvise_sequential]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T06:51:30.975 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:51:30.975 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 55.621928476000001, 2024-09-11T06:51:30.975 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.048993760999999997, 2024-09-11T06:51:30.975 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:30.975 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:30.975 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:30.975 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:30.975 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5235", 2024-09-11T06:51:30.975 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2008977164", 2024-09-11T06:51:30.976 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 18 2024-09-11T06:51:30.976 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.976 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:30.976 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.976 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:30.976 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:51:30.976 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.976 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.976 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.976 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:30.976 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:51:30.976 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.976 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.976 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.977 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:30.977 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261797+0000", 2024-09-11T06:51:30.977 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2059999999999999e-06 2024-09-11T06:51:30.977 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.977 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.977 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:30.977 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261812+0000", 2024-09-11T06:51:30.977 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5159e-05 2024-09-11T06:51:30.977 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.977 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.977 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:30.977 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261852+0000", 2024-09-11T06:51:30.977 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.9906e-05 2024-09-11T06:51:30.977 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.978 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.978 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.978 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261865+0000", 2024-09-11T06:51:30.978 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.312e-05 2024-09-11T06:51:30.978 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.978 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.978 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:30.978 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.263166+0000", 2024-09-11T06:51:30.978 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001301134 2024-09-11T06:51:30.978 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.978 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.978 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for missing object", 2024-09-11T06:51:30.978 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.263529+0000", 2024-09-11T06:51:30.978 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036279900000000002 2024-09-11T06:51:30.979 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.979 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.979 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:30.979 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.300567+0000", 2024-09-11T06:51:30.979 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.037038326000000003 2024-09-11T06:51:30.979 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.979 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.979 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:30.979 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.300800+0000", 2024-09-11T06:51:30.979 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023323300000000001 2024-09-11T06:51:30.979 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.979 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.979 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:30.980 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.306709+0000", 2024-09-11T06:51:30.980 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0059087690000000003 2024-09-11T06:51:30.980 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.980 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.980 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:30.980 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.306850+0000", 2024-09-11T06:51:30.980 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014106200000000001 2024-09-11T06:51:30.980 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.980 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.980 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:30.980 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.307121+0000", 2024-09-11T06:51:30.980 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027083599999999999 2024-09-11T06:51:30.980 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.981 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.981 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:30.981 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.308809+0000", 2024-09-11T06:51:30.981 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0016880759999999999 2024-09-11T06:51:30.981 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.981 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.981 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:30.981 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310382+0000", 2024-09-11T06:51:30.981 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0015732840000000001 2024-09-11T06:51:30.981 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.981 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.981 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:30.982 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310501+0000", 2024-09-11T06:51:30.982 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000118276 2024-09-11T06:51:30.982 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.982 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.982 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:30.982 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310787+0000", 2024-09-11T06:51:30.982 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00028657500000000001 2024-09-11T06:51:30.982 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.982 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:30.982 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.982 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.982 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.982 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:70 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 393216~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:30.983 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596118+0000", 2024-09-11T06:51:30.983 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 42.287604074000001, 2024-09-11T06:51:30.983 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012279145, 2024-09-11T06:51:30.983 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:30.983 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:30.983 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:30.983 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:30.983 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:30.983 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:30.983 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 70 2024-09-11T06:51:30.983 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.983 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:30.983 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.983 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:30.984 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596118+0000", 2024-09-11T06:51:30.984 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.984 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.984 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.984 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:30.984 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596118+0000", 2024-09-11T06:51:30.984 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.984 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.984 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.984 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:30.984 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596119+0000", 2024-09-11T06:51:30.984 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0299999999999997e-07 2024-09-11T06:51:30.984 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.984 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.985 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:30.985 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596238+0000", 2024-09-11T06:51:30.985 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000118798 2024-09-11T06:51:30.985 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.985 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.985 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:30.985 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596250+0000", 2024-09-11T06:51:30.985 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2775999999999999e-05 2024-09-11T06:51:30.985 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.985 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.985 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.985 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596259+0000", 2024-09-11T06:51:30.985 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3100000000000001e-06 2024-09-11T06:51:30.986 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.986 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.986 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:30.986 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.603389+0000", 2024-09-11T06:51:30.986 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0071300759999999999 2024-09-11T06:51:30.986 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.986 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.986 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:30.986 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.603540+0000", 2024-09-11T06:51:30.986 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015111700000000001 2024-09-11T06:51:30.986 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.986 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.986 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:30.987 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.603766+0000", 2024-09-11T06:51:30.987 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022570899999999999 2024-09-11T06:51:30.987 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.987 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.987 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:30.987 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.606676+0000", 2024-09-11T06:51:30.987 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0029106380000000001 2024-09-11T06:51:30.987 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.987 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.987 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:30.987 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608173+0000", 2024-09-11T06:51:30.987 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014967629999999999 2024-09-11T06:51:30.987 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.987 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.988 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:30.988 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608277+0000", 2024-09-11T06:51:30.988 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000104398 2024-09-11T06:51:30.988 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.988 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.988 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:30.988 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608397+0000", 2024-09-11T06:51:30.988 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000119657 2024-09-11T06:51:30.988 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.988 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:30.988 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.988 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.988 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.989 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:71 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 524288~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:30.989 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596272+0000", 2024-09-11T06:51:30.989 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 42.287449821000003, 2024-09-11T06:51:30.989 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013390681999999999, 2024-09-11T06:51:30.989 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:30.989 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:30.989 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:30.989 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:30.989 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:30.989 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:30.989 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 71 2024-09-11T06:51:30.989 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.990 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:30.990 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.990 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:30.990 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596272+0000", 2024-09-11T06:51:30.990 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.990 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.990 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.990 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:30.990 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596272+0000", 2024-09-11T06:51:30.990 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.990 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.990 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.990 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:30.990 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596273+0000", 2024-09-11T06:51:30.991 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.9999999999999996e-07 2024-09-11T06:51:30.991 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.991 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.991 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:30.991 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596395+0000", 2024-09-11T06:51:30.991 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012204799999999999 2024-09-11T06:51:30.991 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.991 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.991 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:30.991 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596411+0000", 2024-09-11T06:51:30.991 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5746000000000001e-05 2024-09-11T06:51:30.991 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.991 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.992 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.992 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596419+0000", 2024-09-11T06:51:30.992 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2600000000000005e-06 2024-09-11T06:51:30.992 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.992 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.992 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:30.992 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.605311+0000", 2024-09-11T06:51:30.992 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0088915179999999993 2024-09-11T06:51:30.992 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.992 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.992 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:30.992 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.605436+0000", 2024-09-11T06:51:30.992 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000125203 2024-09-11T06:51:30.993 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.993 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.993 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:30.993 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.605625+0000", 2024-09-11T06:51:30.993 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018961299999999999 2024-09-11T06:51:30.993 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.993 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.993 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:30.993 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609477+0000", 2024-09-11T06:51:30.993 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0038513549999999999 2024-09-11T06:51:30.993 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.993 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.993 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:30.994 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609499+0000", 2024-09-11T06:51:30.994 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2215e-05 2024-09-11T06:51:30.994 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.994 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.994 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:30.994 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609589+0000", 2024-09-11T06:51:30.994 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0029000000000001e-05 2024-09-11T06:51:30.994 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.994 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.994 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:30.994 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609663+0000", 2024-09-11T06:51:30.994 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3894999999999996e-05 2024-09-11T06:51:30.994 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.994 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:30.995 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:30.995 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.995 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.995 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:72 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 655360~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:30.995 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596432+0000", 2024-09-11T06:51:30.995 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 42.287290333999998, 2024-09-11T06:51:30.995 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01414727, 2024-09-11T06:51:30.995 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:30.995 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:30.995 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:30.995 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:30.995 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:30.996 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:30.996 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 72 2024-09-11T06:51:30.996 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.996 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:30.996 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.996 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:30.996 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596432+0000", 2024-09-11T06:51:30.996 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.996 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.996 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.996 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:30.996 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596432+0000", 2024-09-11T06:51:30.996 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:30.996 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.997 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.997 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:30.997 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596433+0000", 2024-09-11T06:51:30.997 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1999999999999998e-07 2024-09-11T06:51:30.997 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.997 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.997 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:30.997 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596556+0000", 2024-09-11T06:51:30.997 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012379000000000001 2024-09-11T06:51:30.997 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.997 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.997 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:30.997 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596571+0000", 2024-09-11T06:51:30.998 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4749999999999999e-05 2024-09-11T06:51:30.998 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.998 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.998 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:30.998 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596579+0000", 2024-09-11T06:51:30.998 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2500000000000006e-06 2024-09-11T06:51:30.998 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.998 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.998 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:30.998 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.606784+0000", 2024-09-11T06:51:30.998 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010204757 2024-09-11T06:51:30.998 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.999 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.999 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:30.999 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.606950+0000", 2024-09-11T06:51:30.999 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000166229 2024-09-11T06:51:30.999 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.999 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.999 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:30.999 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.607176+0000", 2024-09-11T06:51:30.999 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022549899999999999 2024-09-11T06:51:30.999 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:30.999 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:30.999 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:30.999 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610363+0000", 2024-09-11T06:51:31.000 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031874690000000001 2024-09-11T06:51:31.000 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.000 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.000 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:31.000 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610490+0000", 2024-09-11T06:51:31.000 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012639899999999999 2024-09-11T06:51:31.000 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.000 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.000 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:31.000 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610529+0000", 2024-09-11T06:51:31.000 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.8936000000000001e-05 2024-09-11T06:51:31.000 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.000 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.001 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:31.001 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610579+0000", 2024-09-11T06:51:31.001 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0371000000000003e-05 2024-09-11T06:51:31.001 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.001 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:31.001 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.001 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.001 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.001 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:73 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 786432~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:31.001 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596592+0000", 2024-09-11T06:51:31.001 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 42.287130081000001, 2024-09-11T06:51:31.001 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014914779, 2024-09-11T06:51:31.001 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:31.002 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:31.002 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:31.002 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:31.002 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:31.002 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:31.002 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 73 2024-09-11T06:51:31.002 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.002 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:31.002 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.002 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:31.002 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596592+0000", 2024-09-11T06:51:31.002 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.002 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.003 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.003 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:31.003 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596592+0000", 2024-09-11T06:51:31.003 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.003 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.003 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.003 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:31.003 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596593+0000", 2024-09-11T06:51:31.003 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.0699999999999996e-07 2024-09-11T06:51:31.003 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.003 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.003 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:31.003 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596786+0000", 2024-09-11T06:51:31.003 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019278200000000001 2024-09-11T06:51:31.004 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.004 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.004 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:31.004 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596801+0000", 2024-09-11T06:51:31.004 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5690000000000001e-05 2024-09-11T06:51:31.004 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.004 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.004 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:31.004 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596815+0000", 2024-09-11T06:51:31.004 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3753e-05 2024-09-11T06:51:31.004 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.004 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.004 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:31.004 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608442+0000", 2024-09-11T06:51:31.005 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011627385 2024-09-11T06:51:31.005 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.005 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.005 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:31.005 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608570+0000", 2024-09-11T06:51:31.005 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012740399999999999 2024-09-11T06:51:31.005 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.005 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.005 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:31.005 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608754+0000", 2024-09-11T06:51:31.005 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018392899999999999 2024-09-11T06:51:31.005 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.005 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.005 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:31.006 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610370+0000", 2024-09-11T06:51:31.006 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001615998 2024-09-11T06:51:31.006 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.006 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.006 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:31.006 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611420+0000", 2024-09-11T06:51:31.006 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010502790000000001 2024-09-11T06:51:31.006 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.006 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.006 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:31.006 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611458+0000", 2024-09-11T06:51:31.006 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7749000000000001e-05 2024-09-11T06:51:31.006 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.007 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.007 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:31.007 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611507+0000", 2024-09-11T06:51:31.007 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9002999999999998e-05 2024-09-11T06:51:31.007 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.007 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:31.007 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.007 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.007 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.007 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:74 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 917504~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:31.007 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596849+0000", 2024-09-11T06:51:31.007 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 42.286873442999998, 2024-09-11T06:51:31.007 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015202963999999999, 2024-09-11T06:51:31.007 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:31.007 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:31.008 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:31.008 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:31.008 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:31.008 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:31.008 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 74 2024-09-11T06:51:31.008 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.008 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:31.008 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.008 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:31.008 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596849+0000", 2024-09-11T06:51:31.008 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.008 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.008 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.009 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:31.009 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596849+0000", 2024-09-11T06:51:31.009 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.009 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.009 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.009 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:31.009 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596850+0000", 2024-09-11T06:51:31.009 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3170000000000001e-06 2024-09-11T06:51:31.009 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.009 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.009 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:31.009 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.597118+0000", 2024-09-11T06:51:31.009 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026772999999999999 2024-09-11T06:51:31.009 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.010 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.010 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:31.010 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.597136+0000", 2024-09-11T06:51:31.010 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8467000000000001e-05 2024-09-11T06:51:31.010 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.010 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.010 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:31.010 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.597148+0000", 2024-09-11T06:51:31.010 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1486e-05 2024-09-11T06:51:31.010 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.010 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.010 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:31.010 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609714+0000", 2024-09-11T06:51:31.011 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012566124 2024-09-11T06:51:31.011 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.011 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.011 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:31.011 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609792+0000", 2024-09-11T06:51:31.011 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.8579999999999996e-05 2024-09-11T06:51:31.011 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.011 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.011 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:31.011 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609899+0000", 2024-09-11T06:51:31.011 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00010716499999999999 2024-09-11T06:51:31.011 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.011 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.011 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:31.011 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610865+0000", 2024-09-11T06:51:31.012 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00096540299999999999 2024-09-11T06:51:31.012 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.012 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.012 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:31.012 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611970+0000", 2024-09-11T06:51:31.012 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001105118 2024-09-11T06:51:31.012 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.012 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.012 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:31.012 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.612005+0000", 2024-09-11T06:51:31.012 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4789000000000002e-05 2024-09-11T06:51:31.012 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.012 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.012 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:31.013 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.612052+0000", 2024-09-11T06:51:31.013 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6785e-05 2024-09-11T06:51:31.013 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.013 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:31.013 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.013 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.013 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:31.013 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:31.014 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:51:31.015 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:31.009+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:31.115 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:31.109+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:31.116 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.0 to 1 (10000) 2024-09-11T06:51:31.144 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-11T06:51:31.190 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:31.190 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:31.190 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:51:31.190 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:31.204 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:51:31.233 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:31.230+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:31.317 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:31.313+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:31.367 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:31.368 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:31.368 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:51:31.368 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:51:31.368 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:31.382 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:51:31.418 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:31.414+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:31.519 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:31.518+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:31.548 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:31.548 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:51:31.548 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:51:31.548 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:51:31.548 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.548 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4558.0:1 3.3 3:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-11T06:51:31.548 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:51:31.548 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 74.453554467000004, 2024-09-11T06:51:31.548 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0085520600000000002, 2024-09-11T06:51:31.549 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:31.549 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:31.549 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:31.549 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:31.549 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4558", 2024-09-11T06:51:31.549 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1478388898", 2024-09-11T06:51:31.549 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:51:31.549 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.549 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:31.549 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.549 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:31.549 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:51:31.549 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.550 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:31.550 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:51:31.550 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.550 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:31.550 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090533+0000", 2024-09-11T06:51:31.550 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2629999999999999e-06 2024-09-11T06:51:31.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.550 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:31.551 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090546+0000", 2024-09-11T06:51:31.551 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2918999999999999e-05 2024-09-11T06:51:31.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.551 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.551 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:31.551 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090572+0000", 2024-09-11T06:51:31.551 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6412000000000001e-05 2024-09-11T06:51:31.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.551 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.551 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:31.551 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090594+0000", 2024-09-11T06:51:31.551 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2476e-05 2024-09-11T06:51:31.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.552 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.552 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:31.552 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090938+0000", 2024-09-11T06:51:31.552 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034325099999999998 2024-09-11T06:51:31.552 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.552 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.552 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:31.552 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.091149+0000", 2024-09-11T06:51:31.552 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021151300000000001 2024-09-11T06:51:31.552 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.552 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.552 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:31.552 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.091377+0000", 2024-09-11T06:51:31.552 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022785399999999999 2024-09-11T06:51:31.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.553 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:31.553 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.092912+0000", 2024-09-11T06:51:31.553 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0015346089999999999 2024-09-11T06:51:31.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.553 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:31.553 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098919+0000", 2024-09-11T06:51:31.553 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0060076799999999996 2024-09-11T06:51:31.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.553 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:31.553 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.099009+0000", 2024-09-11T06:51:31.554 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0056000000000006e-05 2024-09-11T06:51:31.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.554 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.554 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:31.554 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.099081+0000", 2024-09-11T06:51:31.554 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2026999999999993e-05 2024-09-11T06:51:31.554 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.554 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:31.554 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.554 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.554 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:51:31.554 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:51:31.554 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 68.412065386999998, 2024-09-11T06:51:31.555 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0071924390000000001, 2024-09-11T06:51:31.555 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:31.555 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:31.555 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:31.555 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:31.555 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.555 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:31.555 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:51:31.555 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.555 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.555 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:31.555 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:51:31.556 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.556 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.556 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.556 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:31.556 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132020+0000", 2024-09-11T06:51:31.556 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.846e-06 2024-09-11T06:51:31.556 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.556 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.556 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:31.556 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135688+0000", 2024-09-11T06:51:31.556 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036677350000000001 2024-09-11T06:51:31.556 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.556 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.557 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:31.557 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135747+0000", 2024-09-11T06:51:31.557 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.9314999999999997e-05 2024-09-11T06:51:31.557 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.557 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.557 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:31.557 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135797+0000", 2024-09-11T06:51:31.557 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9984000000000001e-05 2024-09-11T06:51:31.557 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.557 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.557 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:31.557 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136143+0000", 2024-09-11T06:51:31.557 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034594299999999998 2024-09-11T06:51:31.557 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.558 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.558 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:31.558 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136179+0000", 2024-09-11T06:51:31.558 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5636000000000002e-05 2024-09-11T06:51:31.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.558 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.558 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:31.558 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.139211+0000", 2024-09-11T06:51:31.558 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030319800000000001 2024-09-11T06:51:31.558 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.558 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:31.558 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.559 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.559 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:51:31.559 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:51:31.559 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 68.410002133000006, 2024-09-11T06:51:31.559 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064954139999999997, 2024-09-11T06:51:31.559 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:31.559 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:31.559 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:31.559 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:31.559 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.559 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:31.559 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:51:31.559 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.560 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:31.560 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:51:31.560 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.560 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:31.560 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134084+0000", 2024-09-11T06:51:31.560 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.894e-06 2024-09-11T06:51:31.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.561 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:31.561 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136513+0000", 2024-09-11T06:51:31.561 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024290969999999999 2024-09-11T06:51:31.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.561 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.561 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:31.561 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136554+0000", 2024-09-11T06:51:31.561 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.0932e-05 2024-09-11T06:51:31.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.561 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.561 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:31.561 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136584+0000", 2024-09-11T06:51:31.561 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0056000000000002e-05 2024-09-11T06:51:31.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.562 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:31.562 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136805+0000", 2024-09-11T06:51:31.562 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022086100000000001 2024-09-11T06:51:31.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.562 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:31.562 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136864+0000", 2024-09-11T06:51:31.562 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.9907000000000001e-05 2024-09-11T06:51:31.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.562 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:31.562 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.140577+0000", 2024-09-11T06:51:31.563 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0037126669999999998 2024-09-11T06:51:31.563 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.563 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:31.563 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.563 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.563 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:31.563 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:51:31.563 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 68.401711332999994, 2024-09-11T06:51:31.563 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035763180000000002, 2024-09-11T06:51:31.563 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:31.563 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:31.563 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:31.564 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:31.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.564 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:31.564 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:51:31.564 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.564 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:31.564 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:51:31.564 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.564 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:31.564 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142374+0000", 2024-09-11T06:51:31.565 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5460000000000001e-06 2024-09-11T06:51:31.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.565 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.565 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:31.565 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143902+0000", 2024-09-11T06:51:31.565 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00152808 2024-09-11T06:51:31.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.565 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.565 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:31.565 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143946+0000", 2024-09-11T06:51:31.565 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3874e-05 2024-09-11T06:51:31.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.565 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.565 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:31.566 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143978+0000", 2024-09-11T06:51:31.566 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2425999999999998e-05 2024-09-11T06:51:31.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.566 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.566 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:31.566 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.144177+0000", 2024-09-11T06:51:31.566 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019837500000000001 2024-09-11T06:51:31.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.566 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.566 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:31.566 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.144198+0000", 2024-09-11T06:51:31.566 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1639000000000001e-05 2024-09-11T06:51:31.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.566 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.567 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:31.567 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.145949+0000", 2024-09-11T06:51:31.567 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0017503779999999999 2024-09-11T06:51:31.567 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.567 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:31.567 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.567 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.567 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.567 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:31.567 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:51:31.567 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 68.397395630999995, 2024-09-11T06:51:31.567 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0057693220000000003, 2024-09-11T06:51:31.567 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:31.567 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:31.568 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:31.568 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:31.568 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.568 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:31.568 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:51:31.568 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.568 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.568 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:31.568 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:51:31.568 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.568 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.569 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:31.569 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146690+0000", 2024-09-11T06:51:31.569 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.607e-06 2024-09-11T06:51:31.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.569 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:31.569 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148693+0000", 2024-09-11T06:51:31.569 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0020029739999999998 2024-09-11T06:51:31.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.569 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:31.569 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148735+0000", 2024-09-11T06:51:31.569 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1958e-05 2024-09-11T06:51:31.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.570 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:31.570 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148764+0000", 2024-09-11T06:51:31.570 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8999000000000001e-05 2024-09-11T06:51:31.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.570 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:31.570 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149069+0000", 2024-09-11T06:51:31.570 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000305032 2024-09-11T06:51:31.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.570 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:31.570 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149204+0000", 2024-09-11T06:51:31.570 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013495 2024-09-11T06:51:31.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.571 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:31.571 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.152458+0000", 2024-09-11T06:51:31.571 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0032538020000000001 2024-09-11T06:51:31.571 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.571 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:31.571 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.571 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:31.571 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:51:31.571 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 68.396497089999997, 2024-09-11T06:51:31.572 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0042425329999999997, 2024-09-11T06:51:31.572 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:31.572 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:31.572 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:31.572 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:31.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.572 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:31.572 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:51:31.572 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.572 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:31.572 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:51:31.572 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.573 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:31.573 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147588+0000", 2024-09-11T06:51:31.573 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.277e-06 2024-09-11T06:51:31.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.573 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:31.573 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149051+0000", 2024-09-11T06:51:31.573 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001462669 2024-09-11T06:51:31.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.573 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:31.574 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149090+0000", 2024-09-11T06:51:31.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.8967999999999997e-05 2024-09-11T06:51:31.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.574 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:31.574 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149120+0000", 2024-09-11T06:51:31.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0766000000000003e-05 2024-09-11T06:51:31.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.574 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:31.574 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149395+0000", 2024-09-11T06:51:31.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027484900000000002 2024-09-11T06:51:31.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.575 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:31.575 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149439+0000", 2024-09-11T06:51:31.575 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3940999999999998e-05 2024-09-11T06:51:31.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.575 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:31.575 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.151829+0000", 2024-09-11T06:51:31.575 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002390063 2024-09-11T06:51:31.575 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.575 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:31.575 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.576 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:31.576 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:51:31.576 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 68.390356459000003, 2024-09-11T06:51:31.576 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0046568219999999997, 2024-09-11T06:51:31.576 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:31.576 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:31.576 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:31.576 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:31.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.576 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:31.576 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:51:31.576 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:31.577 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:51:31.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:31.577 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153729+0000", 2024-09-11T06:51:31.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5629999999999999e-06 2024-09-11T06:51:31.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:31.577 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155179+0000", 2024-09-11T06:51:31.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001450469 2024-09-11T06:51:31.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.578 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:31.578 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155221+0000", 2024-09-11T06:51:31.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1448e-05 2024-09-11T06:51:31.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.578 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:31.578 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155249+0000", 2024-09-11T06:51:31.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8433e-05 2024-09-11T06:51:31.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.579 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:31.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155420+0000", 2024-09-11T06:51:31.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017070599999999999 2024-09-11T06:51:31.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.579 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:31.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155445+0000", 2024-09-11T06:51:31.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4813000000000001e-05 2024-09-11T06:51:31.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.579 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:31.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.158384+0000", 2024-09-11T06:51:31.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0029393900000000001 2024-09-11T06:51:31.579 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.580 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:31.580 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.580 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:31.580 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:51:31.580 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 68.388329709999994, 2024-09-11T06:51:31.580 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0046826020000000001, 2024-09-11T06:51:31.580 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:31.580 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:31.580 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:31.580 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:31.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.581 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:31.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:51:31.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.581 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:31.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:51:31.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.581 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:31.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155756+0000", 2024-09-11T06:51:31.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.384e-06 2024-09-11T06:51:31.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:31.582 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157205+0000", 2024-09-11T06:51:31.582 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014496260000000001 2024-09-11T06:51:31.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:31.582 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157252+0000", 2024-09-11T06:51:31.582 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6368000000000001e-05 2024-09-11T06:51:31.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:31.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157286+0000", 2024-09-11T06:51:31.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4944999999999998e-05 2024-09-11T06:51:31.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:31.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157540+0000", 2024-09-11T06:51:31.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00025307999999999998 2024-09-11T06:51:31.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:31.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157576+0000", 2024-09-11T06:51:31.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.6878999999999997e-05 2024-09-11T06:51:31.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.584 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:31.584 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.160437+0000", 2024-09-11T06:51:31.584 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00286032 2024-09-11T06:51:31.584 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.584 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:31.584 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.584 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:51:31.584 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.280679+0000", 2024-09-11T06:51:31.584 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 56.263404721000001, 2024-09-11T06:51:31.584 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0040934309999999998, 2024-09-11T06:51:31.584 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:31.585 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:31.585 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:51:31.585 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:31.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.585 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:31.585 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280679+0000", 2024-09-11T06:51:31.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.585 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:31.585 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280679+0000", 2024-09-11T06:51:31.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:31.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280681+0000", 2024-09-11T06:51:31.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2529999999999999e-06 2024-09-11T06:51:31.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:31.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280713+0000", 2024-09-11T06:51:31.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.1936000000000001e-05 2024-09-11T06:51:31.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:31.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280758+0000", 2024-09-11T06:51:31.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5124999999999999e-05 2024-09-11T06:51:31.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.587 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:31.587 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280779+0000", 2024-09-11T06:51:31.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0936000000000001e-05 2024-09-11T06:51:31.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.587 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:31.587 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280932+0000", 2024-09-11T06:51:31.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015220899999999999 2024-09-11T06:51:31.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.587 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:31.588 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.284773+0000", 2024-09-11T06:51:31.588 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0038409719999999998 2024-09-11T06:51:31.588 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.588 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:31.588 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.588 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:51:31.588 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.293438+0000", 2024-09-11T06:51:31.588 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 56.250645984999998, 2024-09-11T06:51:31.588 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0061479350000000002, 2024-09-11T06:51:31.588 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:31.589 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:31.589 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:51:31.589 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:31.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.589 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:31.589 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293438+0000", 2024-09-11T06:51:31.589 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.589 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:31.589 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293438+0000", 2024-09-11T06:51:31.589 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:31.590 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293439+0000", 2024-09-11T06:51:31.590 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:51:31.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:31.590 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293459+0000", 2024-09-11T06:51:31.590 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9922e-05 2024-09-11T06:51:31.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:31.590 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293482+0000", 2024-09-11T06:51:31.590 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2969e-05 2024-09-11T06:51:31.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.591 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:31.591 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293494+0000", 2024-09-11T06:51:31.591 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.131e-05 2024-09-11T06:51:31.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.591 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:31.591 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.295901+0000", 2024-09-11T06:51:31.591 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024077949999999999 2024-09-11T06:51:31.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.591 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:31.591 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.299586+0000", 2024-09-11T06:51:31.592 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036845020000000001 2024-09-11T06:51:31.592 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.592 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:31.592 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.592 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(client.5239.0:37 2.7 e91/69 2:eab0161e:::rbd_data.1314f0a2bfa2.0000000000000001:head v 91'134, mlcod=91'130)", 2024-09-11T06:51:31.592 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:08.068368+0000", 2024-09-11T06:51:31.592 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 23.475715907000001, 2024-09-11T06:51:31.592 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003850118, 2024-09-11T06:51:31.592 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:31.592 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:31.592 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:31.592 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:31.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.593 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:31.593 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068368+0000", 2024-09-11T06:51:31.593 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.593 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:31.593 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068368+0000", 2024-09-11T06:51:31.593 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.593 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:31.593 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068369+0000", 2024-09-11T06:51:31.593 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2699999999999998e-07 2024-09-11T06:51:31.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.594 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:31.594 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068458+0000", 2024-09-11T06:51:31.594 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9075999999999999e-05 2024-09-11T06:51:31.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.594 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:31.594 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068471+0000", 2024-09-11T06:51:31.594 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.269e-05 2024-09-11T06:51:31.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.594 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:31.594 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068480+0000", 2024-09-11T06:51:31.595 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.6199999999999994e-06 2024-09-11T06:51:31.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.595 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:31.595 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.069523+0000", 2024-09-11T06:51:31.595 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001042465 2024-09-11T06:51:31.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.595 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:31.595 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.069542+0000", 2024-09-11T06:51:31.595 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9386e-05 2024-09-11T06:51:31.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.595 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:31.596 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072122+0000", 2024-09-11T06:51:31.596 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025799540000000002 2024-09-11T06:51:31.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.596 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:31.596 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072218+0000", 2024-09-11T06:51:31.596 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.6100000000000005e-05 2024-09-11T06:51:31.596 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.596 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:31.596 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.596 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(client.5239.0:41 2.7 e91/69 2:eab0161e:::rbd_data.1314f0a2bfa2.0000000000000001:head v 91'138, mlcod=91'132)", 2024-09-11T06:51:31.596 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:08.071899+0000", 2024-09-11T06:51:31.597 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 23.472185129, 2024-09-11T06:51:31.597 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0037511240000000002, 2024-09-11T06:51:31.597 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:31.597 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:31.597 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:31.597 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:31.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.597 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:31.597 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.071899+0000", 2024-09-11T06:51:31.597 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.597 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:31.597 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.071899+0000", 2024-09-11T06:51:31.598 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.598 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:31.598 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.071900+0000", 2024-09-11T06:51:31.598 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3500000000000005e-07 2024-09-11T06:51:31.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.598 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:31.598 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072027+0000", 2024-09-11T06:51:31.598 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012720299999999999 2024-09-11T06:51:31.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.598 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:31.599 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072043+0000", 2024-09-11T06:51:31.599 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6475999999999999e-05 2024-09-11T06:51:31.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.599 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:31.599 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072053+0000", 2024-09-11T06:51:31.599 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2860000000000005e-06 2024-09-11T06:51:31.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.599 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:31.599 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072256+0000", 2024-09-11T06:51:31.599 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000203876 2024-09-11T06:51:31.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.600 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:31.600 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072282+0000", 2024-09-11T06:51:31.600 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5811000000000001e-05 2024-09-11T06:51:31.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.600 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:31.600 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.075544+0000", 2024-09-11T06:51:31.600 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0032621500000000001 2024-09-11T06:51:31.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.600 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:31.600 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.075650+0000", 2024-09-11T06:51:31.600 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000105487 2024-09-11T06:51:31.600 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.601 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:31.601 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.601 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.4 101/100 [PushOp(2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head, version: 54'23, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head@54'23, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:51:31.601 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.525431+0000", 2024-09-11T06:51:31.601 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 11.018652567, 2024-09-11T06:51:31.601 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0042697100000000003, 2024-09-11T06:51:31.601 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:31.601 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:31.601 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:31.601 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:31.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.601 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:31.602 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.525431+0000", 2024-09-11T06:51:31.602 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.602 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:31.602 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.525431+0000", 2024-09-11T06:51:31.602 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.602 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:31.602 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.525433+0000", 2024-09-11T06:51:31.602 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5829999999999999e-06 2024-09-11T06:51:31.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.603 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:31.603 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.527430+0000", 2024-09-11T06:51:31.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0019966350000000001 2024-09-11T06:51:31.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.603 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:31.603 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.527477+0000", 2024-09-11T06:51:31.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7681e-05 2024-09-11T06:51:31.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.603 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:31.603 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.527525+0000", 2024-09-11T06:51:31.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.8244000000000001e-05 2024-09-11T06:51:31.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.604 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:31.604 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.527795+0000", 2024-09-11T06:51:31.604 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026935199999999999 2024-09-11T06:51:31.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.604 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:31.604 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.527816+0000", 2024-09-11T06:51:31.604 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1112e-05 2024-09-11T06:51:31.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.604 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:31.604 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.529701+0000", 2024-09-11T06:51:31.604 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001885103 2024-09-11T06:51:31.604 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.605 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:31.605 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.605 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.4 101/100 [PushOp(2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head, version: 54'23, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head@54'23, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:31.605 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.534714+0000", 2024-09-11T06:51:31.605 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 11.009370184, 2024-09-11T06:51:31.605 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003413124, 2024-09-11T06:51:31.605 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:31.605 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:31.605 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:31.605 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:31.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.606 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:31.606 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.534714+0000", 2024-09-11T06:51:31.606 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.606 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.606 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:31.606 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.534714+0000", 2024-09-11T06:51:31.606 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.606 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.606 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:31.606 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.534715+0000", 2024-09-11T06:51:31.606 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6199999999999999e-06 2024-09-11T06:51:31.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.607 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:31.607 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.536302+0000", 2024-09-11T06:51:31.607 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001586269 2024-09-11T06:51:31.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.607 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:31.607 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.536347+0000", 2024-09-11T06:51:31.607 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5101000000000001e-05 2024-09-11T06:51:31.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.607 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:31.607 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.536379+0000", 2024-09-11T06:51:31.607 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2546000000000003e-05 2024-09-11T06:51:31.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.608 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:31.608 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.536541+0000", 2024-09-11T06:51:31.608 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000161812 2024-09-11T06:51:31.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.609 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:31.609 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.536561+0000", 2024-09-11T06:51:31.609 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0400000000000001e-05 2024-09-11T06:51:31.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.610 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:31.610 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.538127+0000", 2024-09-11T06:51:31.610 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0015653760000000001 2024-09-11T06:51:31.610 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.610 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:31.610 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.610 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(client.6081.0:30 2.4 e101/100 2:2d66d25d:::journal_data.2.1314f0a2bfa2.2:head v 101'171, mlcod=0'0)", 2024-09-11T06:51:31.610 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.536426+0000", 2024-09-11T06:51:31.610 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 11.007657774, 2024-09-11T06:51:31.610 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033841259999999999, 2024-09-11T06:51:31.610 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:31.611 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:31.611 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:31.611 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:31.611 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.611 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:31.611 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.536426+0000", 2024-09-11T06:51:31.611 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.611 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.611 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.611 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:31.611 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.536426+0000", 2024-09-11T06:51:31.611 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.611 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.611 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.612 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:31.612 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.536427+0000", 2024-09-11T06:51:31.612 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2839999999999999e-06 2024-09-11T06:51:31.612 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.612 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.612 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:31.612 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.536441+0000", 2024-09-11T06:51:31.612 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3682e-05 2024-09-11T06:51:31.612 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.612 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.612 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:31.612 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.536457+0000", 2024-09-11T06:51:31.612 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5775999999999999e-05 2024-09-11T06:51:31.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.613 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.613 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:31.613 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.536466+0000", 2024-09-11T06:51:31.613 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0000000000000002e-06 2024-09-11T06:51:31.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.613 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.613 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:31.613 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.538179+0000", 2024-09-11T06:51:31.613 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0017128639999999999 2024-09-11T06:51:31.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.613 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.613 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:31.614 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.538222+0000", 2024-09-11T06:51:31.614 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.2894e-05 2024-09-11T06:51:31.614 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.614 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.614 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:31.614 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.539719+0000", 2024-09-11T06:51:31.614 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00149752 2024-09-11T06:51:31.614 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.614 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.614 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:31.614 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.539810+0000", 2024-09-11T06:51:31.614 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.1106000000000005e-05 2024-09-11T06:51:31.614 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.615 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:31.615 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.615 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.615 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.615 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.4 101/100 [PushOp(2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head, version: 54'23, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head@54'23, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:31.615 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.540925+0000", 2024-09-11T06:51:31.615 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 11.003159171, 2024-09-11T06:51:31.615 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0040908539999999997, 2024-09-11T06:51:31.615 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:31.615 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:31.615 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:31.615 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:31.615 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.615 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:31.616 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.540925+0000", 2024-09-11T06:51:31.616 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.616 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.616 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.616 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:31.616 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.540925+0000", 2024-09-11T06:51:31.616 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.616 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.616 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.616 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:31.616 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.540926+0000", 2024-09-11T06:51:31.616 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.813e-06 2024-09-11T06:51:31.616 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.617 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.617 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:31.617 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542405+0000", 2024-09-11T06:51:31.617 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001478011 2024-09-11T06:51:31.617 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.617 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.617 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:31.617 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542453+0000", 2024-09-11T06:51:31.617 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.8677999999999997e-05 2024-09-11T06:51:31.617 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.617 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.617 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:31.617 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542483+0000", 2024-09-11T06:51:31.618 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9921999999999999e-05 2024-09-11T06:51:31.618 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.618 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.618 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:31.618 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542758+0000", 2024-09-11T06:51:31.618 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027447499999999999 2024-09-11T06:51:31.618 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.618 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.618 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:31.618 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542802+0000", 2024-09-11T06:51:31.618 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4288e-05 2024-09-11T06:51:31.619 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.619 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.619 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:31.619 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.545016+0000", 2024-09-11T06:51:31.619 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0022136669999999999 2024-09-11T06:51:31.619 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.619 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:31.619 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.619 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.619 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.620 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.4 101/100 [PushOp(2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head, version: 54'23, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head@54'23, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:31.620 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.547946+0000", 2024-09-11T06:51:31.620 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 10.996137684000001, 2024-09-11T06:51:31.620 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035631679999999998, 2024-09-11T06:51:31.620 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:31.620 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:31.620 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:31.620 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:31.620 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.620 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:31.620 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.547946+0000", 2024-09-11T06:51:31.620 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.621 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.621 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:31.621 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.547946+0000", 2024-09-11T06:51:31.621 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.621 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.621 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:31.621 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.547948+0000", 2024-09-11T06:51:31.621 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4529999999999999e-06 2024-09-11T06:51:31.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.621 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.621 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:31.621 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549382+0000", 2024-09-11T06:51:31.622 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014345759999999999 2024-09-11T06:51:31.622 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.622 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.622 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:31.622 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549425+0000", 2024-09-11T06:51:31.622 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.2528999999999999e-05 2024-09-11T06:51:31.622 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.622 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.622 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:31.622 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549453+0000", 2024-09-11T06:51:31.622 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7980999999999999e-05 2024-09-11T06:51:31.622 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.622 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.622 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:31.623 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549623+0000", 2024-09-11T06:51:31.623 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000170753 2024-09-11T06:51:31.623 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.623 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.623 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:31.623 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549644+0000", 2024-09-11T06:51:31.623 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0628999999999998e-05 2024-09-11T06:51:31.623 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.623 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.623 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:31.623 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.551509+0000", 2024-09-11T06:51:31.624 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001865247 2024-09-11T06:51:31.624 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.624 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:31.624 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.624 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.624 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.624 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(client.6081.0:37 2.4 e101/100 2:27499dce:::rbd_object_map.1314f0a2bfa2:head v 101'172, mlcod=54'22)", 2024-09-11T06:51:31.624 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.549499+0000", 2024-09-11T06:51:31.624 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 10.994584817, 2024-09-11T06:51:31.624 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0087992420000000005, 2024-09-11T06:51:31.624 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:31.624 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:31.624 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:31.625 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:31.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.625 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:31.625 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549499+0000", 2024-09-11T06:51:31.625 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.625 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:31.625 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549499+0000", 2024-09-11T06:51:31.625 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.625 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:31.626 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549500+0000", 2024-09-11T06:51:31.626 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.217e-06 2024-09-11T06:51:31.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.626 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.626 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:31.626 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549511+0000", 2024-09-11T06:51:31.626 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1143e-05 2024-09-11T06:51:31.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.626 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.626 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:31.626 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549532+0000", 2024-09-11T06:51:31.626 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0513000000000002e-05 2024-09-11T06:51:31.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.627 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:31.627 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549545+0000", 2024-09-11T06:51:31.627 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3209e-05 2024-09-11T06:51:31.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.627 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:31.627 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.551649+0000", 2024-09-11T06:51:31.627 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002103696 2024-09-11T06:51:31.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.627 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:31.627 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.551694+0000", 2024-09-11T06:51:31.627 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5674999999999999e-05 2024-09-11T06:51:31.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.628 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.628 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:31.628 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.558099+0000", 2024-09-11T06:51:31.628 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064040679999999997 2024-09-11T06:51:31.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.628 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.628 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:31.628 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.558298+0000", 2024-09-11T06:51:31.628 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019972100000000001 2024-09-11T06:51:31.629 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.629 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:31.629 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.629 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.629 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.4 101/100 [PushOp(2:272abc33:ns::rbd_data.11adc335555f.000000000000000c:head, version: 54'24, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:272abc33:ns::rbd_data.11adc335555f.000000000000000c:head@54'24, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:31.629 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.572958+0000", 2024-09-11T06:51:31.629 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 10.971125390999999, 2024-09-11T06:51:31.629 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035425080000000002, 2024-09-11T06:51:31.629 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:31.629 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:31.629 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:31.629 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:31.630 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.630 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:31.630 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.572958+0000", 2024-09-11T06:51:31.630 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.631 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:31.631 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.572958+0000", 2024-09-11T06:51:31.631 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.631 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:31.631 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.572960+0000", 2024-09-11T06:51:31.631 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.333e-06 2024-09-11T06:51:31.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.631 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:31.631 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.574357+0000", 2024-09-11T06:51:31.632 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0013975540000000001 2024-09-11T06:51:31.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.632 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:31.632 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.574398+0000", 2024-09-11T06:51:31.632 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.0908000000000002e-05 2024-09-11T06:51:31.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.632 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:31.632 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.574425+0000", 2024-09-11T06:51:31.632 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6899000000000001e-05 2024-09-11T06:51:31.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.633 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:31.633 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.574604+0000", 2024-09-11T06:51:31.633 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000178359 2024-09-11T06:51:31.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.633 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:31.633 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.574626+0000", 2024-09-11T06:51:31.633 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2768999999999998e-05 2024-09-11T06:51:31.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.633 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:31.633 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.576501+0000", 2024-09-11T06:51:31.633 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001874686 2024-09-11T06:51:31.634 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.634 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:31.634 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.634 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.634 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(osd.3.0:263 2.6 e107/99 2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:6 v 107'47, mlcod=74'43)", 2024-09-11T06:51:31.634 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:30.106025+0000", 2024-09-11T06:51:31.634 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 1.438059252, 2024-09-11T06:51:31.634 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0038585270000000001, 2024-09-11T06:51:31.634 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:31.634 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:31.634 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:31.634 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:31.634 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.635 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:31.635 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.106025+0000", 2024-09-11T06:51:31.635 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.635 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.635 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:31.635 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.106025+0000", 2024-09-11T06:51:31.635 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:31.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.635 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.635 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:31.635 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.106026+0000", 2024-09-11T06:51:31.635 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7399999999999991e-07 2024-09-11T06:51:31.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.636 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.636 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:31.636 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.106038+0000", 2024-09-11T06:51:31.636 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2696e-05 2024-09-11T06:51:31.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.636 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.636 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:31.636 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.106102+0000", 2024-09-11T06:51:31.636 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3627000000000006e-05 2024-09-11T06:51:31.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.636 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.636 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:31.636 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.106126+0000", 2024-09-11T06:51:31.636 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3676000000000002e-05 2024-09-11T06:51:31.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.637 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:31.637 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.106278+0000", 2024-09-11T06:51:31.637 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015224800000000001 2024-09-11T06:51:31.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.637 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:31.637 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.106316+0000", 2024-09-11T06:51:31.637 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.8418999999999999e-05 2024-09-11T06:51:31.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.637 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:31.638 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.109781+0000", 2024-09-11T06:51:31.638 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034645040000000002 2024-09-11T06:51:31.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:31.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:31.638 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:31.638 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.109883+0000", 2024-09-11T06:51:31.638 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000102383 2024-09-11T06:51:31.638 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.638 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:31.638 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.638 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:31.638 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:31.638 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:31.639 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T06:51:31.641 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:51:31.641 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:31.634+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:31.742 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:31.737+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:31.767 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:31.768 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:31.768 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:51:31.768 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:31.782 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:51:31.786 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:51:31.787 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:51:31.803 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:51:31.803 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:51:31.819 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:51:31.819 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:51:31.837 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:51:31.837 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:51:31.855 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:51:31.856 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:51:31.873 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:31.869+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:31.875 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:51:31.876 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:51:31.944 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:31.938+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:31.952 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:31.952 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:31.966 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:51:32.053 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:32.049+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:32.124 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.2 to 1 (10000) 2024-09-11T06:51:32.133 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T06:51:32.139 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:32.139 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:32.147 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:32.141+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:32.153 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-11T06:51:32.160 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:51:32.246 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:32.241+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:32.338 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:32.338 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:32.347 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:32.342+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:32.353 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:51:32.448 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:32.441+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:32.511 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:32.511 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:32.524 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:51:32.549 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:32.545+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:32.639 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:32.639 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:32.650 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:32.649+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:32.652 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:51:32.751 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:32.749+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:32.770 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:32.770 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:32.783 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:51:32.852 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:32.846+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:32.901 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:32.901 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:32.915 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:51:32.953 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:32.953+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:33.054 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:33.049+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:33.078 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:33.078 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:33.092 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:51:33.131 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.4 to 1 (10000) 2024-09-11T06:51:33.131 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is set 2024-09-11T06:51:33.136 INFO:tasks.workunit.client.0.smithi176.stderr: Removing all snapshots: 25% complete... Removing all snapshots: 50% complete... Removing all snapshots: 75% complete... Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2024-09-11T06:51:33.145 INFO:tasks.workunit.client.0.smithi176.stderr:2024-09-11T06:51:33.138+0000 7f3abdbb8640 0 -- 172.21.15.176:0/3214439167 >> [v2:172.21.15.71:6816/2926882521,v1:172.21.15.71:6817/2926882521] conn(0x7f3aa4008c60 msgr2=0x7f3aa400b0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-11T06:51:33.155 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:33.149+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:33.158 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-11T06:51:33.158 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd -p rbd --namespace '' remove testrbdnbd21607 2024-09-11T06:51:33.222 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:33.222 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:33.237 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:51:33.285 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:33.282+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:33.300 INFO:tasks.workunit.client.0.smithi176.stderr:2024-09-11T06:51:33.294+0000 7f7cb286d640 0 -- 172.21.15.176:0/3641776616 >> [v2:172.21.15.89:6816/3878488003,v1:172.21.15.89:6817/3878488003] conn(0x55728739ae10 msgr2=0x55728739d200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-11T06:51:33.323 INFO:tasks.workunit.client.0.smithi176.stderr: Removing image: 6% complete... Removing image: 12% complete... Removing image: 18% complete... Removing image: 25% complete... Removing image: 31% complete... Removing image: 37% complete... Removing image: 43% complete... Removing image: 50% complete... Removing image: 56% complete... Removing image: 62% complete... Removing image: 68% complete... Removing image: 75% complete... Removing image: 81% complete... Removing image: 87% complete... Removing image: 93% complete...2024-09-11T06:51:33.318+0000 7f7cb0e0a640 0 -- 172.21.15.176:0/3641776616 >> [v2:172.21.15.71:6816/2926882521,v1:172.21.15.71:6817/2926882521] conn(0x7f7c9403b830 msgr2=0x7f7c9403dc60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-11T06:51:33.339 INFO:tasks.workunit.client.0.smithi176.stderr: Removing image: 100% complete...done. 2024-09-11T06:51:33.347 INFO:tasks.workunit.client.0.smithi176.stderr:+ for ns in '' ${NS} 2024-09-11T06:51:33.347 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd -p rbd --namespace ns status testrbdnbd21607 2024-09-11T06:51:33.356 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-11T06:51:33.350+0000 7ff972bab640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25182) UID: 0 2024-09-11T06:51:33.397 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:33.397 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:33.408 INFO:tasks.workunit.client.0.smithi176.stdout:Watchers: none 2024-09-11T06:51:33.411 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:51:33.412 INFO:tasks.workunit.client.0.smithi176.stderr:+ for s in 0.5 1 2 4 8 16 32 2024-09-11T06:51:33.412 INFO:tasks.workunit.client.0.smithi176.stderr:+ sleep 0.5 2024-09-11T06:51:33.497 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:33.494+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:33.559 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:33.557+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:33.559 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:33.559 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:33.573 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:51:33.660 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:33.654+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:33.703 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:33.703 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:33.718 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:51:33.765 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:33.761+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:33.836 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:33.836 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:33.851 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:51:33.893 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T06:51:33.897 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:33.897+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:33.914 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd -p rbd --namespace ns status testrbdnbd21607 2024-09-11T06:51:33.914 INFO:tasks.workunit.client.0.smithi176.stderr:+ grep 'Watchers: none' 2024-09-11T06:51:33.961 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:33.958+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:33.974 INFO:tasks.workunit.client.0.smithi176.stdout:Watchers: none 2024-09-11T06:51:33.974 INFO:tasks.workunit.client.0.smithi176.stderr:+ break 2024-09-11T06:51:33.974 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd -p rbd --namespace ns snap purge testrbdnbd21607 2024-09-11T06:51:34.024 INFO:teuthology.orchestra.run.smithi089.stdout:ERROR: (22) Invalid argument 2024-09-11T06:51:34.024 INFO:teuthology.orchestra.run.smithi089.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T06:51:34.041 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:51:34.042 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-11T06:51:34.043 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T06:51:34.059 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-11T06:51:34.059 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T06:51:34.075 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-11T06:51:34.076 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T06:51:34.085 INFO:tasks.ceph.osd.3.smithi089.stderr:2024-09-11T06:51:34.081+0000 7f472aadf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25216) UID: 0 2024-09-11T06:51:34.096 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-11T06:51:34.096 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T06:51:34.116 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-11T06:51:34.116 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T06:51:34.136 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.3 to 1 (10000) 2024-09-11T06:51:34.137 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-11T06:51:34.137 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T06:51:34.149 INFO:tasks.workunit.client.0.smithi176.stderr: Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2024-09-11T06:51:34.160 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-11T06:51:34.161 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd -p rbd --namespace ns remove testrbdnbd21607 2024-09-11T06:51:34.162 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:34.157+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:34.165 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:34.165 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:34.165 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:51:34.165 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:34.179 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:51:34.262 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:34.258+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:34.288 INFO:tasks.workunit.client.0.smithi176.stderr: Removing image: 6% complete... Removing image: 12% complete... Removing image: 18% complete... Removing image: 25% complete... Removing image: 31% complete... Removing image: 37% complete... Removing image: 43% complete... Removing image: 50% complete... Removing image: 56% complete...2024-09-11T06:51:34.282+0000 7f1d947f8640 0 -- 172.21.15.176:0/3725019196 >> [v2:172.21.15.71:6816/2926882521,v1:172.21.15.71:6817/2926882521] conn(0x7f1d78013500 msgr2=0x7f1d7800bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-11T06:51:34.350 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:34.350 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:34.350 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:51:34.350 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:51:34.350 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:34.363 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:34.358+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:34.364 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:51:34.463 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:34.461+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:34.490 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:34.490 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:51:34.490 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:51:34.490 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:51:34.490 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.490 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4558.0:1 3.3 3:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-11T06:51:34.490 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:51:34.490 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 77.395573404999993, 2024-09-11T06:51:34.490 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0085520600000000002, 2024-09-11T06:51:34.490 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:34.491 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:34.491 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:34.491 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:34.491 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4558", 2024-09-11T06:51:34.491 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1478388898", 2024-09-11T06:51:34.491 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:51:34.491 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.491 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:34.491 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.491 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:34.491 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:51:34.492 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.492 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.492 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.492 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:34.492 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:51:34.492 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.492 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.492 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.492 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:34.492 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090533+0000", 2024-09-11T06:51:34.493 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2629999999999999e-06 2024-09-11T06:51:34.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.493 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.493 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:34.493 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090546+0000", 2024-09-11T06:51:34.493 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2918999999999999e-05 2024-09-11T06:51:34.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.493 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.493 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:34.493 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090572+0000", 2024-09-11T06:51:34.494 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6412000000000001e-05 2024-09-11T06:51:34.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.494 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.494 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:34.494 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090594+0000", 2024-09-11T06:51:34.494 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2476e-05 2024-09-11T06:51:34.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.494 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.494 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:34.494 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090938+0000", 2024-09-11T06:51:34.495 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034325099999999998 2024-09-11T06:51:34.495 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.495 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.495 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:34.495 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.091149+0000", 2024-09-11T06:51:34.495 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021151300000000001 2024-09-11T06:51:34.495 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.495 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.495 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:34.495 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.091377+0000", 2024-09-11T06:51:34.495 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022785399999999999 2024-09-11T06:51:34.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.496 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.496 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:34.496 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.092912+0000", 2024-09-11T06:51:34.496 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0015346089999999999 2024-09-11T06:51:34.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.496 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.496 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:34.496 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098919+0000", 2024-09-11T06:51:34.497 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0060076799999999996 2024-09-11T06:51:34.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.497 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.497 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:34.497 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.099009+0000", 2024-09-11T06:51:34.497 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0056000000000006e-05 2024-09-11T06:51:34.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.497 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.497 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:34.497 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.099081+0000", 2024-09-11T06:51:34.498 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2026999999999993e-05 2024-09-11T06:51:34.498 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.498 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:34.498 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.498 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.498 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:51:34.498 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:51:34.498 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 71.354084325000002, 2024-09-11T06:51:34.498 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0071924390000000001, 2024-09-11T06:51:34.499 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:34.499 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:34.499 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:34.499 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:34.499 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.499 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:34.499 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:51:34.499 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.499 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.499 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:34.499 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:51:34.500 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.500 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.500 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:34.500 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132020+0000", 2024-09-11T06:51:34.500 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.846e-06 2024-09-11T06:51:34.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.500 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.500 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:34.500 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135688+0000", 2024-09-11T06:51:34.500 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036677350000000001 2024-09-11T06:51:34.501 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.501 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.501 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:34.501 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135747+0000", 2024-09-11T06:51:34.501 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.9314999999999997e-05 2024-09-11T06:51:34.501 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.501 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.501 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:34.502 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135797+0000", 2024-09-11T06:51:34.502 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9984000000000001e-05 2024-09-11T06:51:34.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.502 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.502 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:34.502 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136143+0000", 2024-09-11T06:51:34.502 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034594299999999998 2024-09-11T06:51:34.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.502 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.502 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:34.502 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136179+0000", 2024-09-11T06:51:34.503 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5636000000000002e-05 2024-09-11T06:51:34.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.503 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:34.503 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.139211+0000", 2024-09-11T06:51:34.503 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030319800000000001 2024-09-11T06:51:34.503 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.503 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:34.503 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.504 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:51:34.504 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:51:34.504 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 71.352021070999996, 2024-09-11T06:51:34.504 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064954139999999997, 2024-09-11T06:51:34.504 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:34.504 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:34.504 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:34.504 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:34.504 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.504 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:34.504 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:51:34.505 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.505 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:34.505 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:51:34.505 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.505 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:34.505 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134084+0000", 2024-09-11T06:51:34.505 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.894e-06 2024-09-11T06:51:34.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.506 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.506 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:34.506 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136513+0000", 2024-09-11T06:51:34.506 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024290969999999999 2024-09-11T06:51:34.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.506 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.506 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:34.506 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136554+0000", 2024-09-11T06:51:34.506 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.0932e-05 2024-09-11T06:51:34.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.506 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.507 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:34.507 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136584+0000", 2024-09-11T06:51:34.507 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0056000000000002e-05 2024-09-11T06:51:34.507 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.507 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.507 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:34.507 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136805+0000", 2024-09-11T06:51:34.507 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022086100000000001 2024-09-11T06:51:34.507 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.507 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.507 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:34.508 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136864+0000", 2024-09-11T06:51:34.508 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.9907000000000001e-05 2024-09-11T06:51:34.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.508 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.508 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:34.508 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.140577+0000", 2024-09-11T06:51:34.508 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0037126669999999998 2024-09-11T06:51:34.508 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.508 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:34.508 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.508 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.509 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:34.509 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:51:34.509 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 71.343730270999998, 2024-09-11T06:51:34.509 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035763180000000002, 2024-09-11T06:51:34.509 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:34.509 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:34.509 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:34.509 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:34.509 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.509 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:34.509 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:51:34.510 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.510 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:34.510 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:51:34.510 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.510 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:34.510 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142374+0000", 2024-09-11T06:51:34.510 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5460000000000001e-06 2024-09-11T06:51:34.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.511 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.511 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:34.511 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143902+0000", 2024-09-11T06:51:34.511 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00152808 2024-09-11T06:51:34.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.511 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.511 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:34.511 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143946+0000", 2024-09-11T06:51:34.511 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3874e-05 2024-09-11T06:51:34.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.512 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.512 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:34.512 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143978+0000", 2024-09-11T06:51:34.512 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2425999999999998e-05 2024-09-11T06:51:34.512 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.512 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.512 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:34.512 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.144177+0000", 2024-09-11T06:51:34.512 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019837500000000001 2024-09-11T06:51:34.512 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.512 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.513 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:34.513 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.144198+0000", 2024-09-11T06:51:34.513 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1639000000000001e-05 2024-09-11T06:51:34.513 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.513 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.513 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:34.513 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.145949+0000", 2024-09-11T06:51:34.513 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0017503779999999999 2024-09-11T06:51:34.513 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.513 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:34.513 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.513 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.513 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.514 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:34.514 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:51:34.514 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 71.339414568999999, 2024-09-11T06:51:34.514 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0057693220000000003, 2024-09-11T06:51:34.514 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:34.514 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:34.514 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:34.514 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:34.514 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.514 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:34.515 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:51:34.515 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.515 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.515 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:34.515 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:51:34.515 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.515 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.515 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:34.515 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146690+0000", 2024-09-11T06:51:34.515 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.607e-06 2024-09-11T06:51:34.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.516 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.516 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:34.516 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148693+0000", 2024-09-11T06:51:34.516 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0020029739999999998 2024-09-11T06:51:34.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.516 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.516 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:34.516 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148735+0000", 2024-09-11T06:51:34.517 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1958e-05 2024-09-11T06:51:34.517 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.517 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.517 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:34.517 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148764+0000", 2024-09-11T06:51:34.517 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8999000000000001e-05 2024-09-11T06:51:34.517 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.517 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.517 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:34.517 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149069+0000", 2024-09-11T06:51:34.517 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000305032 2024-09-11T06:51:34.518 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.518 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.518 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:34.518 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149204+0000", 2024-09-11T06:51:34.518 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013495 2024-09-11T06:51:34.518 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.518 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.518 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:34.518 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.152458+0000", 2024-09-11T06:51:34.518 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0032538020000000001 2024-09-11T06:51:34.519 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.519 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:34.519 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.519 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.519 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.519 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:34.519 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:51:34.519 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 71.338516028000001, 2024-09-11T06:51:34.519 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0042425329999999997, 2024-09-11T06:51:34.520 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:34.520 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:34.520 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:34.520 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:34.520 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.520 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:34.520 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:51:34.520 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.520 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.521 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.521 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:34.521 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:51:34.521 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.521 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.521 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.521 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:34.521 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147588+0000", 2024-09-11T06:51:34.521 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.277e-06 2024-09-11T06:51:34.521 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.521 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.522 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:34.522 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149051+0000", 2024-09-11T06:51:34.522 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001462669 2024-09-11T06:51:34.522 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.522 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.522 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:34.522 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149090+0000", 2024-09-11T06:51:34.522 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.8967999999999997e-05 2024-09-11T06:51:34.522 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.522 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.522 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:34.522 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149120+0000", 2024-09-11T06:51:34.523 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0766000000000003e-05 2024-09-11T06:51:34.523 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.523 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.523 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:34.523 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149395+0000", 2024-09-11T06:51:34.523 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027484900000000002 2024-09-11T06:51:34.523 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.523 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.523 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:34.523 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149439+0000", 2024-09-11T06:51:34.523 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3940999999999998e-05 2024-09-11T06:51:34.524 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.524 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.524 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:34.524 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.151829+0000", 2024-09-11T06:51:34.524 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002390063 2024-09-11T06:51:34.524 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.524 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:34.524 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.524 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.524 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.524 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:34.525 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:51:34.525 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 71.332375397000007, 2024-09-11T06:51:34.525 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0046568219999999997, 2024-09-11T06:51:34.525 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:34.525 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:34.525 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:34.525 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:34.525 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.525 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:34.525 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:51:34.525 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.526 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.526 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.526 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:34.526 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:51:34.526 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.526 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.526 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.526 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:34.526 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153729+0000", 2024-09-11T06:51:34.526 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5629999999999999e-06 2024-09-11T06:51:34.526 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.526 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.527 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:34.527 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155179+0000", 2024-09-11T06:51:34.527 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001450469 2024-09-11T06:51:34.527 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.527 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.527 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:34.527 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155221+0000", 2024-09-11T06:51:34.527 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1448e-05 2024-09-11T06:51:34.527 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.527 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.527 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:34.527 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155249+0000", 2024-09-11T06:51:34.527 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8433e-05 2024-09-11T06:51:34.528 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.528 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.528 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:34.528 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155420+0000", 2024-09-11T06:51:34.528 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017070599999999999 2024-09-11T06:51:34.528 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.528 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.528 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:34.528 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155445+0000", 2024-09-11T06:51:34.528 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4813000000000001e-05 2024-09-11T06:51:34.528 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.528 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.529 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:34.529 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.158384+0000", 2024-09-11T06:51:34.529 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0029393900000000001 2024-09-11T06:51:34.529 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.529 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:34.529 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.529 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.529 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.529 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:34.529 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:51:34.530 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 71.330348647999998, 2024-09-11T06:51:34.530 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0046826020000000001, 2024-09-11T06:51:34.530 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:34.530 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:34.530 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:34.530 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:34.530 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.530 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:34.530 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:51:34.530 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.530 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.530 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.531 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:34.531 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:51:34.531 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.531 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.531 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.531 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:34.531 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155756+0000", 2024-09-11T06:51:34.531 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.384e-06 2024-09-11T06:51:34.531 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.531 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.531 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:34.532 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157205+0000", 2024-09-11T06:51:34.532 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014496260000000001 2024-09-11T06:51:34.532 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.532 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.532 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:34.532 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157252+0000", 2024-09-11T06:51:34.532 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6368000000000001e-05 2024-09-11T06:51:34.532 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.532 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.532 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:34.532 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157286+0000", 2024-09-11T06:51:34.532 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4944999999999998e-05 2024-09-11T06:51:34.532 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.533 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.533 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:34.533 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157540+0000", 2024-09-11T06:51:34.533 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00025307999999999998 2024-09-11T06:51:34.533 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.533 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.533 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:34.533 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157576+0000", 2024-09-11T06:51:34.533 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.6878999999999997e-05 2024-09-11T06:51:34.533 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.533 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.533 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:34.534 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.160437+0000", 2024-09-11T06:51:34.534 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00286032 2024-09-11T06:51:34.534 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.534 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:34.534 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.534 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.534 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.534 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:51:34.534 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.280679+0000", 2024-09-11T06:51:34.534 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 59.205423658999997, 2024-09-11T06:51:34.534 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0040934309999999998, 2024-09-11T06:51:34.534 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:34.534 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:34.535 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:51:34.535 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:34.535 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.535 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:34.535 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280679+0000", 2024-09-11T06:51:34.535 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.535 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.535 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.535 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:34.535 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280679+0000", 2024-09-11T06:51:34.535 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.536 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.536 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:34.536 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280681+0000", 2024-09-11T06:51:34.536 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2529999999999999e-06 2024-09-11T06:51:34.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.536 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.536 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:34.536 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280713+0000", 2024-09-11T06:51:34.536 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.1936000000000001e-05 2024-09-11T06:51:34.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.537 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.537 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:34.537 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280758+0000", 2024-09-11T06:51:34.537 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5124999999999999e-05 2024-09-11T06:51:34.537 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.537 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.537 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:34.537 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280779+0000", 2024-09-11T06:51:34.537 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0936000000000001e-05 2024-09-11T06:51:34.537 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.537 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.538 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:34.538 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280932+0000", 2024-09-11T06:51:34.538 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015220899999999999 2024-09-11T06:51:34.538 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.538 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.538 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:34.538 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.284773+0000", 2024-09-11T06:51:34.538 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0038409719999999998 2024-09-11T06:51:34.538 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.538 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:34.539 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.539 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.539 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:51:34.539 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.293438+0000", 2024-09-11T06:51:34.539 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 59.192664923000002, 2024-09-11T06:51:34.539 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0061479350000000002, 2024-09-11T06:51:34.539 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:34.539 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:34.539 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:51:34.539 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:34.540 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.540 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:34.540 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293438+0000", 2024-09-11T06:51:34.540 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.540 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.540 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.540 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:34.540 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293438+0000", 2024-09-11T06:51:34.540 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.541 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.541 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.541 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:34.541 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293439+0000", 2024-09-11T06:51:34.541 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:51:34.541 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.541 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.541 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:34.542 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293459+0000", 2024-09-11T06:51:34.542 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9922e-05 2024-09-11T06:51:34.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.542 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.542 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:34.542 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293482+0000", 2024-09-11T06:51:34.542 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2969e-05 2024-09-11T06:51:34.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.542 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.542 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:34.542 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293494+0000", 2024-09-11T06:51:34.543 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.131e-05 2024-09-11T06:51:34.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.543 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.543 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:34.543 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.295901+0000", 2024-09-11T06:51:34.543 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024077949999999999 2024-09-11T06:51:34.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.543 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.543 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:34.543 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.299586+0000", 2024-09-11T06:51:34.543 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036845020000000001 2024-09-11T06:51:34.544 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.544 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:34.544 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.544 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.544 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.544 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(client.5239.0:37 2.7 e91/69 2:eab0161e:::rbd_data.1314f0a2bfa2.0000000000000001:head v 91'134, mlcod=91'130)", 2024-09-11T06:51:34.544 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:08.068368+0000", 2024-09-11T06:51:34.544 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 26.417734844999998, 2024-09-11T06:51:34.544 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003850118, 2024-09-11T06:51:34.544 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:34.544 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:34.545 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:34.545 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:34.545 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.545 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:34.545 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068368+0000", 2024-09-11T06:51:34.545 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.545 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.545 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:34.546 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068368+0000", 2024-09-11T06:51:34.546 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.546 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.546 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.546 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:34.546 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068369+0000", 2024-09-11T06:51:34.546 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2699999999999998e-07 2024-09-11T06:51:34.546 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.546 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.547 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:34.547 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068458+0000", 2024-09-11T06:51:34.547 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9075999999999999e-05 2024-09-11T06:51:34.547 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.547 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.547 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:34.547 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068471+0000", 2024-09-11T06:51:34.547 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.269e-05 2024-09-11T06:51:34.547 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.548 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.548 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:34.548 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068480+0000", 2024-09-11T06:51:34.548 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.6199999999999994e-06 2024-09-11T06:51:34.548 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.548 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.548 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:34.548 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.069523+0000", 2024-09-11T06:51:34.549 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001042465 2024-09-11T06:51:34.549 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.549 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.549 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:34.549 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.069542+0000", 2024-09-11T06:51:34.549 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9386e-05 2024-09-11T06:51:34.549 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.549 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.549 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:34.550 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072122+0000", 2024-09-11T06:51:34.550 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025799540000000002 2024-09-11T06:51:34.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.550 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:34.550 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072218+0000", 2024-09-11T06:51:34.550 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.6100000000000005e-05 2024-09-11T06:51:34.550 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.550 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:34.551 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.551 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.551 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(client.5239.0:41 2.7 e91/69 2:eab0161e:::rbd_data.1314f0a2bfa2.0000000000000001:head v 91'138, mlcod=91'132)", 2024-09-11T06:51:34.551 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:08.071899+0000", 2024-09-11T06:51:34.551 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 26.414204067, 2024-09-11T06:51:34.551 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0037511240000000002, 2024-09-11T06:51:34.551 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:34.551 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:34.552 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:34.552 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:34.552 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.552 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:34.552 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.071899+0000", 2024-09-11T06:51:34.552 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.552 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.552 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.552 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:34.552 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.071899+0000", 2024-09-11T06:51:34.553 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.553 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:34.553 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.071900+0000", 2024-09-11T06:51:34.553 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3500000000000005e-07 2024-09-11T06:51:34.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.553 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:34.553 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072027+0000", 2024-09-11T06:51:34.554 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012720299999999999 2024-09-11T06:51:34.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.554 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.554 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:34.554 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072043+0000", 2024-09-11T06:51:34.554 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6475999999999999e-05 2024-09-11T06:51:34.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.554 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.554 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:34.554 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072053+0000", 2024-09-11T06:51:34.554 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2860000000000005e-06 2024-09-11T06:51:34.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.554 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.555 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:34.555 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072256+0000", 2024-09-11T06:51:34.555 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000203876 2024-09-11T06:51:34.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.555 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.555 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:34.555 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072282+0000", 2024-09-11T06:51:34.555 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5811000000000001e-05 2024-09-11T06:51:34.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.555 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.555 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:34.555 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.075544+0000", 2024-09-11T06:51:34.556 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0032621500000000001 2024-09-11T06:51:34.556 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.556 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.556 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:34.556 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.075650+0000", 2024-09-11T06:51:34.556 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000105487 2024-09-11T06:51:34.556 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.556 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:34.556 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.556 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.557 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.557 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.4 101/100 [PushOp(2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head, version: 54'23, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head@54'23, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:51:34.557 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.525431+0000", 2024-09-11T06:51:34.557 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 13.960671505000001, 2024-09-11T06:51:34.557 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0042697100000000003, 2024-09-11T06:51:34.557 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:34.557 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:34.557 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:34.557 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:34.557 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.557 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:34.558 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.525431+0000", 2024-09-11T06:51:34.558 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.558 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.558 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:34.558 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.525431+0000", 2024-09-11T06:51:34.558 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.558 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.558 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:34.558 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.525433+0000", 2024-09-11T06:51:34.558 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5829999999999999e-06 2024-09-11T06:51:34.559 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.559 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.559 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:34.559 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.527430+0000", 2024-09-11T06:51:34.559 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0019966350000000001 2024-09-11T06:51:34.559 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.559 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.559 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:34.559 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.527477+0000", 2024-09-11T06:51:34.559 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7681e-05 2024-09-11T06:51:34.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.560 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:34.560 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.527525+0000", 2024-09-11T06:51:34.560 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.8244000000000001e-05 2024-09-11T06:51:34.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.560 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:34.560 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.527795+0000", 2024-09-11T06:51:34.560 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026935199999999999 2024-09-11T06:51:34.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.561 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.561 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:34.561 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.527816+0000", 2024-09-11T06:51:34.561 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1112e-05 2024-09-11T06:51:34.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.561 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.561 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:34.561 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.529701+0000", 2024-09-11T06:51:34.561 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001885103 2024-09-11T06:51:34.561 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.562 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:34.562 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.562 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.4 101/100 [PushOp(2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head, version: 54'23, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head@54'23, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:34.562 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.540925+0000", 2024-09-11T06:51:34.562 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 13.945178109, 2024-09-11T06:51:34.562 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0040908539999999997, 2024-09-11T06:51:34.562 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:34.562 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:34.563 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:34.563 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:34.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.563 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:34.563 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.540925+0000", 2024-09-11T06:51:34.563 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.563 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.563 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:34.563 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.540925+0000", 2024-09-11T06:51:34.564 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.564 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:34.564 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.540926+0000", 2024-09-11T06:51:34.564 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.813e-06 2024-09-11T06:51:34.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.564 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:34.564 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542405+0000", 2024-09-11T06:51:34.564 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001478011 2024-09-11T06:51:34.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.565 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.565 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:34.565 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542453+0000", 2024-09-11T06:51:34.565 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.8677999999999997e-05 2024-09-11T06:51:34.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.565 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.565 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:34.565 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542483+0000", 2024-09-11T06:51:34.565 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9921999999999999e-05 2024-09-11T06:51:34.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.566 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.566 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:34.566 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542758+0000", 2024-09-11T06:51:34.566 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027447499999999999 2024-09-11T06:51:34.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.566 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.566 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:34.566 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542802+0000", 2024-09-11T06:51:34.566 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4288e-05 2024-09-11T06:51:34.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.567 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.567 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:34.567 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.545016+0000", 2024-09-11T06:51:34.567 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0022136669999999999 2024-09-11T06:51:34.567 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.567 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:34.567 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.567 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.567 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.567 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.4 101/100 [PushOp(2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head, version: 54'23, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head@54'23, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:34.568 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.547946+0000", 2024-09-11T06:51:34.568 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 13.938156621999999, 2024-09-11T06:51:34.568 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035631679999999998, 2024-09-11T06:51:34.568 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:34.568 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:34.568 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:34.568 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:34.568 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.568 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:34.568 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.547946+0000", 2024-09-11T06:51:34.568 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.569 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:34.569 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.547946+0000", 2024-09-11T06:51:34.569 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.569 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:34.569 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.547948+0000", 2024-09-11T06:51:34.569 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4529999999999999e-06 2024-09-11T06:51:34.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.570 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:34.570 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549382+0000", 2024-09-11T06:51:34.570 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014345759999999999 2024-09-11T06:51:34.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.570 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:34.570 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549425+0000", 2024-09-11T06:51:34.570 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.2528999999999999e-05 2024-09-11T06:51:34.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.571 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:34.571 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549453+0000", 2024-09-11T06:51:34.571 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7980999999999999e-05 2024-09-11T06:51:34.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.571 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:34.571 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549623+0000", 2024-09-11T06:51:34.571 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000170753 2024-09-11T06:51:34.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.572 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:34.572 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549644+0000", 2024-09-11T06:51:34.572 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0628999999999998e-05 2024-09-11T06:51:34.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.572 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:34.572 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.551509+0000", 2024-09-11T06:51:34.572 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001865247 2024-09-11T06:51:34.572 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.572 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:34.572 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.573 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(client.6081.0:37 2.4 e101/100 2:27499dce:::rbd_object_map.1314f0a2bfa2:head v 101'172, mlcod=54'22)", 2024-09-11T06:51:34.573 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.549499+0000", 2024-09-11T06:51:34.573 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 13.936603755, 2024-09-11T06:51:34.573 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0087992420000000005, 2024-09-11T06:51:34.573 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:34.573 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:34.573 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:34.573 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:34.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.574 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:34.574 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549499+0000", 2024-09-11T06:51:34.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.574 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:34.574 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549499+0000", 2024-09-11T06:51:34.574 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.575 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:34.575 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549500+0000", 2024-09-11T06:51:34.575 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.217e-06 2024-09-11T06:51:34.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.575 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:34.575 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549511+0000", 2024-09-11T06:51:34.575 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1143e-05 2024-09-11T06:51:34.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.576 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:34.576 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549532+0000", 2024-09-11T06:51:34.576 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0513000000000002e-05 2024-09-11T06:51:34.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.576 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:34.576 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549545+0000", 2024-09-11T06:51:34.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3209e-05 2024-09-11T06:51:34.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:34.577 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.551649+0000", 2024-09-11T06:51:34.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002103696 2024-09-11T06:51:34.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.577 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:34.577 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.551694+0000", 2024-09-11T06:51:34.577 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5674999999999999e-05 2024-09-11T06:51:34.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.578 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:34.578 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.558099+0000", 2024-09-11T06:51:34.578 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064040679999999997 2024-09-11T06:51:34.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.579 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:34.579 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.558298+0000", 2024-09-11T06:51:34.579 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019972100000000001 2024-09-11T06:51:34.579 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.579 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:34.579 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.579 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.4 101/100 [PushOp(2:272abc33:ns::rbd_data.11adc335555f.000000000000000c:head, version: 54'24, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:272abc33:ns::rbd_data.11adc335555f.000000000000000c:head@54'24, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:34.579 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.572958+0000", 2024-09-11T06:51:34.580 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 13.913144329, 2024-09-11T06:51:34.580 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035425080000000002, 2024-09-11T06:51:34.580 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:34.580 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:34.580 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:34.580 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:34.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.580 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:34.580 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.572958+0000", 2024-09-11T06:51:34.580 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.581 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:34.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.572958+0000", 2024-09-11T06:51:34.581 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.581 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:34.581 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.572960+0000", 2024-09-11T06:51:34.582 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.333e-06 2024-09-11T06:51:34.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.582 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:34.582 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.574357+0000", 2024-09-11T06:51:34.582 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0013975540000000001 2024-09-11T06:51:34.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:34.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.574398+0000", 2024-09-11T06:51:34.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.0908000000000002e-05 2024-09-11T06:51:34.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:34.583 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.574425+0000", 2024-09-11T06:51:34.583 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6899000000000001e-05 2024-09-11T06:51:34.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.583 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:34.584 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.574604+0000", 2024-09-11T06:51:34.584 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000178359 2024-09-11T06:51:34.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.584 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:34.584 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.574626+0000", 2024-09-11T06:51:34.584 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2768999999999998e-05 2024-09-11T06:51:34.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.584 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:34.584 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.576501+0000", 2024-09-11T06:51:34.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001874686 2024-09-11T06:51:34.585 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.585 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:34.585 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.585 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(osd.3.0:263 2.6 e107/99 2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:6 v 107'47, mlcod=74'43)", 2024-09-11T06:51:34.585 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:30.106025+0000", 2024-09-11T06:51:34.585 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.3800781899999999, 2024-09-11T06:51:34.585 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0038585270000000001, 2024-09-11T06:51:34.585 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:34.585 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:34.586 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:34.586 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:34.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:34.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.106025+0000", 2024-09-11T06:51:34.586 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.586 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:34.586 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.106025+0000", 2024-09-11T06:51:34.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.587 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:34.587 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.106026+0000", 2024-09-11T06:51:34.587 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7399999999999991e-07 2024-09-11T06:51:34.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.587 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:34.587 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.106038+0000", 2024-09-11T06:51:34.588 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2696e-05 2024-09-11T06:51:34.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.588 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:34.588 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.106102+0000", 2024-09-11T06:51:34.588 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3627000000000006e-05 2024-09-11T06:51:34.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.588 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:34.588 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.106126+0000", 2024-09-11T06:51:34.588 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3676000000000002e-05 2024-09-11T06:51:34.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.589 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:34.589 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.106278+0000", 2024-09-11T06:51:34.589 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015224800000000001 2024-09-11T06:51:34.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.589 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:34.589 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.106316+0000", 2024-09-11T06:51:34.589 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.8418999999999999e-05 2024-09-11T06:51:34.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:34.590 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.109781+0000", 2024-09-11T06:51:34.590 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034645040000000002 2024-09-11T06:51:34.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.590 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:34.590 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.109883+0000", 2024-09-11T06:51:34.590 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000102383 2024-09-11T06:51:34.591 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.591 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:34.591 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.591 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5723.0:82 2.4 2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head [delete] snapc 4=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e111)", 2024-09-11T06:51:34.591 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:34.289044+0000", 2024-09-11T06:51:34.591 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.197058493, 2024-09-11T06:51:34.591 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064595520000000003, 2024-09-11T06:51:34.592 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:34.592 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:34.592 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:34.592 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:34.592 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5723", 2024-09-11T06:51:34.592 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3725019196", 2024-09-11T06:51:34.592 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 82 2024-09-11T06:51:34.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.592 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:34.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.593 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:34.593 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.289044+0000", 2024-09-11T06:51:34.593 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.593 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:34.593 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.289044+0000", 2024-09-11T06:51:34.593 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.594 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:34.594 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.289046+0000", 2024-09-11T06:51:34.594 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.903e-06 2024-09-11T06:51:34.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.594 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:34.594 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.289052+0000", 2024-09-11T06:51:34.594 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7969999999999997e-06 2024-09-11T06:51:34.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.595 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:34.595 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.289066+0000", 2024-09-11T06:51:34.595 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3521e-05 2024-09-11T06:51:34.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.595 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:34.595 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.289073+0000", 2024-09-11T06:51:34.595 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.8180000000000004e-06 2024-09-11T06:51:34.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.595 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:34.596 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.289184+0000", 2024-09-11T06:51:34.596 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000110561 2024-09-11T06:51:34.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.596 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:34.596 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.289375+0000", 2024-09-11T06:51:34.596 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019082100000000001 2024-09-11T06:51:34.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.596 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:34.596 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.289557+0000", 2024-09-11T06:51:34.597 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018186999999999999 2024-09-11T06:51:34.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.597 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:34.597 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.292237+0000", 2024-09-11T06:51:34.597 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0026806849999999999 2024-09-11T06:51:34.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.597 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:34.598 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.295362+0000", 2024-09-11T06:51:34.598 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031244549999999999 2024-09-11T06:51:34.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.598 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:34.598 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.295444+0000", 2024-09-11T06:51:34.598 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2074000000000001e-05 2024-09-11T06:51:34.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.598 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:34.598 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.295504+0000", 2024-09-11T06:51:34.599 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0047000000000003e-05 2024-09-11T06:51:34.599 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.599 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:34.599 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.599 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5723.0:93 2.4 2:272abc33:ns::rbd_data.11adc335555f.000000000000000c:head [delete] snapc 4=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e111)", 2024-09-11T06:51:34.599 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:34.295765+0000", 2024-09-11T06:51:34.599 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.19033736400000001, 2024-09-11T06:51:34.599 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052823560000000002, 2024-09-11T06:51:34.600 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:34.600 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:34.600 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:34.600 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:34.600 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5723", 2024-09-11T06:51:34.600 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3725019196", 2024-09-11T06:51:34.600 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 93 2024-09-11T06:51:34.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.600 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:34.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.600 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:34.600 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.295765+0000", 2024-09-11T06:51:34.601 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.601 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:34.601 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.295765+0000", 2024-09-11T06:51:34.601 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:34.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.601 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:34.601 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.295767+0000", 2024-09-11T06:51:34.601 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2860000000000001e-06 2024-09-11T06:51:34.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.602 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:34.602 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.295772+0000", 2024-09-11T06:51:34.602 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.378e-06 2024-09-11T06:51:34.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.602 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:34.602 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.295786+0000", 2024-09-11T06:51:34.602 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3543999999999999e-05 2024-09-11T06:51:34.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.603 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:34.603 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.295792+0000", 2024-09-11T06:51:34.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5509999999999996e-06 2024-09-11T06:51:34.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.603 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:34.603 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.295851+0000", 2024-09-11T06:51:34.603 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8649e-05 2024-09-11T06:51:34.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.604 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:34.604 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.296018+0000", 2024-09-11T06:51:34.604 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016670399999999999 2024-09-11T06:51:34.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.604 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:34.604 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.296167+0000", 2024-09-11T06:51:34.604 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014995300000000001 2024-09-11T06:51:34.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.604 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:34.605 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298223+0000", 2024-09-11T06:51:34.605 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0020557430000000001 2024-09-11T06:51:34.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.605 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:34.605 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.300936+0000", 2024-09-11T06:51:34.605 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027126670000000002 2024-09-11T06:51:34.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.605 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:34.605 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.300982+0000", 2024-09-11T06:51:34.605 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5745999999999999e-05 2024-09-11T06:51:34.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:34.606 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:34.606 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:34.606 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.301048+0000", 2024-09-11T06:51:34.606 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.6135000000000003e-05 2024-09-11T06:51:34.606 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.606 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:34.606 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.606 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:34.606 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:34.607 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:34.608 INFO:tasks.workunit.client.0.smithi176.stderr: Removing image: 62% complete... Removing image: 68% complete... Removing image: 75% complete... Removing image: 81% complete... Removing image: 87% complete... Removing image: 93% complete...2024-09-11T06:51:34.510+0000 7f1d947f8640 0 -- 172.21.15.176:0/3725019196 >> [v2:172.21.15.89:6808/558942315,v1:172.21.15.89:6809/558942315] conn(0x5624f836bf60 msgr2=0x5624f836e350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-11T06:51:34.608 INFO:tasks.workunit.client.0.smithi176.stderr: Removing image: 100% complete...done. 2024-09-11T06:51:34.609 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd namespace remove rbd/ns 2024-09-11T06:51:34.609 INFO:tasks.workunit.client.0.smithi176.stderr:+ rbd config global rm global rbd_default_pool 2024-09-11T06:51:34.609 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:51:34.610 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:34.601+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:34.648 INFO:tasks.workunit.client.0.smithi176.stderr:+ ceph osd pool delete new_default_pool21607 new_default_pool21607 --yes-i-really-really-mean-it 2024-09-11T06:51:34.709 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:34.705+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:34.785 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:34.786 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:34.786 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:51:34.786 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:34.799 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:51:34.810 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-11T06:51:34.806+0000 7f5be4a5a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25179) UID: 0 2024-09-11T06:51:34.911 INFO:tasks.ceph.osd.4.smithi089.stderr:2024-09-11T06:51:34.909+0000 7fd526c1e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25217) UID: 0 2024-09-11T06:51:34.959 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:34.960 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:34.960 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:51:34.960 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:51:34.960 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:34.975 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:51:35.012 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:35.006+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:35.113 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:35.113+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:35.143 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.5 to 1 (10000) 2024-09-11T06:51:35.145 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:35.145 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:51:35.145 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:51:35.145 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:51:35.145 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.145 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:51:35.145 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:51:35.145 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 143.91057237699999, 2024-09-11T06:51:35.145 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010553425999999999, 2024-09-11T06:51:35.145 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:35.145 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:35.146 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.146 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:35.146 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:51:35.146 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:51:35.146 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:51:35.146 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.146 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:35.146 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.146 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:35.146 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:51:35.146 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.146 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.146 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.146 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:35.146 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:51:35.147 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.147 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.147 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.147 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:35.147 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230274+0000", 2024-09-11T06:51:35.147 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.533e-06 2024-09-11T06:51:35.147 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.147 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.147 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:35.147 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230285+0000", 2024-09-11T06:51:35.147 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0540000000000001e-05 2024-09-11T06:51:35.147 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.147 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.147 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:35.148 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230309+0000", 2024-09-11T06:51:35.148 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4223000000000002e-05 2024-09-11T06:51:35.148 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.148 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.148 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.148 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230327+0000", 2024-09-11T06:51:35.148 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:51:35.148 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.148 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.148 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.148 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230478+0000", 2024-09-11T06:51:35.148 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000151459 2024-09-11T06:51:35.148 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.148 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.149 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:35.149 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230490+0000", 2024-09-11T06:51:35.149 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.147e-05 2024-09-11T06:51:35.149 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.149 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.149 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.149 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231803+0000", 2024-09-11T06:51:35.149 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001313158 2024-09-11T06:51:35.149 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.149 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.149 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:51:35.149 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231817+0000", 2024-09-11T06:51:35.149 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4197e-05 2024-09-11T06:51:35.150 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.150 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.150 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.150 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.235945+0000", 2024-09-11T06:51:35.150 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0041282480000000002 2024-09-11T06:51:35.150 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.150 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.150 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:35.150 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236312+0000", 2024-09-11T06:51:35.150 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036652700000000001 2024-09-11T06:51:35.150 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.150 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.150 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:35.150 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236689+0000", 2024-09-11T06:51:35.151 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000377117 2024-09-11T06:51:35.151 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.151 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.151 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:35.151 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238460+0000", 2024-09-11T06:51:35.151 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001770918 2024-09-11T06:51:35.151 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.151 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.151 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:35.151 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240627+0000", 2024-09-11T06:51:35.151 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021670779999999998 2024-09-11T06:51:35.152 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.152 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.152 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:35.152 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240751+0000", 2024-09-11T06:51:35.152 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012384900000000001 2024-09-11T06:51:35.152 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.152 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.152 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:35.152 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240821+0000", 2024-09-11T06:51:35.152 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0516e-05 2024-09-11T06:51:35.152 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.152 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:35.152 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.153 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.153 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.153 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-11T06:51:35.153 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:51:35.153 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 142.90840153400001, 2024-09-11T06:51:35.153 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069904770000000001, 2024-09-11T06:51:35.153 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:35.153 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:35.153 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.153 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:35.153 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:51:35.153 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:51:35.154 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 5 2024-09-11T06:51:35.154 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.154 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:35.154 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.154 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:35.154 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:51:35.154 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.154 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.154 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.154 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:35.154 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:51:35.154 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.154 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.155 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.155 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:35.155 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232442+0000", 2024-09-11T06:51:35.155 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.044e-06 2024-09-11T06:51:35.155 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.155 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.155 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:35.155 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232460+0000", 2024-09-11T06:51:35.155 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7863000000000002e-05 2024-09-11T06:51:35.155 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.155 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.155 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:35.155 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232492+0000", 2024-09-11T06:51:35.156 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2812999999999999e-05 2024-09-11T06:51:35.156 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.156 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.156 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.156 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232510+0000", 2024-09-11T06:51:35.156 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7552999999999999e-05 2024-09-11T06:51:35.156 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.156 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.156 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.156 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232640+0000", 2024-09-11T06:51:35.156 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013029900000000001 2024-09-11T06:51:35.156 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.157 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.157 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:35.157 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232650+0000", 2024-09-11T06:51:35.157 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0023999999999999e-05 2024-09-11T06:51:35.157 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.157 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.157 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.157 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235641+0000", 2024-09-11T06:51:35.157 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00299089 2024-09-11T06:51:35.157 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.157 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.157 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:35.157 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235980+0000", 2024-09-11T06:51:35.158 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000338907 2024-09-11T06:51:35.158 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.158 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.158 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:35.158 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.236351+0000", 2024-09-11T06:51:35.158 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037087699999999998 2024-09-11T06:51:35.158 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.158 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.158 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:35.158 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238074+0000", 2024-09-11T06:51:35.158 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001722859 2024-09-11T06:51:35.158 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.159 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.159 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:35.159 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239160+0000", 2024-09-11T06:51:35.159 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001086074 2024-09-11T06:51:35.159 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.159 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.159 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:35.159 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239299+0000", 2024-09-11T06:51:35.159 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013933500000000001 2024-09-11T06:51:35.159 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.159 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.160 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:35.160 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239429+0000", 2024-09-11T06:51:35.160 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129939 2024-09-11T06:51:35.160 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.160 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:35.160 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.160 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.160 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.160 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-11T06:51:35.160 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:51:35.160 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 141.894313582, 2024-09-11T06:51:35.161 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069253250000000004, 2024-09-11T06:51:35.161 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:35.161 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:35.161 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.161 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:35.161 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:51:35.161 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:51:35.161 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 7 2024-09-11T06:51:35.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.161 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:35.162 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.162 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:35.162 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:51:35.162 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.162 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.162 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:35.162 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:51:35.162 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.162 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.163 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:35.163 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246530+0000", 2024-09-11T06:51:35.163 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9469999999999999e-06 2024-09-11T06:51:35.163 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.163 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.163 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:35.163 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246545+0000", 2024-09-11T06:51:35.163 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5973000000000001e-05 2024-09-11T06:51:35.163 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.163 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.163 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:35.164 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246578+0000", 2024-09-11T06:51:35.164 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2759999999999998e-05 2024-09-11T06:51:35.164 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.164 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.164 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.164 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246595+0000", 2024-09-11T06:51:35.164 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6939000000000002e-05 2024-09-11T06:51:35.164 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.164 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.164 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.165 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246770+0000", 2024-09-11T06:51:35.165 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000174723 2024-09-11T06:51:35.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.165 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.165 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:35.165 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246789+0000", 2024-09-11T06:51:35.165 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9159000000000001e-05 2024-09-11T06:51:35.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.165 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.165 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.166 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250337+0000", 2024-09-11T06:51:35.166 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035480759999999998 2024-09-11T06:51:35.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.166 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.166 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:35.166 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250549+0000", 2024-09-11T06:51:35.166 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021192199999999999 2024-09-11T06:51:35.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.166 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.166 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:35.166 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250750+0000", 2024-09-11T06:51:35.167 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020104199999999999 2024-09-11T06:51:35.167 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.167 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.167 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:35.167 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.252455+0000", 2024-09-11T06:51:35.167 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001705249 2024-09-11T06:51:35.167 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.167 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.167 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:35.167 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253305+0000", 2024-09-11T06:51:35.167 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00084941200000000002 2024-09-11T06:51:35.167 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.167 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.167 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:35.168 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253388+0000", 2024-09-11T06:51:35.168 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3642999999999999e-05 2024-09-11T06:51:35.168 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.168 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.168 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:35.168 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253452+0000", 2024-09-11T06:51:35.168 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3479999999999995e-05 2024-09-11T06:51:35.168 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.168 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:35.168 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.168 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.168 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.169 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:57 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:35.169 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.568529+0000", 2024-09-11T06:51:35.169 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 46.572310662, 2024-09-11T06:51:35.169 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004228229, 2024-09-11T06:51:35.169 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:35.169 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:35.169 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:35.169 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:35.169 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:35.169 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:35.169 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 57 2024-09-11T06:51:35.169 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.169 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:35.170 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.170 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:35.170 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568529+0000", 2024-09-11T06:51:35.170 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.170 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.170 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.170 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:35.170 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568529+0000", 2024-09-11T06:51:35.170 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.170 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.170 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.170 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:35.171 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568532+0000", 2024-09-11T06:51:35.171 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0000000000000001e-06 2024-09-11T06:51:35.171 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.171 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.171 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:35.171 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568547+0000", 2024-09-11T06:51:35.171 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4219999999999999e-05 2024-09-11T06:51:35.171 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.171 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.171 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:35.171 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568580+0000", 2024-09-11T06:51:35.171 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.3328999999999999e-05 2024-09-11T06:51:35.172 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.172 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.172 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.172 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568596+0000", 2024-09-11T06:51:35.172 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5990000000000001e-05 2024-09-11T06:51:35.172 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.172 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.172 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.172 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568728+0000", 2024-09-11T06:51:35.172 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013175400000000001 2024-09-11T06:51:35.172 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.173 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.173 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:35.173 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568738+0000", 2024-09-11T06:51:35.173 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0196e-05 2024-09-11T06:51:35.173 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.173 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.173 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.173 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.572464+0000", 2024-09-11T06:51:35.173 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0037264160000000002 2024-09-11T06:51:35.173 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.173 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.174 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:35.174 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.572584+0000", 2024-09-11T06:51:35.174 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000119438 2024-09-11T06:51:35.174 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.174 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.174 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:35.174 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.572758+0000", 2024-09-11T06:51:35.174 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017388599999999999 2024-09-11T06:51:35.174 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.174 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:35.174 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.174 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.175 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.175 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5386.0:17 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [watch unwatch cookie 94226507267584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:35.175 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:12.563293+0000", 2024-09-11T06:51:35.175 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 22.57754692, 2024-09-11T06:51:35.175 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0028623239999999999, 2024-09-11T06:51:35.175 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:35.175 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:35.175 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.175 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:35.175 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5386", 2024-09-11T06:51:35.175 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3377056437", 2024-09-11T06:51:35.176 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 17 2024-09-11T06:51:35.176 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.176 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:35.176 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.176 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:35.176 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563293+0000", 2024-09-11T06:51:35.176 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.176 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.176 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.176 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:35.176 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563293+0000", 2024-09-11T06:51:35.176 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.177 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.177 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.177 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:35.177 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563294+0000", 2024-09-11T06:51:35.177 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5300000000000004e-07 2024-09-11T06:51:35.177 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.177 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.177 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:35.177 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563296+0000", 2024-09-11T06:51:35.177 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6079999999999998e-06 2024-09-11T06:51:35.177 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.178 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.178 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:35.178 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563302+0000", 2024-09-11T06:51:35.178 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2549999999999997e-06 2024-09-11T06:51:35.178 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.178 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.178 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.178 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563305+0000", 2024-09-11T06:51:35.178 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2949999999999998e-06 2024-09-11T06:51:35.178 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.178 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.178 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.178 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563792+0000", 2024-09-11T06:51:35.179 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00048731399999999998 2024-09-11T06:51:35.179 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.179 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.179 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:35.179 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563899+0000", 2024-09-11T06:51:35.179 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000106977 2024-09-11T06:51:35.179 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.179 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.179 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:35.179 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.564145+0000", 2024-09-11T06:51:35.179 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00024587600000000002 2024-09-11T06:51:35.179 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.180 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.180 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:35.180 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.564760+0000", 2024-09-11T06:51:35.180 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00061473099999999998 2024-09-11T06:51:35.180 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.180 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.180 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:35.180 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566046+0000", 2024-09-11T06:51:35.180 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012856810000000001 2024-09-11T06:51:35.180 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.180 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.181 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:35.181 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566085+0000", 2024-09-11T06:51:35.181 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.9499000000000002e-05 2024-09-11T06:51:35.181 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.181 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.181 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:35.181 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566156+0000", 2024-09-11T06:51:35.181 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0434999999999999e-05 2024-09-11T06:51:35.181 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.181 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:35.181 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.181 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.182 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.182 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5826.0:131 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.client_list in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:35.182 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:12.563833+0000", 2024-09-11T06:51:35.182 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 22.577007364, 2024-09-11T06:51:35.182 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0028764329999999999, 2024-09-11T06:51:35.182 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:35.182 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:35.182 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:35.182 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:35.182 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5826", 2024-09-11T06:51:35.182 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2236190565", 2024-09-11T06:51:35.183 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 131 2024-09-11T06:51:35.183 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.183 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:35.183 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.183 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:35.183 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563833+0000", 2024-09-11T06:51:35.183 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.183 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.183 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.183 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:35.184 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563833+0000", 2024-09-11T06:51:35.184 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.184 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.184 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.184 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:35.184 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563835+0000", 2024-09-11T06:51:35.184 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8109999999999999e-06 2024-09-11T06:51:35.184 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.184 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.184 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:35.185 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563840+0000", 2024-09-11T06:51:35.185 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7599999999999999e-06 2024-09-11T06:51:35.185 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.185 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.185 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:35.185 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563854+0000", 2024-09-11T06:51:35.185 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3332999999999999e-05 2024-09-11T06:51:35.185 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.185 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.185 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.185 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563874+0000", 2024-09-11T06:51:35.185 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0630999999999999e-05 2024-09-11T06:51:35.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.186 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.186 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.186 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.564553+0000", 2024-09-11T06:51:35.186 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00067821199999999998 2024-09-11T06:51:35.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.186 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.186 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:35.186 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.564610+0000", 2024-09-11T06:51:35.186 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7494999999999998e-05 2024-09-11T06:51:35.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.186 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.186 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.186 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566240+0000", 2024-09-11T06:51:35.187 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0016295940000000001 2024-09-11T06:51:35.187 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.187 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.187 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:35.187 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566365+0000", 2024-09-11T06:51:35.187 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000124876 2024-09-11T06:51:35.187 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.187 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.187 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:35.187 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566709+0000", 2024-09-11T06:51:35.187 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034472099999999998 2024-09-11T06:51:35.187 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.187 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:35.187 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.187 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.188 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.188 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5447.0:10 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T06:51:35.188 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:13.624024+0000", 2024-09-11T06:51:35.188 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 21.516816470999999, 2024-09-11T06:51:35.188 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0038665190000000001, 2024-09-11T06:51:35.188 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:35.188 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:35.188 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:35.188 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:35.188 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5447", 2024-09-11T06:51:35.188 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3862443986", 2024-09-11T06:51:35.188 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 10 2024-09-11T06:51:35.188 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.188 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:35.189 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.189 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:35.189 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624024+0000", 2024-09-11T06:51:35.189 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.189 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.189 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:35.189 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624024+0000", 2024-09-11T06:51:35.189 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.189 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.189 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:35.189 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624026+0000", 2024-09-11T06:51:35.189 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8090000000000001e-06 2024-09-11T06:51:35.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.190 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.190 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:35.190 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624037+0000", 2024-09-11T06:51:35.190 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.061e-05 2024-09-11T06:51:35.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.190 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.190 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:35.190 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624059+0000", 2024-09-11T06:51:35.190 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2269e-05 2024-09-11T06:51:35.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.190 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.190 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.190 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624074+0000", 2024-09-11T06:51:35.191 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4982e-05 2024-09-11T06:51:35.191 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.191 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.191 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.191 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624238+0000", 2024-09-11T06:51:35.191 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163191 2024-09-11T06:51:35.191 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.191 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.191 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:35.191 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624261+0000", 2024-09-11T06:51:35.191 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3779000000000001e-05 2024-09-11T06:51:35.191 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.191 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.191 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.191 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.627590+0000", 2024-09-11T06:51:35.192 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033287450000000001 2024-09-11T06:51:35.192 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.192 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.192 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:35.192 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.627723+0000", 2024-09-11T06:51:35.192 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013311599999999999 2024-09-11T06:51:35.192 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.192 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.192 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:35.192 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.627890+0000", 2024-09-11T06:51:35.192 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016701800000000001 2024-09-11T06:51:35.192 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.192 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:35.192 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.193 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.193 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.6081.0:32 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.set_minimum_set in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-11T06:51:35.193 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.541248+0000", 2024-09-11T06:51:35.193 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 14.599592144000001, 2024-09-11T06:51:35.193 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0028300869999999998, 2024-09-11T06:51:35.193 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:35.193 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:35.193 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:35.193 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:35.193 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.6081", 2024-09-11T06:51:35.193 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3800899964", 2024-09-11T06:51:35.193 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 32 2024-09-11T06:51:35.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.194 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:35.194 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.194 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:35.194 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541248+0000", 2024-09-11T06:51:35.194 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.194 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.194 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.194 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:35.194 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541248+0000", 2024-09-11T06:51:35.194 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.194 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.194 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.194 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:35.194 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541251+0000", 2024-09-11T06:51:35.195 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5299999999999999e-06 2024-09-11T06:51:35.195 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.195 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.195 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:35.195 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541266+0000", 2024-09-11T06:51:35.195 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5139e-05 2024-09-11T06:51:35.195 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.195 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.195 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:35.195 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541298+0000", 2024-09-11T06:51:35.195 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2152e-05 2024-09-11T06:51:35.195 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.195 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.195 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.195 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541313+0000", 2024-09-11T06:51:35.196 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5532999999999998e-05 2024-09-11T06:51:35.196 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.196 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.196 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.196 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541554+0000", 2024-09-11T06:51:35.196 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000240306 2024-09-11T06:51:35.196 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.196 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.196 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:35.196 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541957+0000", 2024-09-11T06:51:35.196 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00040357799999999998 2024-09-11T06:51:35.196 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.196 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.196 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:35.197 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.544078+0000", 2024-09-11T06:51:35.197 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021208490000000002 2024-09-11T06:51:35.197 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.197 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:35.197 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.197 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.197 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.197 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.6081.0:33 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-11T06:51:35.197 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.541352+0000", 2024-09-11T06:51:35.197 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 14.599488019000001, 2024-09-11T06:51:35.197 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034084850000000002, 2024-09-11T06:51:35.197 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:35.197 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:35.197 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.197 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:35.198 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.6081", 2024-09-11T06:51:35.198 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3800899964", 2024-09-11T06:51:35.198 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 33 2024-09-11T06:51:35.198 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.198 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:35.198 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.198 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:35.198 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541352+0000", 2024-09-11T06:51:35.198 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.198 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.198 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.198 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:35.198 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541352+0000", 2024-09-11T06:51:35.199 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.199 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.199 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:35.199 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541353+0000", 2024-09-11T06:51:35.199 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7000000000000003e-07 2024-09-11T06:51:35.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.199 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.199 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:35.199 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541358+0000", 2024-09-11T06:51:35.199 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7400000000000004e-06 2024-09-11T06:51:35.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.199 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.200 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:35.200 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541374+0000", 2024-09-11T06:51:35.200 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5670000000000001e-05 2024-09-11T06:51:35.200 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.200 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.200 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.200 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541386+0000", 2024-09-11T06:51:35.200 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2359e-05 2024-09-11T06:51:35.200 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.200 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.200 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.200 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542684+0000", 2024-09-11T06:51:35.201 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012984590000000001 2024-09-11T06:51:35.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.201 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.201 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:35.201 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542820+0000", 2024-09-11T06:51:35.201 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013536700000000001 2024-09-11T06:51:35.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.201 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.201 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:35.201 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542972+0000", 2024-09-11T06:51:35.201 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000151843 2024-09-11T06:51:35.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.201 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.201 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:35.202 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.544110+0000", 2024-09-11T06:51:35.202 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011381060000000001 2024-09-11T06:51:35.202 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.202 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.202 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:35.202 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.544510+0000", 2024-09-11T06:51:35.202 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00040019799999999999 2024-09-11T06:51:35.202 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.202 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.202 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:35.202 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.544643+0000", 2024-09-11T06:51:35.202 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013313499999999999 2024-09-11T06:51:35.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.203 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.203 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:35.203 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.544761+0000", 2024-09-11T06:51:35.203 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000117638 2024-09-11T06:51:35.203 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.203 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:35.203 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.203 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.203 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5542.0:10 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-11T06:51:35.203 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:22.002114+0000", 2024-09-11T06:51:35.203 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 13.138726569999999, 2024-09-11T06:51:35.204 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00564235, 2024-09-11T06:51:35.204 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:35.204 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:35.204 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:35.204 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:35.204 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5542", 2024-09-11T06:51:35.204 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1546942093", 2024-09-11T06:51:35.204 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 10 2024-09-11T06:51:35.204 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.204 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:35.204 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.204 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:35.205 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002114+0000", 2024-09-11T06:51:35.205 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.205 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.205 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:35.205 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002114+0000", 2024-09-11T06:51:35.205 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.205 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.205 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:35.206 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002115+0000", 2024-09-11T06:51:35.206 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.872e-06 2024-09-11T06:51:35.206 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.206 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.206 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:35.206 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002123+0000", 2024-09-11T06:51:35.206 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3309999999999998e-06 2024-09-11T06:51:35.206 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.206 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.206 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:35.206 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002136+0000", 2024-09-11T06:51:35.207 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2816000000000001e-05 2024-09-11T06:51:35.207 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.207 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.207 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.207 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002145+0000", 2024-09-11T06:51:35.207 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T06:51:35.210 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-11T06:51:35.210 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:51:35.210 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.8570000000000007e-06 2024-09-11T06:51:35.210 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.211 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.211 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.211 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002214+0000", 2024-09-11T06:51:35.211 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.8832999999999996e-05 2024-09-11T06:51:35.211 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.211 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.211 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:35.211 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002220+0000", 2024-09-11T06:51:35.211 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4249999999999997e-06 2024-09-11T06:51:35.211 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.211 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.212 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.212 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.007445+0000", 2024-09-11T06:51:35.212 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052251659999999998 2024-09-11T06:51:35.212 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.212 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.212 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:35.212 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.007586+0000", 2024-09-11T06:51:35.212 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014103100000000001 2024-09-11T06:51:35.212 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.212 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.213 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:35.213 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.007756+0000", 2024-09-11T06:51:35.213 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017001900000000001 2024-09-11T06:51:35.213 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.213 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:35.213 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.213 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.213 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.213 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5542.0:21 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.create in=13b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-11T06:51:35.213 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:22.028810+0000", 2024-09-11T06:51:35.213 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 13.112030526, 2024-09-11T06:51:35.213 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034172709999999999, 2024-09-11T06:51:35.214 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:35.214 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:35.214 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.214 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:35.214 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5542", 2024-09-11T06:51:35.214 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1546942093", 2024-09-11T06:51:35.214 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 21 2024-09-11T06:51:35.214 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.214 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:35.214 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.214 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:35.214 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.028810+0000", 2024-09-11T06:51:35.215 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.215 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.215 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.215 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:35.215 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.028810+0000", 2024-09-11T06:51:35.215 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.215 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.215 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.215 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:35.215 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.028812+0000", 2024-09-11T06:51:35.215 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8100000000000002e-06 2024-09-11T06:51:35.215 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.215 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.216 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:35.216 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.028826+0000", 2024-09-11T06:51:35.216 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3882e-05 2024-09-11T06:51:35.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.216 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.216 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:35.216 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.028859+0000", 2024-09-11T06:51:35.216 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2225999999999999e-05 2024-09-11T06:51:35.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.216 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.217 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.217 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.028880+0000", 2024-09-11T06:51:35.217 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1755000000000001e-05 2024-09-11T06:51:35.217 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.217 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.217 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.217 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.029048+0000", 2024-09-11T06:51:35.217 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016811699999999999 2024-09-11T06:51:35.217 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.217 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.217 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:35.217 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.029198+0000", 2024-09-11T06:51:35.217 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000149753 2024-09-11T06:51:35.218 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.218 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.218 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:35.218 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.029612+0000", 2024-09-11T06:51:35.218 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00041426899999999999 2024-09-11T06:51:35.218 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.218 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.218 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:35.218 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.030882+0000", 2024-09-11T06:51:35.218 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012693030000000001 2024-09-11T06:51:35.218 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.218 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.219 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:35.219 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.032087+0000", 2024-09-11T06:51:35.219 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012054279999999999 2024-09-11T06:51:35.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.219 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.219 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:35.219 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.032167+0000", 2024-09-11T06:51:35.219 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.0177000000000002e-05 2024-09-11T06:51:35.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.219 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.219 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:35.219 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.032227+0000", 2024-09-11T06:51:35.219 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.9551e-05 2024-09-11T06:51:35.220 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.220 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:35.220 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.220 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.220 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.220 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5542.0:23 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.tag_create in=17b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-11T06:51:35.220 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:22.033580+0000", 2024-09-11T06:51:35.220 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 13.107260075999999, 2024-09-11T06:51:35.220 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002804384, 2024-09-11T06:51:35.220 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:35.220 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:35.220 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.221 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:35.221 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5542", 2024-09-11T06:51:35.221 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1546942093", 2024-09-11T06:51:35.221 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 23 2024-09-11T06:51:35.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.221 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:35.221 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.221 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:35.221 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033580+0000", 2024-09-11T06:51:35.221 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.222 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.222 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:35.222 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033580+0000", 2024-09-11T06:51:35.222 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.222 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.222 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:35.222 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033581+0000", 2024-09-11T06:51:35.222 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.217e-06 2024-09-11T06:51:35.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.222 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.222 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:35.222 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033591+0000", 2024-09-11T06:51:35.222 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2660000000000007e-06 2024-09-11T06:51:35.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.223 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:35.223 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033611+0000", 2024-09-11T06:51:35.223 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0418999999999999e-05 2024-09-11T06:51:35.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.223 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.223 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033628+0000", 2024-09-11T06:51:35.223 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7316000000000002e-05 2024-09-11T06:51:35.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.223 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.223 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033803+0000", 2024-09-11T06:51:35.223 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017436200000000001 2024-09-11T06:51:35.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.224 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.224 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:35.224 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033944+0000", 2024-09-11T06:51:35.224 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000141688 2024-09-11T06:51:35.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.224 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.224 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:35.224 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.034475+0000", 2024-09-11T06:51:35.224 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00053032399999999999 2024-09-11T06:51:35.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.224 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.224 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:35.224 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.035684+0000", 2024-09-11T06:51:35.225 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001209452 2024-09-11T06:51:35.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.225 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:35.225 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.036254+0000", 2024-09-11T06:51:35.225 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00056982599999999995 2024-09-11T06:51:35.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.225 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:35.225 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.036328+0000", 2024-09-11T06:51:35.225 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4499999999999995e-05 2024-09-11T06:51:35.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.225 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:35.226 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.036384+0000", 2024-09-11T06:51:35.226 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.6014000000000003e-05 2024-09-11T06:51:35.226 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.226 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:35.226 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.226 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.226 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5536.0:76 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-11T06:51:35.226 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:25.786870+0000", 2024-09-11T06:51:35.226 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 9.3539699879999993, 2024-09-11T06:51:35.226 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0038803930000000002, 2024-09-11T06:51:35.226 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:35.226 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:35.226 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:35.227 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:35.227 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5536", 2024-09-11T06:51:35.227 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/612379138", 2024-09-11T06:51:35.227 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 76 2024-09-11T06:51:35.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.227 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:35.227 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.227 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:35.227 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.786870+0000", 2024-09-11T06:51:35.227 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.227 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.227 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:35.227 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.786870+0000", 2024-09-11T06:51:35.227 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.228 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:35.228 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.786873+0000", 2024-09-11T06:51:35.228 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2729999999999998e-06 2024-09-11T06:51:35.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.228 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:35.228 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.786891+0000", 2024-09-11T06:51:35.228 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7153e-05 2024-09-11T06:51:35.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.228 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:35.228 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.786925+0000", 2024-09-11T06:51:35.229 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4227999999999998e-05 2024-09-11T06:51:35.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.229 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.229 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.229 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.786941+0000", 2024-09-11T06:51:35.229 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6413e-05 2024-09-11T06:51:35.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.229 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.229 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.229 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.787108+0000", 2024-09-11T06:51:35.229 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016641 2024-09-11T06:51:35.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.229 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.229 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:35.230 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.787132+0000", 2024-09-11T06:51:35.230 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3946000000000001e-05 2024-09-11T06:51:35.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.230 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.230 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.230 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.790458+0000", 2024-09-11T06:51:35.230 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033267790000000002 2024-09-11T06:51:35.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.230 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.230 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:35.230 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.790589+0000", 2024-09-11T06:51:35.230 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013019400000000001 2024-09-11T06:51:35.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.230 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.231 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:35.231 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.790751+0000", 2024-09-11T06:51:35.231 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016199699999999999 2024-09-11T06:51:35.231 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.231 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:35.231 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.231 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.231 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.231 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5536.0:99 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.tag_create in=17b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-11T06:51:35.231 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:25.830334+0000", 2024-09-11T06:51:35.232 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 9.3105056840000007, 2024-09-11T06:51:35.232 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0028281019999999999, 2024-09-11T06:51:35.232 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:35.232 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:35.232 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.232 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:35.232 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5536", 2024-09-11T06:51:35.232 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/612379138", 2024-09-11T06:51:35.232 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 99 2024-09-11T06:51:35.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.232 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:35.232 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.232 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:35.233 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.830334+0000", 2024-09-11T06:51:35.233 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.233 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.233 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:35.233 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.830334+0000", 2024-09-11T06:51:35.233 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.233 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.233 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:35.233 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.830337+0000", 2024-09-11T06:51:35.233 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1100000000000001e-06 2024-09-11T06:51:35.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.234 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.234 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:35.234 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.830347+0000", 2024-09-11T06:51:35.234 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.075e-05 2024-09-11T06:51:35.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.234 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.234 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:35.234 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.830380+0000", 2024-09-11T06:51:35.234 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2191999999999999e-05 2024-09-11T06:51:35.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.234 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.235 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.235 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.830400+0000", 2024-09-11T06:51:35.235 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0886e-05 2024-09-11T06:51:35.235 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.235 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.235 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.235 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.830537+0000", 2024-09-11T06:51:35.235 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000136964 2024-09-11T06:51:35.235 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.235 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.235 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:35.235 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.830762+0000", 2024-09-11T06:51:35.236 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022447100000000001 2024-09-11T06:51:35.236 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.236 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.236 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:35.236 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.831663+0000", 2024-09-11T06:51:35.236 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00090116800000000004 2024-09-11T06:51:35.236 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.236 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.236 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:35.236 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.832966+0000", 2024-09-11T06:51:35.237 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001303305 2024-09-11T06:51:35.237 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.237 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.237 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:35.237 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.832992+0000", 2024-09-11T06:51:35.237 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6126e-05 2024-09-11T06:51:35.237 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.237 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.237 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:35.237 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.833096+0000", 2024-09-11T06:51:35.237 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103082 2024-09-11T06:51:35.237 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.238 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.238 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:35.238 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.833163+0000", 2024-09-11T06:51:35.238 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7047999999999999e-05 2024-09-11T06:51:35.238 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.238 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:35.238 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.238 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.238 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.238 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5660.0:32 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+full_try+supports_pool_eio e107)", 2024-09-11T06:51:35.238 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:30.113546+0000", 2024-09-11T06:51:35.239 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 5.027294124, 2024-09-11T06:51:35.239 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0057805510000000001, 2024-09-11T06:51:35.239 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:35.239 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:35.239 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:35.239 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:35.239 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5660", 2024-09-11T06:51:35.239 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3214439167", 2024-09-11T06:51:35.239 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 32 2024-09-11T06:51:35.239 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.239 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:35.239 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.240 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:35.240 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.113546+0000", 2024-09-11T06:51:35.240 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.240 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.240 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.240 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:35.240 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.113546+0000", 2024-09-11T06:51:35.240 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.240 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.240 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.240 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:35.240 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.113551+0000", 2024-09-11T06:51:35.241 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.8230000000000002e-06 2024-09-11T06:51:35.241 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.241 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.241 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:35.241 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.113569+0000", 2024-09-11T06:51:35.241 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8436000000000001e-05 2024-09-11T06:51:35.241 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.241 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.241 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:35.241 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.113621+0000", 2024-09-11T06:51:35.241 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1452000000000001e-05 2024-09-11T06:51:35.241 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.241 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.242 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.242 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.113648+0000", 2024-09-11T06:51:35.242 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7379e-05 2024-09-11T06:51:35.242 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.242 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.242 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.242 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.113844+0000", 2024-09-11T06:51:35.242 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019605600000000001 2024-09-11T06:51:35.242 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.242 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.242 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:35.243 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.113862+0000", 2024-09-11T06:51:35.243 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7917000000000001e-05 2024-09-11T06:51:35.243 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.243 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.243 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.243 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.118846+0000", 2024-09-11T06:51:35.243 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0049840209999999999 2024-09-11T06:51:35.243 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.243 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.243 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:35.243 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.119030+0000", 2024-09-11T06:51:35.243 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000184067 2024-09-11T06:51:35.244 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.244 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.244 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:35.244 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.119327+0000", 2024-09-11T06:51:35.244 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00029639999999999999 2024-09-11T06:51:35.244 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.244 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:35.244 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.244 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.244 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.244 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5660.0:53 2.2 2:4510a54f:::rbd_object_map.1314f0a2bfa2.0000000000000007:head [call rbd.object_map_snap_remove in=25b] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e108)", 2024-09-11T06:51:35.245 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:31.138066+0000", 2024-09-11T06:51:35.245 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.0027741959999998, 2024-09-11T06:51:35.245 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003789766, 2024-09-11T06:51:35.245 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:35.245 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:35.245 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:35.245 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:35.245 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5660", 2024-09-11T06:51:35.245 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3214439167", 2024-09-11T06:51:35.245 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 53 2024-09-11T06:51:35.245 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.245 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:35.246 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.246 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:35.246 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:31.138066+0000", 2024-09-11T06:51:35.246 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.246 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.246 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.246 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:35.246 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:31.138066+0000", 2024-09-11T06:51:35.246 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.246 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.246 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.246 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:35.247 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:31.138068+0000", 2024-09-11T06:51:35.247 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9290000000000001e-06 2024-09-11T06:51:35.247 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.247 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.247 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:35.247 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:31.138075+0000", 2024-09-11T06:51:35.247 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1180000000000003e-06 2024-09-11T06:51:35.247 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.247 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.247 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:35.247 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:31.138093+0000", 2024-09-11T06:51:35.247 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7915e-05 2024-09-11T06:51:35.248 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.248 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.248 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.248 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:31.138102+0000", 2024-09-11T06:51:35.248 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.7539999999999995e-06 2024-09-11T06:51:35.248 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.248 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.248 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.248 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:31.138193+0000", 2024-09-11T06:51:35.248 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.1221e-05 2024-09-11T06:51:35.248 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.248 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.249 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:35.249 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:31.138312+0000", 2024-09-11T06:51:35.249 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011957599999999999 2024-09-11T06:51:35.249 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.249 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.249 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:35.249 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:31.141856+0000", 2024-09-11T06:51:35.249 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035432530000000001 2024-09-11T06:51:35.249 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.249 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:35.249 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.249 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.250 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.250 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5723.0:3 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+full_try+supports_pool_eio e111)", 2024-09-11T06:51:35.250 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:34.208333+0000", 2024-09-11T06:51:35.250 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.93250718200000005, 2024-09-11T06:51:35.250 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0029589500000000001, 2024-09-11T06:51:35.250 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:35.250 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:35.250 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:35.250 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:35.250 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5723", 2024-09-11T06:51:35.250 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3725019196", 2024-09-11T06:51:35.250 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 3 2024-09-11T06:51:35.251 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.251 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:35.251 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.251 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:35.251 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.208333+0000", 2024-09-11T06:51:35.251 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.251 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.251 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.251 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:35.251 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.208333+0000", 2024-09-11T06:51:35.251 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.251 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.251 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.251 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:35.252 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.208335+0000", 2024-09-11T06:51:35.252 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.827e-06 2024-09-11T06:51:35.252 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.252 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.252 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:35.252 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.208341+0000", 2024-09-11T06:51:35.252 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0419999999999999e-06 2024-09-11T06:51:35.252 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.252 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.252 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:35.252 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.208353+0000", 2024-09-11T06:51:35.252 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2058999999999999e-05 2024-09-11T06:51:35.252 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.252 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.253 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.253 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.208361+0000", 2024-09-11T06:51:35.253 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2840000000000006e-06 2024-09-11T06:51:35.253 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.253 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.253 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.253 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.208425+0000", 2024-09-11T06:51:35.253 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.4105999999999999e-05 2024-09-11T06:51:35.253 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.253 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.253 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:35.253 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.208431+0000", 2024-09-11T06:51:35.254 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.857e-06 2024-09-11T06:51:35.254 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.254 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.254 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.254 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.211055+0000", 2024-09-11T06:51:35.254 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0026241139999999999 2024-09-11T06:51:35.254 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.254 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.254 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:35.254 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.211192+0000", 2024-09-11T06:51:35.254 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013648299999999999 2024-09-11T06:51:35.254 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.254 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.254 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:35.255 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.211292+0000", 2024-09-11T06:51:35.255 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000100178 2024-09-11T06:51:35.255 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.255 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:35.255 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.255 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.255 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.255 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5723.0:81 2.2 2:56207b30:ns::rbd_data.11adc335555f.0000000000000000:head [delete] snapc 4=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e111)", 2024-09-11T06:51:35.255 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:34.287933+0000", 2024-09-11T06:51:35.255 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.85290712199999996, 2024-09-11T06:51:35.255 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010316603000000001, 2024-09-11T06:51:35.255 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:35.255 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:35.255 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.256 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:35.256 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5723", 2024-09-11T06:51:35.256 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3725019196", 2024-09-11T06:51:35.256 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 81 2024-09-11T06:51:35.256 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.256 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:35.256 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.256 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:35.256 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.287933+0000", 2024-09-11T06:51:35.256 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.256 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.256 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.256 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:35.256 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.287933+0000", 2024-09-11T06:51:35.256 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.257 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.257 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.257 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:35.257 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.287935+0000", 2024-09-11T06:51:35.257 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.823e-06 2024-09-11T06:51:35.257 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.257 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.257 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:35.257 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.287942+0000", 2024-09-11T06:51:35.257 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1149999999999996e-06 2024-09-11T06:51:35.257 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.257 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.257 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:35.257 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.287964+0000", 2024-09-11T06:51:35.258 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1747999999999999e-05 2024-09-11T06:51:35.258 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.258 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.258 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.258 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.287974+0000", 2024-09-11T06:51:35.258 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0387000000000001e-05 2024-09-11T06:51:35.258 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.258 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.258 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.258 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.288058+0000", 2024-09-11T06:51:35.258 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4376000000000007e-05 2024-09-11T06:51:35.259 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.259 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.259 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:35.259 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.288164+0000", 2024-09-11T06:51:35.259 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00010584100000000001 2024-09-11T06:51:35.259 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.259 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.259 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:35.259 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.288301+0000", 2024-09-11T06:51:35.259 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000137107 2024-09-11T06:51:35.259 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.259 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.259 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:35.259 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.290499+0000", 2024-09-11T06:51:35.260 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021976779999999998 2024-09-11T06:51:35.260 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.260 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.260 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:35.260 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298176+0000", 2024-09-11T06:51:35.260 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0076773259999999999 2024-09-11T06:51:35.260 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.260 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.260 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:35.260 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298221+0000", 2024-09-11T06:51:35.260 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4152e-05 2024-09-11T06:51:35.260 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.261 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.261 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:35.261 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298250+0000", 2024-09-11T06:51:35.261 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9050000000000001e-05 2024-09-11T06:51:35.261 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.261 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:35.261 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.261 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.261 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.261 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5723.0:91 2.2 2:45bb0ce9:ns::rbd_data.11adc335555f.000000000000000a:head [delete] snapc 4=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e111)", 2024-09-11T06:51:35.261 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:34.291968+0000", 2024-09-11T06:51:35.261 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.84887185200000004, 2024-09-11T06:51:35.261 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0066095399999999997, 2024-09-11T06:51:35.262 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:35.262 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:35.262 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.262 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:35.262 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5723", 2024-09-11T06:51:35.262 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3725019196", 2024-09-11T06:51:35.262 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 91 2024-09-11T06:51:35.262 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.262 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:35.262 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.262 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:35.262 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.291968+0000", 2024-09-11T06:51:35.262 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.263 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.263 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.263 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:35.263 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.291968+0000", 2024-09-11T06:51:35.263 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.263 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.263 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.263 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:35.263 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.291970+0000", 2024-09-11T06:51:35.263 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.668e-06 2024-09-11T06:51:35.263 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.263 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.264 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:35.264 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.291977+0000", 2024-09-11T06:51:35.264 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7270000000000001e-06 2024-09-11T06:51:35.264 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.264 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.264 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:35.264 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.291991+0000", 2024-09-11T06:51:35.264 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4521999999999999e-05 2024-09-11T06:51:35.264 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.264 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.264 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.264 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.292000+0000", 2024-09-11T06:51:35.264 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6640000000000004e-06 2024-09-11T06:51:35.265 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.265 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.265 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.265 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.292080+0000", 2024-09-11T06:51:35.265 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.0092999999999994e-05 2024-09-11T06:51:35.265 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.265 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.265 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:35.265 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.292162+0000", 2024-09-11T06:51:35.265 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2210000000000001e-05 2024-09-11T06:51:35.265 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.265 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.266 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:35.266 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.292278+0000", 2024-09-11T06:51:35.266 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000115555 2024-09-11T06:51:35.266 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.266 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.266 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:35.266 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.293847+0000", 2024-09-11T06:51:35.266 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0015695010000000001 2024-09-11T06:51:35.266 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.266 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.266 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:35.266 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298503+0000", 2024-09-11T06:51:35.267 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0046562440000000004 2024-09-11T06:51:35.267 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.267 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.267 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:35.267 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298546+0000", 2024-09-11T06:51:35.267 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.2749e-05 2024-09-11T06:51:35.267 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.267 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.267 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:35.267 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298578+0000", 2024-09-11T06:51:35.267 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.1606999999999997e-05 2024-09-11T06:51:35.267 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.268 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:35.268 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.268 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.268 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.268 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5723.0:95 2.3 2:d0a3a233:ns::rbd_data.11adc335555f.000000000000000e:head [delete] snapc 4=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e111)", 2024-09-11T06:51:35.268 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:34.298831+0000", 2024-09-11T06:51:35.268 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.842009283, 2024-09-11T06:51:35.268 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003269582, 2024-09-11T06:51:35.268 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:35.268 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:35.268 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.268 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:35.268 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5723", 2024-09-11T06:51:35.269 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3725019196", 2024-09-11T06:51:35.269 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 95 2024-09-11T06:51:35.269 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.269 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:35.269 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.269 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:35.269 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298831+0000", 2024-09-11T06:51:35.269 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.269 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.269 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.269 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:35.270 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298831+0000", 2024-09-11T06:51:35.270 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:35.270 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.270 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.270 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:35.270 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298832+0000", 2024-09-11T06:51:35.270 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.145e-06 2024-09-11T06:51:35.270 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.270 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.270 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:35.270 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298837+0000", 2024-09-11T06:51:35.271 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0839999999999999e-06 2024-09-11T06:51:35.271 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.271 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.271 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:35.271 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298851+0000", 2024-09-11T06:51:35.271 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4166e-05 2024-09-11T06:51:35.271 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.271 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.271 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.271 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298861+0000", 2024-09-11T06:51:35.271 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0169e-05 2024-09-11T06:51:35.271 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.272 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.272 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:35.272 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298917+0000", 2024-09-11T06:51:35.272 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5195999999999998e-05 2024-09-11T06:51:35.272 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.272 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.272 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:35.272 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298969+0000", 2024-09-11T06:51:35.272 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2550999999999999e-05 2024-09-11T06:51:35.272 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.272 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.272 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:35.273 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.299036+0000", 2024-09-11T06:51:35.273 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7070000000000007e-05 2024-09-11T06:51:35.273 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.273 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.273 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:35.273 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.300825+0000", 2024-09-11T06:51:35.273 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001788913 2024-09-11T06:51:35.273 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.273 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.273 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:35.273 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.301997+0000", 2024-09-11T06:51:35.274 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011715180000000001 2024-09-11T06:51:35.274 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.274 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.274 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:35.274 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.302042+0000", 2024-09-11T06:51:35.274 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5306000000000003e-05 2024-09-11T06:51:35.274 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:35.274 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:35.274 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:35.275 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.302100+0000", 2024-09-11T06:51:35.275 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8464000000000001e-05 2024-09-11T06:51:35.275 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.275 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:35.275 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.275 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:35.275 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:35.275 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:35.276 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T06:51:35.278 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:35.274+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:35.278 INFO:tasks.workunit.client.0.smithi176.stderr:pool 'new_default_pool21607' does not exist 2024-09-11T06:51:35.279 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-11T06:51:35.279 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-11T06:51:35.332 INFO:tasks.workunit:Stopping ['rbd/rbd-nbd.sh'] on client.0... 2024-09-11T06:51:35.332 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-11T06:51:35.377 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:35.374+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:35.408 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:35.408 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:35.408 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:51:35.408 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:35.425 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T06:51:35.479 INFO:tasks.ceph.osd.5.smithi089.stderr:2024-09-11T06:51:35.477+0000 7f4b64054640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25220) UID: 0 2024-09-11T06:51:35.574 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:51:35.574 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:51:35.579 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:35.574+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:35.585 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:35.585 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:35.585 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:51:35.585 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:51:35.585 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:35.591 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":183,"stamp":"2024-09-11T06:51:35.144390+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":67699305,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":6925,"num_read_kb":120307,"num_write":1098,"num_write_kb":275366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":262474437,"num_keys_recovered":41,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1183,"ondisk_log_size":1183,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":294604,"kb_used_data":138648,"kb_used_omap":7,"kb_used_meta":155832,"kb_avail":471564596,"statfs":{"total":483183820800,"available":482882146304,"internally_reserved":0,"allocated":141975552,"data_stored":137935736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7971,"internal_metadata":159572189},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-83956026,"num_objects":-50,"num_object_clones":-20,"num_object_copies":-100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-50,"num_whiteouts":-12,"num_read":485,"num_read_kb":403,"num_write":65,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-9,"num_bytes_recovered":-124,"num_keys_recovered":-10,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.027067"},"pg_stats":[{"pgid":"1.0","version":"14'117","reported_seq":358,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-11T06:51:33.131904+0000","last_change":"2024-09-11T06:50:37.941212+0000","last_active":"2024-09-11T06:51:33.131904+0000","last_peered":"2024-09-11T06:51:33.131904+0000","last_clean":"2024-09-11T06:51:33.131904+0000","last_became_active":"2024-09-11T06:50:37.940448+0000","last_became_peered":"2024-09-11T06:50:37.940448+0000","last_unstale":"2024-09-11T06:51:33.131904+0000","last_undegraded":"2024-09-11T06:51:33.131904+0000","last_fullsized":"2024-09-11T06:51:33.131904+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_clean_scrub_stamp":"2024-09-11T06:49:05.088839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:48:59.833858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"110'127","reported_seq":395,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-11T06:51:33.337661+0000","last_change":"2024-09-11T06:51:33.212360+0000","last_active":"2024-09-11T06:51:33.337661+0000","last_peered":"2024-09-11T06:51:33.337661+0000","last_clean":"2024-09-11T06:51:33.337661+0000","last_became_active":"2024-09-11T06:50:47.306211+0000","last_became_peered":"2024-09-11T06:50:47.306211+0000","last_unstale":"2024-09-11T06:51:33.337661+0000","last_undegraded":"2024-09-11T06:51:33.337661+0000","last_fullsized":"2024-09-11T06:51:33.337661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:02.000399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091565099999999996,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":383,"num_read_kb":8504,"num_write":97,"num_write_kb":28714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":25165862,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"3"}]},{"pgid":"2.0","version":"109'81","reported_seq":317,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-11T06:51:33.132712+0000","last_change":"2024-09-11T06:51:33.133348+0000","last_active":"2024-09-11T06:51:33.132712+0000","last_peered":"2024-09-11T06:51:33.132712+0000","last_clean":"2024-09-11T06:51:33.132712+0000","last_became_active":"2024-09-11T06:51:18.986870+0000","last_became_peered":"2024-09-11T06:51:18.986870+0000","last_unstale":"2024-09-11T06:51:33.132712+0000","last_undegraded":"2024-09-11T06:51:33.132712+0000","last_fullsized":"2024-09-11T06:51:33.132712+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:40:04.828405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001177885,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":23060,"num_write":59,"num_write_kb":45056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":62914560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"3"}]},{"pgid":"2.1","version":"110'250","reported_seq":2211,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-11T06:51:34.041659+0000","last_change":"2024-09-11T06:51:33.130678+0000","last_active":"2024-09-11T06:51:34.041659+0000","last_peered":"2024-09-11T06:51:34.041659+0000","last_clean":"2024-09-11T06:51:34.041659+0000","last_became_active":"2024-09-11T06:51:21.988695+0000","last_became_peered":"2024-09-11T06:51:21.988695+0000","last_unstale":"2024-09-11T06:51:34.041659+0000","last_undegraded":"2024-09-11T06:51:34.041659+0000","last_fullsized":"2024-09-11T06:51:34.041659+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:42:19.055371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074871099999999999,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4173,"num_read_kb":27885,"num_write":140,"num_write_kb":41060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":25231476,"num_keys_recovered":13,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"3"}]},{"pgid":"2.2","version":"109'10","reported_seq":208,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-11T06:51:33.966747+0000","last_change":"2024-09-11T06:51:33.212649+0000","last_active":"2024-09-11T06:51:33.966747+0000","last_peered":"2024-09-11T06:51:33.966747+0000","last_clean":"2024-09-11T06:51:33.966747+0000","last_became_active":"2024-09-11T06:51:18.987217+0000","last_became_peered":"2024-09-11T06:51:18.987217+0000","last_unstale":"2024-09-11T06:51:33.966747+0000","last_undegraded":"2024-09-11T06:51:33.966747+0000","last_fullsized":"2024-09-11T06:51:33.966747+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:13.143403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011855349999999999,"stat_sum":{"num_bytes":8388627,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":89,"num_read_kb":86,"num_write":8,"num_write_kb":8196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":16777427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"3"}]},{"pgid":"2.4","version":"110'198","reported_seq":500,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-11T06:51:33.319501+0000","last_change":"2024-09-11T06:51:33.141895+0000","last_active":"2024-09-11T06:51:33.319501+0000","last_peered":"2024-09-11T06:51:33.319501+0000","last_clean":"2024-09-11T06:51:33.319501+0000","last_became_active":"2024-09-11T06:51:21.991171+0000","last_became_peered":"2024-09-11T06:51:21.991171+0000","last_unstale":"2024-09-11T06:51:33.319501+0000","last_undegraded":"2024-09-11T06:51:33.319501+0000","last_fullsized":"2024-09-11T06:51:33.319501+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:31:01.579481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027543540000000001,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":854,"num_read_kb":8870,"num_write":298,"num_write_kb":20705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":12582950,"num_keys_recovered":3,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"3"}]},{"pgid":"2.5","version":"110'164","reported_seq":633,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-11T06:51:34.016493+0000","last_change":"2024-09-11T06:51:33.141183+0000","last_active":"2024-09-11T06:51:34.016493+0000","last_peered":"2024-09-11T06:51:34.016493+0000","last_clean":"2024-09-11T06:51:34.016493+0000","last_became_active":"2024-09-11T06:50:35.236333+0000","last_became_peered":"2024-09-11T06:50:35.236333+0000","last_unstale":"2024-09-11T06:51:34.016493+0000","last_undegraded":"2024-09-11T06:51:34.016493+0000","last_fullsized":"2024-09-11T06:51:34.016493+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:52:55.118390+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0089129259999999998,"stat_sum":{"num_bytes":12582928,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":619,"num_read_kb":21604,"num_write":145,"num_write_kb":45181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":41943104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"3"}]},{"pgid":"2.6","version":"110'54","reported_seq":298,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-11T06:51:34.036620+0000","last_change":"2024-09-11T06:51:33.133667+0000","last_active":"2024-09-11T06:51:34.036620+0000","last_peered":"2024-09-11T06:51:34.036620+0000","last_clean":"2024-09-11T06:51:34.036620+0000","last_became_active":"2024-09-11T06:51:18.984613+0000","last_became_peered":"2024-09-11T06:51:18.984613+0000","last_unstale":"2024-09-11T06:51:34.036620+0000","last_undegraded":"2024-09-11T06:51:34.036620+0000","last_fullsized":"2024-09-11T06:51:34.036620+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:56:03.861829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001463385,"stat_sum":{"num_bytes":4194342,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":8293,"num_write":62,"num_write_kb":16421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":25165938,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"3"}]},{"pgid":"2.7","version":"110'182","reported_seq":364,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-11T06:51:34.039844+0000","last_change":"2024-09-11T06:51:33.140084+0000","last_active":"2024-09-11T06:51:34.039844+0000","last_peered":"2024-09-11T06:51:34.039844+0000","last_clean":"2024-09-11T06:51:34.039844+0000","last_became_active":"2024-09-11T06:50:35.238011+0000","last_became_peered":"2024-09-11T06:50:35.238011+0000","last_unstale":"2024-09-11T06:51:34.039844+0000","last_undegraded":"2024-09-11T06:51:34.039844+0000","last_fullsized":"2024-09-11T06:51:34.039844+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:26:38.712163+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0076355750000000003,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":414,"num_read_kb":21905,"num_write":130,"num_write_kb":67617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":50331648,"num_keys_recovered":16,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"3"}]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":67108937,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":6809,"num_read_kb":120207,"num_write":939,"num_write_kb":272950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":260112965,"num_keys_recovered":41,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":193044480,"data_stored":193003898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1066,"ondisk_log_size":1066,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640288,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44764,"kb_used_data":13572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94327076,"statfs":{"total":96636764160,"available":96590925824,"internally_reserved":0,"allocated":13897728,"data_stored":13090348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640289,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48856,"kb_used_data":17664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94322984,"statfs":{"total":96636764160,"available":96586735616,"internally_reserved":0,"allocated":18087936,"data_stored":17284671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640289,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61732,"kb_used_data":30540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94310108,"statfs":{"total":96636764160,"available":96573550592,"internally_reserved":0,"allocated":31272960,"data_stored":30457967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640290,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52948,"kb_used_data":21760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94318892,"statfs":{"total":96636764160,"available":96582545408,"internally_reserved":0,"allocated":22282240,"data_stored":21478975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":47244640290,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640291,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86304,"kb_used_data":55112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94285536,"statfs":{"total":96636764160,"available":96548388864,"internally_reserved":0,"allocated":56434688,"data_stored":55623775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54534144,"data_stored":54525987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":58802176,"data_stored":58786024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20971520,"data_stored":20971520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":29368320,"data_stored":29360182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16781312,"data_stored":16777254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12587008,"data_stored":12582931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:51:35.593 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:51:35.599 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T06:51:35.685 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:35.682+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:35.688 DEBUG:teuthology.parallel:result is None 2024-09-11T06:51:35.688 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T06:51:35.700 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T06:51:35.701 DEBUG:teuthology.orchestra.run.smithi176:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-09-11T06:51:35.754 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:35.754 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:51:35.754 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:51:35.755 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:51:35.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.755 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:56 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'34, mlcod=13'29)", 2024-09-11T06:51:35.755 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:51:35.755 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 149.53990075900001, 2024-09-11T06:51:35.755 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0060310629999999997, 2024-09-11T06:51:35.755 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:35.755 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:35.755 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.755 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:35.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.755 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:35.755 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:51:35.755 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.756 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.756 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.756 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:35.756 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212813+0000", 2024-09-11T06:51:35.756 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.756 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.756 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.756 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:35.756 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212814+0000", 2024-09-11T06:51:35.756 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0869999999999999e-06 2024-09-11T06:51:35.756 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.756 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.756 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:35.757 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212822+0000", 2024-09-11T06:51:35.757 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0870000000000007e-06 2024-09-11T06:51:35.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.757 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.757 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:35.757 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212840+0000", 2024-09-11T06:51:35.757 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:51:35.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.757 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.757 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.757 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.212854+0000", 2024-09-11T06:51:35.757 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4003e-05 2024-09-11T06:51:35.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.757 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.758 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:35.758 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213026+0000", 2024-09-11T06:51:35.758 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017207499999999999 2024-09-11T06:51:35.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.758 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.758 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:35.758 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.213063+0000", 2024-09-11T06:51:35.758 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7187000000000002e-05 2024-09-11T06:51:35.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.759 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:35.759 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218762+0000", 2024-09-11T06:51:35.759 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0056993290000000004 2024-09-11T06:51:35.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.759 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:35.760 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.218844+0000", 2024-09-11T06:51:35.760 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1701999999999998e-05 2024-09-11T06:51:35.760 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.760 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:35.760 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.761 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.761 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:120 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'82, mlcod=13'78)", 2024-09-11T06:51:35.761 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:51:35.761 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 149.286304355, 2024-09-11T06:51:35.761 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0064612549999999999, 2024-09-11T06:51:35.761 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:35.762 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:35.762 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.762 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:35.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.762 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:35.762 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:51:35.762 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.762 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:35.762 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466409+0000", 2024-09-11T06:51:35.763 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.763 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.763 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:35.763 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466410+0000", 2024-09-11T06:51:35.763 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0389999999999999e-06 2024-09-11T06:51:35.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.763 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.764 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:35.764 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466496+0000", 2024-09-11T06:51:35.764 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5519000000000006e-05 2024-09-11T06:51:35.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.764 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:35.764 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466510+0000", 2024-09-11T06:51:35.764 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3842999999999999e-05 2024-09-11T06:51:35.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.765 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.765 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466517+0000", 2024-09-11T06:51:35.765 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5229999999999998e-06 2024-09-11T06:51:35.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.765 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.765 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:35.765 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466595+0000", 2024-09-11T06:51:35.765 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8019e-05 2024-09-11T06:51:35.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.765 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.765 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:35.766 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.466612+0000", 2024-09-11T06:51:35.766 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7013000000000001e-05 2024-09-11T06:51:35.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.766 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:35.766 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472787+0000", 2024-09-11T06:51:35.766 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0061750269999999996 2024-09-11T06:51:35.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.767 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:35.767 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.472870+0000", 2024-09-11T06:51:35.767 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3271999999999998e-05 2024-09-11T06:51:35.767 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.767 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:35.767 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.767 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:121 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'83, mlcod=13'78)", 2024-09-11T06:51:35.767 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:51:35.768 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 149.285079559, 2024-09-11T06:51:35.768 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055613069999999997, 2024-09-11T06:51:35.768 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:35.768 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:35.768 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.768 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:35.768 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.768 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:35.768 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:51:35.769 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.769 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.769 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:35.769 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467634+0000", 2024-09-11T06:51:35.769 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.770 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.770 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:35.770 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467635+0000", 2024-09-11T06:51:35.770 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6899999999999996e-07 2024-09-11T06:51:35.770 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.770 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.770 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:35.770 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467727+0000", 2024-09-11T06:51:35.770 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1800999999999997e-05 2024-09-11T06:51:35.770 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.771 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:35.771 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467740+0000", 2024-09-11T06:51:35.771 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3747999999999999e-05 2024-09-11T06:51:35.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.771 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.771 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467753+0000", 2024-09-11T06:51:35.771 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2807e-05 2024-09-11T06:51:35.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.772 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:35.772 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467812+0000", 2024-09-11T06:51:35.772 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8381999999999998e-05 2024-09-11T06:51:35.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.772 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:35.772 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.467828+0000", 2024-09-11T06:51:35.772 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6728e-05 2024-09-11T06:51:35.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.773 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:35.773 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473112+0000", 2024-09-11T06:51:35.773 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052833619999999998 2024-09-11T06:51:35.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.773 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:35.773 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.473195+0000", 2024-09-11T06:51:35.773 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3609999999999994e-05 2024-09-11T06:51:35.774 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.774 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:35.774 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.774 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:131 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'89, mlcod=13'85)", 2024-09-11T06:51:35.774 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:51:35.774 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 148.933673844, 2024-09-11T06:51:35.774 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052492260000000001, 2024-09-11T06:51:35.775 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:35.775 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:35.775 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.775 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:35.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.775 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:35.775 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:51:35.775 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.775 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:35.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819040+0000", 2024-09-11T06:51:35.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:35.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819041+0000", 2024-09-11T06:51:35.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4360000000000001e-06 2024-09-11T06:51:35.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:35.777 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819100+0000", 2024-09-11T06:51:35.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9225999999999998e-05 2024-09-11T06:51:35.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.777 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:35.777 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819128+0000", 2024-09-11T06:51:35.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.779e-05 2024-09-11T06:51:35.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.777 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.778 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819142+0000", 2024-09-11T06:51:35.778 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3573e-05 2024-09-11T06:51:35.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.778 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:35.778 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819263+0000", 2024-09-11T06:51:35.778 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000121086 2024-09-11T06:51:35.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.778 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:35.779 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.819292+0000", 2024-09-11T06:51:35.779 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9363e-05 2024-09-11T06:51:35.779 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.779 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.779 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:35.779 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824185+0000", 2024-09-11T06:51:35.779 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0048924620000000002 2024-09-11T06:51:35.779 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.779 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.779 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:35.779 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.824289+0000", 2024-09-11T06:51:35.780 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010429000000000001 2024-09-11T06:51:35.780 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.780 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:35.780 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.780 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.780 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.780 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:140 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'98, mlcod=13'93)", 2024-09-11T06:51:35.780 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:51:35.780 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 148.92500678799999, 2024-09-11T06:51:35.780 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0057995929999999996, 2024-09-11T06:51:35.780 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:35.781 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:35.781 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.781 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:35.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:35.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:51:35.781 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:35.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827707+0000", 2024-09-11T06:51:35.782 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.782 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:35.782 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827708+0000", 2024-09-11T06:51:35.782 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.547e-06 2024-09-11T06:51:35.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.783 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:35.783 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827765+0000", 2024-09-11T06:51:35.783 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6266e-05 2024-09-11T06:51:35.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.783 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:35.783 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827791+0000", 2024-09-11T06:51:35.783 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6098999999999999e-05 2024-09-11T06:51:35.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.783 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.784 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827806+0000", 2024-09-11T06:51:35.784 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5537e-05 2024-09-11T06:51:35.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.784 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.784 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:35.784 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827928+0000", 2024-09-11T06:51:35.784 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000122012 2024-09-11T06:51:35.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.784 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.784 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:35.785 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.827961+0000", 2024-09-11T06:51:35.785 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2663999999999998e-05 2024-09-11T06:51:35.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.785 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.785 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:35.785 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833409+0000", 2024-09-11T06:51:35.785 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054479350000000001 2024-09-11T06:51:35.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.785 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.785 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:35.785 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.833506+0000", 2024-09-11T06:51:35.785 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7533000000000003e-05 2024-09-11T06:51:35.785 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.786 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:35.786 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.786 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:141 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'99, mlcod=13'93)", 2024-09-11T06:51:35.786 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:51:35.786 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 148.92411340500001, 2024-09-11T06:51:35.786 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0055394709999999998, 2024-09-11T06:51:35.786 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:35.786 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:35.786 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.786 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:35.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.787 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:35.787 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:51:35.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.787 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:35.787 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828600+0000", 2024-09-11T06:51:35.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.787 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:35.787 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828602+0000", 2024-09-11T06:51:35.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.46e-06 2024-09-11T06:51:35.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.788 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:35.788 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828618+0000", 2024-09-11T06:51:35.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6592999999999999e-05 2024-09-11T06:51:35.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.788 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:35.788 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828646+0000", 2024-09-11T06:51:35.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7552999999999999e-05 2024-09-11T06:51:35.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.789 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.789 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.828663+0000", 2024-09-11T06:51:35.789 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.719e-05 2024-09-11T06:51:35.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.789 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.789 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:35.789 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829023+0000", 2024-09-11T06:51:35.789 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036054799999999998 2024-09-11T06:51:35.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.789 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.789 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:35.790 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.829062+0000", 2024-09-11T06:51:35.790 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8865999999999997e-05 2024-09-11T06:51:35.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.790 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.790 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:35.790 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834078+0000", 2024-09-11T06:51:35.790 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050157480000000004 2024-09-11T06:51:35.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.790 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.790 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:35.790 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.834140+0000", 2024-09-11T06:51:35.790 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1513000000000005e-05 2024-09-11T06:51:35.790 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.791 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:35.791 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.791 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.791 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4165.0:145 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'103, mlcod=13'98)", 2024-09-11T06:51:35.791 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:51:35.791 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 148.91416090800001, 2024-09-11T06:51:35.791 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054455409999999999, 2024-09-11T06:51:35.791 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:35.791 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:35.791 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.791 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:35.791 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.791 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:35.792 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:51:35.792 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.792 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:35.792 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838553+0000", 2024-09-11T06:51:35.792 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.792 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:35.793 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838554+0000", 2024-09-11T06:51:35.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3430000000000001e-06 2024-09-11T06:51:35.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.793 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.793 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:35.793 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838640+0000", 2024-09-11T06:51:35.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5569999999999996e-05 2024-09-11T06:51:35.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.794 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:35.794 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838660+0000", 2024-09-11T06:51:35.794 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0576000000000001e-05 2024-09-11T06:51:35.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.794 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.794 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838674+0000", 2024-09-11T06:51:35.794 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3529999999999999e-05 2024-09-11T06:51:35.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.795 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:35.795 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838838+0000", 2024-09-11T06:51:35.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016468799999999999 2024-09-11T06:51:35.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.795 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:35.795 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.838885+0000", 2024-09-11T06:51:35.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6884e-05 2024-09-11T06:51:35.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.795 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:35.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843901+0000", 2024-09-11T06:51:35.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0050162310000000003 2024-09-11T06:51:35.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:35.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:49:06.843998+0000", 2024-09-11T06:51:35.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.6718999999999993e-05 2024-09-11T06:51:35.796 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.797 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:35.797 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.797 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4558.0:1 3.3 e56/55 3:c4c92e5a:::rbd_trash:head v 56'1, mlcod=0'0)", 2024-09-11T06:51:35.797 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:51:35.797 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 78.661357846000001, 2024-09-11T06:51:35.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0053791810000000002, 2024-09-11T06:51:35.797 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:35.797 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:35.797 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.798 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:35.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:35.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:51:35.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:35.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091356+0000", 2024-09-11T06:51:35.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.799 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:35.799 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091357+0000", 2024-09-11T06:51:35.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.54e-07 2024-09-11T06:51:35.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.799 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:35.799 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091376+0000", 2024-09-11T06:51:35.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9769e-05 2024-09-11T06:51:35.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.800 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.800 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:35.800 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091396+0000", 2024-09-11T06:51:35.800 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0152999999999998e-05 2024-09-11T06:51:35.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.800 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.800 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.800 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091416+0000", 2024-09-11T06:51:35.800 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0006000000000001e-05 2024-09-11T06:51:35.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.801 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:35.801 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091506+0000", 2024-09-11T06:51:35.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9325000000000003e-05 2024-09-11T06:51:35.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.801 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:35.801 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.091523+0000", 2024-09-11T06:51:35.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.77e-05 2024-09-11T06:51:35.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:35.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.096685+0000", 2024-09-11T06:51:35.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0051618569999999997 2024-09-11T06:51:35.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:35.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:17.096735+0000", 2024-09-11T06:51:35.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9517e-05 2024-09-11T06:51:35.802 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.802 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:35.802 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.803 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.5544.0:67 2.5 e82/69 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head v 82'77, mlcod=81'74)", 2024-09-11T06:51:35.803 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.595302+0000", 2024-09-11T06:51:35.803 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 47.157411430000003, 2024-09-11T06:51:35.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0054511999999999998, 2024-09-11T06:51:35.803 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:35.803 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:35.803 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.803 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:35.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.804 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:35.804 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595302+0000", 2024-09-11T06:51:35.804 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.804 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:35.804 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595302+0000", 2024-09-11T06:51:35.804 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.804 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:35.805 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595303+0000", 2024-09-11T06:51:35.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2699999999999999e-07 2024-09-11T06:51:35.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.805 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:35.805 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595522+0000", 2024-09-11T06:51:35.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000218946 2024-09-11T06:51:35.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.805 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:35.805 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595544+0000", 2024-09-11T06:51:35.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2558999999999999e-05 2024-09-11T06:51:35.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595557+0000", 2024-09-11T06:51:35.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2653000000000001e-05 2024-09-11T06:51:35.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:35.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595662+0000", 2024-09-11T06:51:35.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000104862 2024-09-11T06:51:35.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.807 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:35.807 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.595699+0000", 2024-09-11T06:51:35.807 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7376000000000003e-05 2024-09-11T06:51:35.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.807 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:35.807 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.600688+0000", 2024-09-11T06:51:35.807 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0049885279999999999 2024-09-11T06:51:35.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.808 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:35.808 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.600753+0000", 2024-09-11T06:51:35.808 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5549000000000005e-05 2024-09-11T06:51:35.808 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.808 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:35.808 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.808 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.808 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.6081.0:30 2.4 2:2d66d25d:::journal_data.2.1314f0a2bfa2.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+full_force+supports_pool_eio e101)", 2024-09-11T06:51:35.808 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:20.532261+0000", 2024-09-11T06:51:35.808 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.220452435, 2024-09-11T06:51:35.809 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0081325019999999998, 2024-09-11T06:51:35.809 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:35.809 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:35.809 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.809 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:35.809 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.6081", 2024-09-11T06:51:35.809 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3800899964", 2024-09-11T06:51:35.809 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 30 2024-09-11T06:51:35.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.809 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:35.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:35.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.532261+0000", 2024-09-11T06:51:35.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:35.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.532261+0000", 2024-09-11T06:51:35.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.811 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:35.811 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.532263+0000", 2024-09-11T06:51:35.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.34e-06 2024-09-11T06:51:35.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.811 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:35.811 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.532274+0000", 2024-09-11T06:51:35.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.045e-05 2024-09-11T06:51:35.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.811 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:35.812 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.532305+0000", 2024-09-11T06:51:35.812 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0713000000000002e-05 2024-09-11T06:51:35.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.812 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.812 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.532330+0000", 2024-09-11T06:51:35.812 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5066999999999999e-05 2024-09-11T06:51:35.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.812 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:35.812 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.534090+0000", 2024-09-11T06:51:35.812 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0017608369999999999 2024-09-11T06:51:35.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.813 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:35.813 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.534258+0000", 2024-09-11T06:51:35.813 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016717300000000001 2024-09-11T06:51:35.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.813 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 4", 2024-09-11T06:51:35.813 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.534409+0000", 2024-09-11T06:51:35.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000150896 2024-09-11T06:51:35.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.814 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:35.814 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.535023+0000", 2024-09-11T06:51:35.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00061441600000000001 2024-09-11T06:51:35.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.814 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:35.814 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.540274+0000", 2024-09-11T06:51:35.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052512879999999998 2024-09-11T06:51:35.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:35.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.540317+0000", 2024-09-11T06:51:35.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.278e-05 2024-09-11T06:51:35.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:35.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.540394+0000", 2024-09-11T06:51:35.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6542e-05 2024-09-11T06:51:35.816 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.816 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:35.816 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.816 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.6081.0:37 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-11T06:51:35.816 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:20.548027+0000", 2024-09-11T06:51:35.816 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.204686902000001, 2024-09-11T06:51:35.816 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.010236906, 2024-09-11T06:51:35.816 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:35.816 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:35.816 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.817 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:35.817 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.6081", 2024-09-11T06:51:35.817 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3800899964", 2024-09-11T06:51:35.817 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 37 2024-09-11T06:51:35.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.817 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:35.817 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.817 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:35.817 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.548027+0000", 2024-09-11T06:51:35.817 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.818 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:35.818 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.548027+0000", 2024-09-11T06:51:35.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.818 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:35.818 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.548028+0000", 2024-09-11T06:51:35.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7910000000000001e-06 2024-09-11T06:51:35.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:35.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.548035+0000", 2024-09-11T06:51:35.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6039999999999998e-06 2024-09-11T06:51:35.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:35.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.548049+0000", 2024-09-11T06:51:35.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3594000000000001e-05 2024-09-11T06:51:35.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.548057+0000", 2024-09-11T06:51:35.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4749999999999993e-06 2024-09-11T06:51:35.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:35.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.548120+0000", 2024-09-11T06:51:35.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2958000000000003e-05 2024-09-11T06:51:35.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:35.821 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.548197+0000", 2024-09-11T06:51:35.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7260000000000002e-05 2024-09-11T06:51:35.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.821 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 4", 2024-09-11T06:51:35.821 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.548337+0000", 2024-09-11T06:51:35.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013991200000000001 2024-09-11T06:51:35.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.821 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:35.821 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.548933+0000", 2024-09-11T06:51:35.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00059574499999999996 2024-09-11T06:51:35.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.822 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.822 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:35.822 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.558153+0000", 2024-09-11T06:51:35.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0092196959999999994 2024-09-11T06:51:35.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.822 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.822 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:35.822 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.558197+0000", 2024-09-11T06:51:35.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3856999999999997e-05 2024-09-11T06:51:35.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.823 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:35.823 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.558264+0000", 2024-09-11T06:51:35.823 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7014000000000006e-05 2024-09-11T06:51:35.823 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.823 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:35.823 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.824 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.5723.0:81 2.2 e111/99 2:56207b30:ns::rbd_data.11adc335555f.0000000000000000:head v 111'11, mlcod=108'9)", 2024-09-11T06:51:35.824 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:34.288534+0000", 2024-09-11T06:51:35.824 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 1.4641793380000001, 2024-09-11T06:51:35.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0094105249999999994, 2024-09-11T06:51:35.824 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:35.824 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:35.824 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.824 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:35.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.824 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:35.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.288534+0000", 2024-09-11T06:51:35.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:35.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.288534+0000", 2024-09-11T06:51:35.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:35.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.288535+0000", 2024-09-11T06:51:35.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2699999999999999e-07 2024-09-11T06:51:35.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.826 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:35.826 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.288543+0000", 2024-09-11T06:51:35.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0560000000000005e-06 2024-09-11T06:51:35.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.826 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:35.826 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.288571+0000", 2024-09-11T06:51:35.827 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7966999999999998e-05 2024-09-11T06:51:35.827 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.827 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.827 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.827 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.288585+0000", 2024-09-11T06:51:35.827 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4153e-05 2024-09-11T06:51:35.827 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.827 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.827 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:35.827 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.288680+0000", 2024-09-11T06:51:35.827 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4971999999999998e-05 2024-09-11T06:51:35.827 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.827 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.827 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:35.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.288714+0000", 2024-09-11T06:51:35.828 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3661000000000002e-05 2024-09-11T06:51:35.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:35.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.297824+0000", 2024-09-11T06:51:35.828 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0091100069999999998 2024-09-11T06:51:35.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:35.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.297945+0000", 2024-09-11T06:51:35.828 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012118200000000001 2024-09-11T06:51:35.828 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.829 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:35.829 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.829 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.5723.0:84 2.5 e111/69 2:bca04221:ns::rbd_data.11adc335555f.0000000000000003:head v 111'166, mlcod=110'164)", 2024-09-11T06:51:35.829 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:34.288711+0000", 2024-09-11T06:51:35.829 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 1.4640026690000001, 2024-09-11T06:51:35.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.015154005999999999, 2024-09-11T06:51:35.829 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:35.829 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:35.829 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.829 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:35.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:35.830 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.288711+0000", 2024-09-11T06:51:35.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.830 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:35.830 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.288711+0000", 2024-09-11T06:51:35.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.830 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:35.830 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.288713+0000", 2024-09-11T06:51:35.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.843e-06 2024-09-11T06:51:35.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:35.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.288743+0000", 2024-09-11T06:51:35.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0599999999999998e-05 2024-09-11T06:51:35.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:35.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.288763+0000", 2024-09-11T06:51:35.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9505999999999999e-05 2024-09-11T06:51:35.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.288779+0000", 2024-09-11T06:51:35.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6523000000000001e-05 2024-09-11T06:51:35.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:35.832 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.288856+0000", 2024-09-11T06:51:35.832 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6551999999999995e-05 2024-09-11T06:51:35.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:35.832 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.288879+0000", 2024-09-11T06:51:35.832 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3462999999999999e-05 2024-09-11T06:51:35.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:35.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.303759+0000", 2024-09-11T06:51:35.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.014879417000000001 2024-09-11T06:51:35.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.833 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:35.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.303865+0000", 2024-09-11T06:51:35.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010610199999999999 2024-09-11T06:51:35.833 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.833 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:35.833 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.833 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.5723.0:82 2.4 e111/102 2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head v 111'199, mlcod=110'197)", 2024-09-11T06:51:35.833 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:34.289856+0000", 2024-09-11T06:51:35.834 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 1.4628578299999999, 2024-09-11T06:51:35.834 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0052696699999999997, 2024-09-11T06:51:35.834 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:35.834 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:35.834 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.834 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:35.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.834 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:35.834 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.289856+0000", 2024-09-11T06:51:35.834 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:35.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.289856+0000", 2024-09-11T06:51:35.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:35.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.289857+0000", 2024-09-11T06:51:35.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0999999999999997e-07 2024-09-11T06:51:35.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:35.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.289867+0000", 2024-09-11T06:51:35.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.027e-05 2024-09-11T06:51:35.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:35.836 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.289891+0000", 2024-09-11T06:51:35.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3799999999999999e-05 2024-09-11T06:51:35.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.836 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.289910+0000", 2024-09-11T06:51:35.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9321999999999999e-05 2024-09-11T06:51:35.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:35.837 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.290081+0000", 2024-09-11T06:51:35.837 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000171056 2024-09-11T06:51:35.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.837 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.837 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:35.837 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.290134+0000", 2024-09-11T06:51:35.837 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3325999999999997e-05 2024-09-11T06:51:35.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.837 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.837 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:35.837 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.295038+0000", 2024-09-11T06:51:35.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0049037969999999997 2024-09-11T06:51:35.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:35.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.295125+0000", 2024-09-11T06:51:35.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7288999999999997e-05 2024-09-11T06:51:35.838 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.838 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:35.838 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.838 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.5723.0:87 2.5 e111/69 2:b515ded6:ns::rbd_data.11adc335555f.0000000000000006:head v 111'167, mlcod=110'164)", 2024-09-11T06:51:35.838 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:34.290826+0000", 2024-09-11T06:51:35.838 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 1.461887682, 2024-09-11T06:51:35.839 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.013152818, 2024-09-11T06:51:35.839 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:35.839 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:35.839 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.839 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:35.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.839 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:35.839 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.290826+0000", 2024-09-11T06:51:35.839 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.839 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:35.839 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.290826+0000", 2024-09-11T06:51:35.839 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.840 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:35.840 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.290827+0000", 2024-09-11T06:51:35.840 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2529999999999999e-06 2024-09-11T06:51:35.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.840 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:35.840 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.290836+0000", 2024-09-11T06:51:35.840 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3100000000000006e-06 2024-09-11T06:51:35.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:35.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.290855+0000", 2024-09-11T06:51:35.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8756000000000001e-05 2024-09-11T06:51:35.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.290868+0000", 2024-09-11T06:51:35.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2366e-05 2024-09-11T06:51:35.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:35.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.294428+0000", 2024-09-11T06:51:35.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035604109999999999 2024-09-11T06:51:35.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.842 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:35.842 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.294479+0000", 2024-09-11T06:51:35.842 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1484999999999999e-05 2024-09-11T06:51:35.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.842 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:35.842 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.303869+0000", 2024-09-11T06:51:35.842 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0093896280000000006 2024-09-11T06:51:35.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.842 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:35.842 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.303979+0000", 2024-09-11T06:51:35.842 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000109609 2024-09-11T06:51:35.843 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.843 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:35.843 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.843 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.843 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.5723.0:90 2.5 e111/69 2:b0f71fdb:ns::rbd_data.11adc335555f.0000000000000009:head v 111'168, mlcod=110'164)", 2024-09-11T06:51:35.843 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:34.291921+0000", 2024-09-11T06:51:35.843 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 1.4607930009999999, 2024-09-11T06:51:35.843 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.012272585000000001, 2024-09-11T06:51:35.843 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:35.844 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:35.844 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.844 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:35.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.844 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:35.844 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.291921+0000", 2024-09-11T06:51:35.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.844 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:35.844 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.291921+0000", 2024-09-11T06:51:35.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.845 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:35.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.291921+0000", 2024-09-11T06:51:35.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3300000000000001e-07 2024-09-11T06:51:35.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.845 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:35.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.291929+0000", 2024-09-11T06:51:35.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3139999999999998e-06 2024-09-11T06:51:35.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.845 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:35.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.291941+0000", 2024-09-11T06:51:35.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2624e-05 2024-09-11T06:51:35.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.291949+0000", 2024-09-11T06:51:35.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6070000000000002e-06 2024-09-11T06:51:35.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:35.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.299053+0000", 2024-09-11T06:51:35.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0071042630000000004 2024-09-11T06:51:35.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:35.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.299114+0000", 2024-09-11T06:51:35.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0965999999999999e-05 2024-09-11T06:51:35.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.847 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:35.847 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.304084+0000", 2024-09-11T06:51:35.847 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0049695820000000002 2024-09-11T06:51:35.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.847 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:35.847 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.304193+0000", 2024-09-11T06:51:35.847 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010939599999999999 2024-09-11T06:51:35.847 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.847 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:35.847 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.848 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.5723.0:91 2.2 e111/99 2:45bb0ce9:ns::rbd_data.11adc335555f.000000000000000a:head v 111'12, mlcod=108'9)", 2024-09-11T06:51:35.848 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:34.292400+0000", 2024-09-11T06:51:35.848 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 1.4603139139999999, 2024-09-11T06:51:35.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0059090540000000004, 2024-09-11T06:51:35.848 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:35.848 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:35.848 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.848 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:35.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.848 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:35.848 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.292400+0000", 2024-09-11T06:51:35.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.849 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:35.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.292400+0000", 2024-09-11T06:51:35.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.849 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:35.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.292401+0000", 2024-09-11T06:51:35.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.17e-06 2024-09-11T06:51:35.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.849 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:35.850 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.292413+0000", 2024-09-11T06:51:35.850 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2213e-05 2024-09-11T06:51:35.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.850 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:35.850 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.292438+0000", 2024-09-11T06:51:35.850 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5143e-05 2024-09-11T06:51:35.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.850 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.850 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.292457+0000", 2024-09-11T06:51:35.850 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9089999999999998e-05 2024-09-11T06:51:35.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:35.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.293740+0000", 2024-09-11T06:51:35.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012826140000000001 2024-09-11T06:51:35.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:35.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.293778+0000", 2024-09-11T06:51:35.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8470000000000003e-05 2024-09-11T06:51:35.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:35.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.298215+0000", 2024-09-11T06:51:35.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0044362619999999998 2024-09-11T06:51:35.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.852 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.852 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:35.852 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.298309+0000", 2024-09-11T06:51:35.852 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4092000000000006e-05 2024-09-11T06:51:35.852 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.852 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:35.852 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.853 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.853 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.5723.0:83 2.1 e111/102 2:86c3c9db:ns::rbd_data.11adc335555f.0000000000000002:head v 111'251, mlcod=110'249)", 2024-09-11T06:51:35.853 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:34.491414+0000", 2024-09-11T06:51:35.853 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 1.2612996750000001, 2024-09-11T06:51:35.853 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0077254929999999999, 2024-09-11T06:51:35.853 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:35.853 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:35.854 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.854 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:35.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.854 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:35.854 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.491414+0000", 2024-09-11T06:51:35.854 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.854 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:35.854 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.491414+0000", 2024-09-11T06:51:35.854 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.854 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:35.855 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.491416+0000", 2024-09-11T06:51:35.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.863e-06 2024-09-11T06:51:35.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.855 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:35.855 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.491429+0000", 2024-09-11T06:51:35.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2972999999999999e-05 2024-09-11T06:51:35.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.855 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:35.855 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.491456+0000", 2024-09-11T06:51:35.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7460000000000001e-05 2024-09-11T06:51:35.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.855 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.856 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.491471+0000", 2024-09-11T06:51:35.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4477e-05 2024-09-11T06:51:35.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.856 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:35.856 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.491607+0000", 2024-09-11T06:51:35.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013675100000000001 2024-09-11T06:51:35.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.856 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:35.856 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.491675+0000", 2024-09-11T06:51:35.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7760000000000002e-05 2024-09-11T06:51:35.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.857 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:35.857 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.499061+0000", 2024-09-11T06:51:35.857 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0073862540000000001 2024-09-11T06:51:35.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.857 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:35.857 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.499139+0000", 2024-09-11T06:51:35.857 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7954999999999995e-05 2024-09-11T06:51:35.857 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.857 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:35.857 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.857 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.5723.0:85 2.1 e111/102 2:8c2f6822:ns::rbd_data.11adc335555f.0000000000000004:head v 111'252, mlcod=110'249)", 2024-09-11T06:51:35.858 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:34.493769+0000", 2024-09-11T06:51:35.858 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 1.258944077, 2024-09-11T06:51:35.858 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0081260529999999994, 2024-09-11T06:51:35.858 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:35.858 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:35.858 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.858 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:35.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.858 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:35.858 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.493769+0000", 2024-09-11T06:51:35.858 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.858 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.859 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:35.859 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.493769+0000", 2024-09-11T06:51:35.859 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.859 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.859 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:35.859 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.493771+0000", 2024-09-11T06:51:35.859 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.626e-06 2024-09-11T06:51:35.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.859 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.859 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:35.860 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.493780+0000", 2024-09-11T06:51:35.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2070000000000001e-06 2024-09-11T06:51:35.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.860 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:35.860 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.493800+0000", 2024-09-11T06:51:35.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9643000000000001e-05 2024-09-11T06:51:35.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.860 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.860 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.493811+0000", 2024-09-11T06:51:35.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0823e-05 2024-09-11T06:51:35.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.861 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:35.861 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.495750+0000", 2024-09-11T06:51:35.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0019391720000000001 2024-09-11T06:51:35.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.861 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:35.861 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.495809+0000", 2024-09-11T06:51:35.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8635999999999999e-05 2024-09-11T06:51:35.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.861 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:35.861 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.501818+0000", 2024-09-11T06:51:35.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0060095069999999999 2024-09-11T06:51:35.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.862 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:35.862 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.501896+0000", 2024-09-11T06:51:35.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7439000000000002e-05 2024-09-11T06:51:35.862 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.862 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:35.862 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.862 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.5723.0:86 2.1 e111/102 2:92890cf1:ns::rbd_data.11adc335555f.0000000000000005:head v 111'253, mlcod=110'249)", 2024-09-11T06:51:35.862 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:34.496836+0000", 2024-09-11T06:51:35.862 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 1.2558779819999999, 2024-09-11T06:51:35.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0053761379999999999, 2024-09-11T06:51:35.863 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:35.863 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:35.863 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:35.863 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:35.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:35.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.496836+0000", 2024-09-11T06:51:35.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:35.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.496836+0000", 2024-09-11T06:51:35.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:35.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.864 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:35.864 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.496836+0000", 2024-09-11T06:51:35.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5499999999999997e-07 2024-09-11T06:51:35.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.864 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:35.864 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.496842+0000", 2024-09-11T06:51:35.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9719999999999997e-06 2024-09-11T06:51:35.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.864 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:35.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.496853+0000", 2024-09-11T06:51:35.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1165000000000001e-05 2024-09-11T06:51:35.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:35.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.496861+0000", 2024-09-11T06:51:35.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7600000000000002e-06 2024-09-11T06:51:35.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:35.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.499197+0000", 2024-09-11T06:51:35.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0023360579999999998 2024-09-11T06:51:35.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:35.866 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.499248+0000", 2024-09-11T06:51:35.866 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0625999999999999e-05 2024-09-11T06:51:35.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.866 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:35.866 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.502130+0000", 2024-09-11T06:51:35.866 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028822629999999999 2024-09-11T06:51:35.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:35.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:35.866 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:35.866 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.502212+0000", 2024-09-11T06:51:35.866 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1539000000000007e-05 2024-09-11T06:51:35.866 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.867 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:35.867 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.867 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:35.867 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:35.867 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:35.869 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T06:51:35.869 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-11T06:51:35.888 INFO:tasks.thrashosds:joining thrashosds 2024-09-11T06:51:35.888 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T06:51:35.889 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-11T06:51:35.862+0000 7f33c4a63640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25177) UID: 0 2024-09-11T06:51:35.940 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:51:35.940 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:51:35.959 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":183,"stamp":"2024-09-11T06:51:35.144390+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":67699305,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":6925,"num_read_kb":120307,"num_write":1098,"num_write_kb":275366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":262474437,"num_keys_recovered":41,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1183,"ondisk_log_size":1183,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":294604,"kb_used_data":138648,"kb_used_omap":7,"kb_used_meta":155832,"kb_avail":471564596,"statfs":{"total":483183820800,"available":482882146304,"internally_reserved":0,"allocated":141975552,"data_stored":137935736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7971,"internal_metadata":159572189},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-83956026,"num_objects":-50,"num_object_clones":-20,"num_object_copies":-100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-50,"num_whiteouts":-12,"num_read":485,"num_read_kb":403,"num_write":65,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-9,"num_bytes_recovered":-124,"num_keys_recovered":-10,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.027067"},"pg_stats":[{"pgid":"1.0","version":"14'117","reported_seq":358,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-11T06:51:33.131904+0000","last_change":"2024-09-11T06:50:37.941212+0000","last_active":"2024-09-11T06:51:33.131904+0000","last_peered":"2024-09-11T06:51:33.131904+0000","last_clean":"2024-09-11T06:51:33.131904+0000","last_became_active":"2024-09-11T06:50:37.940448+0000","last_became_peered":"2024-09-11T06:50:37.940448+0000","last_unstale":"2024-09-11T06:51:33.131904+0000","last_undegraded":"2024-09-11T06:51:33.131904+0000","last_fullsized":"2024-09-11T06:51:33.131904+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_clean_scrub_stamp":"2024-09-11T06:49:05.088839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:48:59.833858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"110'127","reported_seq":395,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-11T06:51:33.337661+0000","last_change":"2024-09-11T06:51:33.212360+0000","last_active":"2024-09-11T06:51:33.337661+0000","last_peered":"2024-09-11T06:51:33.337661+0000","last_clean":"2024-09-11T06:51:33.337661+0000","last_became_active":"2024-09-11T06:50:47.306211+0000","last_became_peered":"2024-09-11T06:50:47.306211+0000","last_unstale":"2024-09-11T06:51:33.337661+0000","last_undegraded":"2024-09-11T06:51:33.337661+0000","last_fullsized":"2024-09-11T06:51:33.337661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:02.000399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091565099999999996,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":383,"num_read_kb":8504,"num_write":97,"num_write_kb":28714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":25165862,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"3"}]},{"pgid":"2.0","version":"109'81","reported_seq":317,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-11T06:51:33.132712+0000","last_change":"2024-09-11T06:51:33.133348+0000","last_active":"2024-09-11T06:51:33.132712+0000","last_peered":"2024-09-11T06:51:33.132712+0000","last_clean":"2024-09-11T06:51:33.132712+0000","last_became_active":"2024-09-11T06:51:18.986870+0000","last_became_peered":"2024-09-11T06:51:18.986870+0000","last_unstale":"2024-09-11T06:51:33.132712+0000","last_undegraded":"2024-09-11T06:51:33.132712+0000","last_fullsized":"2024-09-11T06:51:33.132712+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:40:04.828405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001177885,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":23060,"num_write":59,"num_write_kb":45056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":62914560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"3"}]},{"pgid":"2.1","version":"110'250","reported_seq":2211,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-11T06:51:34.041659+0000","last_change":"2024-09-11T06:51:33.130678+0000","last_active":"2024-09-11T06:51:34.041659+0000","last_peered":"2024-09-11T06:51:34.041659+0000","last_clean":"2024-09-11T06:51:34.041659+0000","last_became_active":"2024-09-11T06:51:21.988695+0000","last_became_peered":"2024-09-11T06:51:21.988695+0000","last_unstale":"2024-09-11T06:51:34.041659+0000","last_undegraded":"2024-09-11T06:51:34.041659+0000","last_fullsized":"2024-09-11T06:51:34.041659+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:42:19.055371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074871099999999999,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4173,"num_read_kb":27885,"num_write":140,"num_write_kb":41060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":25231476,"num_keys_recovered":13,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"3"}]},{"pgid":"2.2","version":"109'10","reported_seq":208,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-11T06:51:33.966747+0000","last_change":"2024-09-11T06:51:33.212649+0000","last_active":"2024-09-11T06:51:33.966747+0000","last_peered":"2024-09-11T06:51:33.966747+0000","last_clean":"2024-09-11T06:51:33.966747+0000","last_became_active":"2024-09-11T06:51:18.987217+0000","last_became_peered":"2024-09-11T06:51:18.987217+0000","last_unstale":"2024-09-11T06:51:33.966747+0000","last_undegraded":"2024-09-11T06:51:33.966747+0000","last_fullsized":"2024-09-11T06:51:33.966747+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:13.143403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011855349999999999,"stat_sum":{"num_bytes":8388627,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":89,"num_read_kb":86,"num_write":8,"num_write_kb":8196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":16777427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"3"}]},{"pgid":"2.4","version":"110'198","reported_seq":500,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-11T06:51:33.319501+0000","last_change":"2024-09-11T06:51:33.141895+0000","last_active":"2024-09-11T06:51:33.319501+0000","last_peered":"2024-09-11T06:51:33.319501+0000","last_clean":"2024-09-11T06:51:33.319501+0000","last_became_active":"2024-09-11T06:51:21.991171+0000","last_became_peered":"2024-09-11T06:51:21.991171+0000","last_unstale":"2024-09-11T06:51:33.319501+0000","last_undegraded":"2024-09-11T06:51:33.319501+0000","last_fullsized":"2024-09-11T06:51:33.319501+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:31:01.579481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027543540000000001,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":854,"num_read_kb":8870,"num_write":298,"num_write_kb":20705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":12582950,"num_keys_recovered":3,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"3"}]},{"pgid":"2.5","version":"110'164","reported_seq":633,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-11T06:51:34.016493+0000","last_change":"2024-09-11T06:51:33.141183+0000","last_active":"2024-09-11T06:51:34.016493+0000","last_peered":"2024-09-11T06:51:34.016493+0000","last_clean":"2024-09-11T06:51:34.016493+0000","last_became_active":"2024-09-11T06:50:35.236333+0000","last_became_peered":"2024-09-11T06:50:35.236333+0000","last_unstale":"2024-09-11T06:51:34.016493+0000","last_undegraded":"2024-09-11T06:51:34.016493+0000","last_fullsized":"2024-09-11T06:51:34.016493+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:52:55.118390+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0089129259999999998,"stat_sum":{"num_bytes":12582928,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":619,"num_read_kb":21604,"num_write":145,"num_write_kb":45181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":41943104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"3"}]},{"pgid":"2.6","version":"110'54","reported_seq":298,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-11T06:51:34.036620+0000","last_change":"2024-09-11T06:51:33.133667+0000","last_active":"2024-09-11T06:51:34.036620+0000","last_peered":"2024-09-11T06:51:34.036620+0000","last_clean":"2024-09-11T06:51:34.036620+0000","last_became_active":"2024-09-11T06:51:18.984613+0000","last_became_peered":"2024-09-11T06:51:18.984613+0000","last_unstale":"2024-09-11T06:51:34.036620+0000","last_undegraded":"2024-09-11T06:51:34.036620+0000","last_fullsized":"2024-09-11T06:51:34.036620+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:56:03.861829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001463385,"stat_sum":{"num_bytes":4194342,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":8293,"num_write":62,"num_write_kb":16421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":25165938,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"3"}]},{"pgid":"2.7","version":"110'182","reported_seq":364,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-11T06:51:34.039844+0000","last_change":"2024-09-11T06:51:33.140084+0000","last_active":"2024-09-11T06:51:34.039844+0000","last_peered":"2024-09-11T06:51:34.039844+0000","last_clean":"2024-09-11T06:51:34.039844+0000","last_became_active":"2024-09-11T06:50:35.238011+0000","last_became_peered":"2024-09-11T06:50:35.238011+0000","last_unstale":"2024-09-11T06:51:34.039844+0000","last_undegraded":"2024-09-11T06:51:34.039844+0000","last_fullsized":"2024-09-11T06:51:34.039844+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:26:38.712163+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0076355750000000003,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":414,"num_read_kb":21905,"num_write":130,"num_write_kb":67617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":50331648,"num_keys_recovered":16,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"3"}]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":67108937,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":6809,"num_read_kb":120207,"num_write":939,"num_write_kb":272950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":260112965,"num_keys_recovered":41,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":193044480,"data_stored":193003898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1066,"ondisk_log_size":1066,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640288,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44764,"kb_used_data":13572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94327076,"statfs":{"total":96636764160,"available":96590925824,"internally_reserved":0,"allocated":13897728,"data_stored":13090348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640289,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48856,"kb_used_data":17664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94322984,"statfs":{"total":96636764160,"available":96586735616,"internally_reserved":0,"allocated":18087936,"data_stored":17284671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640289,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61732,"kb_used_data":30540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94310108,"statfs":{"total":96636764160,"available":96573550592,"internally_reserved":0,"allocated":31272960,"data_stored":30457967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640290,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52948,"kb_used_data":21760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94318892,"statfs":{"total":96636764160,"available":96582545408,"internally_reserved":0,"allocated":22282240,"data_stored":21478975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":47244640290,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640291,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86304,"kb_used_data":55112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94285536,"statfs":{"total":96636764160,"available":96548388864,"internally_reserved":0,"allocated":56434688,"data_stored":55623775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54534144,"data_stored":54525987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":58802176,"data_stored":58786024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20971520,"data_stored":20971520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":29368320,"data_stored":29360182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16781312,"data_stored":16777254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12587008,"data_stored":12582931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:51:35.960 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-11T06:51:35.960 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T06:51:35.960 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-11T06:51:35.960 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-11T06:51:36.044 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:36.044 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:36.044 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:51:36.044 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:36.059 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T06:51:36.154 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is set 2024-09-11T06:51:36.224 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:36.224 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:36.224 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:51:36.224 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:51:36.224 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:36.238 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T06:51:36.298 INFO:tasks.ceph.mon.a.smithi071.stderr:2024-09-11T06:51:36.294+0000 7f1296554640 -1 mon.a@0(leader).osd e113 definitely_dead 0 2024-09-11T06:51:36.354 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:36.354 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:51:36.354 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:51:36.355 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:51:36.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.355 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4558.0:5 3.2 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-11T06:51:36.355 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:51:36.355 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 78.261115911000005, 2024-09-11T06:51:36.355 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0062353959999999998, 2024-09-11T06:51:36.355 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.356 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.356 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.356 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.356 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4558", 2024-09-11T06:51:36.356 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1478388898", 2024-09-11T06:51:36.356 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 5 2024-09-11T06:51:36.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.356 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.356 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.356 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:51:36.357 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.357 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.357 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088418+0000", 2024-09-11T06:51:36.357 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.357 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.358 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088422+0000", 2024-09-11T06:51:36.358 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7859999999999998e-06 2024-09-11T06:51:36.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.358 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.358 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088440+0000", 2024-09-11T06:51:36.358 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8185999999999998e-05 2024-09-11T06:51:36.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.358 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088479+0000", 2024-09-11T06:51:36.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8248999999999999e-05 2024-09-11T06:51:36.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.359 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088497+0000", 2024-09-11T06:51:36.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8889000000000002e-05 2024-09-11T06:51:36.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.359 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088658+0000", 2024-09-11T06:51:36.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000160271 2024-09-11T06:51:36.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.360 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op must wait for map", 2024-09-11T06:51:36.360 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.088668+0000", 2024-09-11T06:51:36.360 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0409e-05 2024-09-11T06:51:36.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.360 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.360 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092347+0000", 2024-09-11T06:51:36.360 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00367915 2024-09-11T06:51:36.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.360 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.360 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092541+0000", 2024-09-11T06:51:36.361 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000193833 2024-09-11T06:51:36.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.361 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-11T06:51:36.361 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.092732+0000", 2024-09-11T06:51:36.361 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000190772 2024-09-11T06:51:36.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.361 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.361 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094136+0000", 2024-09-11T06:51:36.361 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014040809999999999 2024-09-11T06:51:36.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.362 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.362 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.362 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094506+0000", 2024-09-11T06:51:36.362 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000369959 2024-09-11T06:51:36.362 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.362 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.362 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.362 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094590+0000", 2024-09-11T06:51:36.362 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4033e-05 2024-09-11T06:51:36.362 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.362 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.362 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.362 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:18.094654+0000", 2024-09-11T06:51:36.362 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3777999999999998e-05 2024-09-11T06:51:36.363 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.363 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.363 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.363 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.363 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4645.0:587 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-11T06:51:36.363 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:51:36.363 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 74.120283255999993, 2024-09-11T06:51:36.363 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0023436289999999999, 2024-09-11T06:51:36.363 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.363 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.363 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.363 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.363 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4645", 2024-09-11T06:51:36.364 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2040966562", 2024-09-11T06:51:36.364 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 587 2024-09-11T06:51:36.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.364 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.364 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.364 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:51:36.364 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.364 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.364 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229251+0000", 2024-09-11T06:51:36.365 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.365 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.365 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229252+0000", 2024-09-11T06:51:36.365 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3799999999999999e-06 2024-09-11T06:51:36.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.365 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.365 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229259+0000", 2024-09-11T06:51:36.365 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4760000000000003e-06 2024-09-11T06:51:36.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.365 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.366 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229271+0000", 2024-09-11T06:51:36.366 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2313000000000001e-05 2024-09-11T06:51:36.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.366 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.366 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229279+0000", 2024-09-11T06:51:36.366 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1610000000000003e-06 2024-09-11T06:51:36.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.366 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.366 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229344+0000", 2024-09-11T06:51:36.366 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4375000000000006e-05 2024-09-11T06:51:36.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.366 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.367 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229404+0000", 2024-09-11T06:51:36.367 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0517999999999999e-05 2024-09-11T06:51:36.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.367 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.367 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:36.367 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.229516+0000", 2024-09-11T06:51:36.367 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011129900000000001 2024-09-11T06:51:36.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.367 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.367 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.367 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231502+0000", 2024-09-11T06:51:36.367 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001986238 2024-09-11T06:51:36.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.367 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.368 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.368 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231537+0000", 2024-09-11T06:51:36.368 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4968000000000001e-05 2024-09-11T06:51:36.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.368 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.368 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.368 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231569+0000", 2024-09-11T06:51:36.368 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2459000000000002e-05 2024-09-11T06:51:36.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.368 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.368 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.368 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:22.231595+0000", 2024-09-11T06:51:36.369 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5442000000000001e-05 2024-09-11T06:51:36.369 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.369 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.369 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.369 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:15 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:36.369 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.709418+0000", 2024-09-11T06:51:36.369 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 65.640116102999997, 2024-09-11T06:51:36.369 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027504280000000001, 2024-09-11T06:51:36.370 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.370 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.370 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.370 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.370 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:36.370 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:36.370 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 15 2024-09-11T06:51:36.370 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.370 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.370 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.371 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709418+0000", 2024-09-11T06:51:36.371 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.371 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.371 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709418+0000", 2024-09-11T06:51:36.371 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.372 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.372 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709420+0000", 2024-09-11T06:51:36.372 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2299999999999998e-06 2024-09-11T06:51:36.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.372 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.372 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709432+0000", 2024-09-11T06:51:36.372 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1897e-05 2024-09-11T06:51:36.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709464+0000", 2024-09-11T06:51:36.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1819000000000001e-05 2024-09-11T06:51:36.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709479+0000", 2024-09-11T06:51:36.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4853e-05 2024-09-11T06:51:36.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.374 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709624+0000", 2024-09-11T06:51:36.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144912 2024-09-11T06:51:36.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.375 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.375 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.709773+0000", 2024-09-11T06:51:36.375 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000149529 2024-09-11T06:51:36.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.376 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:36.376 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.710055+0000", 2024-09-11T06:51:36.376 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000281692 2024-09-11T06:51:36.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.376 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.376 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.711412+0000", 2024-09-11T06:51:36.376 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001356931 2024-09-11T06:51:36.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.377 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.377 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.711890+0000", 2024-09-11T06:51:36.377 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00047768700000000002 2024-09-11T06:51:36.377 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.377 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.377 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.377 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.712050+0000", 2024-09-11T06:51:36.377 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016041499999999999 2024-09-11T06:51:36.377 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.377 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.377 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.712169+0000", 2024-09-11T06:51:36.378 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011846299999999999 2024-09-11T06:51:36.378 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.378 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.378 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.378 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:18 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:36.378 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.713502+0000", 2024-09-11T06:51:36.378 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 65.636032677000003, 2024-09-11T06:51:36.378 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032323069999999998, 2024-09-11T06:51:36.379 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.379 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.379 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.379 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.379 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:36.379 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:36.379 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18 2024-09-11T06:51:36.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.380 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.380 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713502+0000", 2024-09-11T06:51:36.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.380 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713502+0000", 2024-09-11T06:51:36.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.381 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713503+0000", 2024-09-11T06:51:36.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8199999999999999e-06 2024-09-11T06:51:36.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.385 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.385 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713513+0000", 2024-09-11T06:51:36.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8539999999999992e-06 2024-09-11T06:51:36.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.385 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.385 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713540+0000", 2024-09-11T06:51:36.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6594999999999999e-05 2024-09-11T06:51:36.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.386 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.386 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713553+0000", 2024-09-11T06:51:36.386 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2843999999999999e-05 2024-09-11T06:51:36.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.386 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.386 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713686+0000", 2024-09-11T06:51:36.386 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013294899999999999 2024-09-11T06:51:36.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.387 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.387 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.713937+0000", 2024-09-11T06:51:36.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025110600000000002 2024-09-11T06:51:36.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.387 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:36.387 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.714412+0000", 2024-09-11T06:51:36.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00047562 2024-09-11T06:51:36.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.387 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.387 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.715867+0000", 2024-09-11T06:51:36.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001455004 2024-09-11T06:51:36.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.388 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.388 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.716569+0000", 2024-09-11T06:51:36.388 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070159400000000005 2024-09-11T06:51:36.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.388 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.388 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.716660+0000", 2024-09-11T06:51:36.388 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1403000000000006e-05 2024-09-11T06:51:36.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.388 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.389 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.716734+0000", 2024-09-11T06:51:36.389 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3517999999999996e-05 2024-09-11T06:51:36.389 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.389 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.389 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.389 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:23 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:36.389 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.723392+0000", 2024-09-11T06:51:36.389 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 65.626142080999998, 2024-09-11T06:51:36.389 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027098489999999999, 2024-09-11T06:51:36.389 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.389 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.389 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.390 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.390 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:36.390 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:36.390 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23 2024-09-11T06:51:36.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.390 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.390 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.390 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723392+0000", 2024-09-11T06:51:36.390 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.390 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.390 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723392+0000", 2024-09-11T06:51:36.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.391 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.391 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723394+0000", 2024-09-11T06:51:36.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0669999999999999e-06 2024-09-11T06:51:36.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.391 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.392 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723406+0000", 2024-09-11T06:51:36.392 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1367e-05 2024-09-11T06:51:36.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.392 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.392 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723436+0000", 2024-09-11T06:51:36.392 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0874999999999998e-05 2024-09-11T06:51:36.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.392 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.392 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723451+0000", 2024-09-11T06:51:36.392 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4192999999999999e-05 2024-09-11T06:51:36.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.393 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.393 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723577+0000", 2024-09-11T06:51:36.393 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000126267 2024-09-11T06:51:36.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.393 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.393 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723718+0000", 2024-09-11T06:51:36.393 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000141206 2024-09-11T06:51:36.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.394 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:36.394 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.723982+0000", 2024-09-11T06:51:36.394 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026338499999999999 2024-09-11T06:51:36.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.394 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.394 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.725377+0000", 2024-09-11T06:51:36.394 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013956999999999999 2024-09-11T06:51:36.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.725908+0000", 2024-09-11T06:51:36.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00053036200000000004 2024-09-11T06:51:36.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.725999+0000", 2024-09-11T06:51:36.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1016999999999999e-05 2024-09-11T06:51:36.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.726102+0000", 2024-09-11T06:51:36.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010341 2024-09-11T06:51:36.396 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.396 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.396 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.396 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:15 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:36.396 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.868921+0000", 2024-09-11T06:51:36.396 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 65.480613039999994, 2024-09-11T06:51:36.396 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0026879, 2024-09-11T06:51:36.397 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.397 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.397 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.397 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.397 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:36.397 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:36.397 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 15 2024-09-11T06:51:36.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.397 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.397 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.397 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.398 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868921+0000", 2024-09-11T06:51:36.398 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.398 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.398 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.398 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.398 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868921+0000", 2024-09-11T06:51:36.398 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.398 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.398 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.398 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.398 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868923+0000", 2024-09-11T06:51:36.398 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8959999999999999e-06 2024-09-11T06:51:36.398 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.398 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.399 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.399 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868934+0000", 2024-09-11T06:51:36.399 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1083e-05 2024-09-11T06:51:36.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.399 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.399 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868965+0000", 2024-09-11T06:51:36.399 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0833000000000001e-05 2024-09-11T06:51:36.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.399 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.399 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.868990+0000", 2024-09-11T06:51:36.399 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4545999999999998e-05 2024-09-11T06:51:36.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.400 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.400 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.869124+0000", 2024-09-11T06:51:36.400 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013489599999999999 2024-09-11T06:51:36.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.400 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.400 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.400 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.869270+0000", 2024-09-11T06:51:36.400 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000145976 2024-09-11T06:51:36.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.400 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.400 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:36.400 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.869538+0000", 2024-09-11T06:51:36.400 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026780599999999999 2024-09-11T06:51:36.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.401 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.401 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.870691+0000", 2024-09-11T06:51:36.401 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00115277 2024-09-11T06:51:36.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.401 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.401 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.871456+0000", 2024-09-11T06:51:36.401 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00076473800000000005 2024-09-11T06:51:36.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.402 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.402 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.402 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.871543+0000", 2024-09-11T06:51:36.402 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7040999999999996e-05 2024-09-11T06:51:36.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.402 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.402 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.402 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.871609+0000", 2024-09-11T06:51:36.402 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6315000000000005e-05 2024-09-11T06:51:36.402 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.402 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.403 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.403 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.403 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:20 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:36.403 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.877429+0000", 2024-09-11T06:51:36.403 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 65.472104751000003, 2024-09-11T06:51:36.403 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0032382629999999999, 2024-09-11T06:51:36.403 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.403 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.403 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.403 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.403 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:36.403 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:36.403 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 20 2024-09-11T06:51:36.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.404 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.404 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.404 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.404 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877429+0000", 2024-09-11T06:51:36.404 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.404 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.404 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.404 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877429+0000", 2024-09-11T06:51:36.404 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.404 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.404 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.405 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877431+0000", 2024-09-11T06:51:36.405 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9429999999999999e-06 2024-09-11T06:51:36.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.405 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.405 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877444+0000", 2024-09-11T06:51:36.405 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2396e-05 2024-09-11T06:51:36.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.405 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.405 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877474+0000", 2024-09-11T06:51:36.405 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0239999999999998e-05 2024-09-11T06:51:36.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.405 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.406 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877496+0000", 2024-09-11T06:51:36.406 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1914999999999999e-05 2024-09-11T06:51:36.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.406 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.406 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877616+0000", 2024-09-11T06:51:36.406 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000119814 2024-09-11T06:51:36.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.406 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.406 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.877836+0000", 2024-09-11T06:51:36.406 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022029600000000001 2024-09-11T06:51:36.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.407 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:36.407 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.878482+0000", 2024-09-11T06:51:36.407 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00064605200000000004 2024-09-11T06:51:36.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.407 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.407 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.879804+0000", 2024-09-11T06:51:36.407 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013216460000000001 2024-09-11T06:51:36.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.407 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.408 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.880519+0000", 2024-09-11T06:51:36.408 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00071547600000000005 2024-09-11T06:51:36.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.408 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.408 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.408 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.880606+0000", 2024-09-11T06:51:36.408 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6907e-05 2024-09-11T06:51:36.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.408 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.408 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.408 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.880668+0000", 2024-09-11T06:51:36.408 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1577999999999999e-05 2024-09-11T06:51:36.408 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.409 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.409 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.409 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.409 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.409 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:23 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:36.409 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.885067+0000", 2024-09-11T06:51:36.409 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 65.464467096999996, 2024-09-11T06:51:36.409 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028127320000000001, 2024-09-11T06:51:36.409 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.409 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.409 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.409 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.409 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:36.409 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:36.410 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23 2024-09-11T06:51:36.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.410 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.410 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.410 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.410 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885067+0000", 2024-09-11T06:51:36.410 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.410 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.410 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.410 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885067+0000", 2024-09-11T06:51:36.410 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.410 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.411 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.411 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885069+0000", 2024-09-11T06:51:36.411 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2000000000000001e-06 2024-09-11T06:51:36.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.411 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.411 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885081+0000", 2024-09-11T06:51:36.411 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1209999999999999e-05 2024-09-11T06:51:36.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.411 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.411 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885111+0000", 2024-09-11T06:51:36.411 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0318999999999999e-05 2024-09-11T06:51:36.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.412 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.412 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885132+0000", 2024-09-11T06:51:36.412 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0649000000000002e-05 2024-09-11T06:51:36.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.412 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.412 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885243+0000", 2024-09-11T06:51:36.412 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000111284 2024-09-11T06:51:36.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.412 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.412 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885387+0000", 2024-09-11T06:51:36.413 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144222 2024-09-11T06:51:36.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.413 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:36.413 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.885699+0000", 2024-09-11T06:51:36.413 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00031239800000000003 2024-09-11T06:51:36.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.413 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.413 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.886817+0000", 2024-09-11T06:51:36.413 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001117191 2024-09-11T06:51:36.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.413 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.414 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.887684+0000", 2024-09-11T06:51:36.414 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00086699499999999996 2024-09-11T06:51:36.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.414 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.414 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.887772+0000", 2024-09-11T06:51:36.414 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8224000000000001e-05 2024-09-11T06:51:36.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.414 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.414 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.887880+0000", 2024-09-11T06:51:36.414 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010804 2024-09-11T06:51:36.414 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.414 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.415 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.415 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.415 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "MOSDPGPull(2.0 e70/69 cost 1048576)", 2024-09-11T06:51:36.415 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:35.528435+0000", 2024-09-11T06:51:36.415 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 60.821099361000002, 2024-09-11T06:51:36.415 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027390520000000001, 2024-09-11T06:51:36.415 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.415 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.415 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-11T06:51:36.415 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.415 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.415 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.415 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528435+0000", 2024-09-11T06:51:36.416 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.416 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.416 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528435+0000", 2024-09-11T06:51:36.416 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.416 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.416 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528437+0000", 2024-09-11T06:51:36.416 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5959999999999998e-06 2024-09-11T06:51:36.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.417 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.417 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528465+0000", 2024-09-11T06:51:36.417 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7436e-05 2024-09-11T06:51:36.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.417 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.417 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528499+0000", 2024-09-11T06:51:36.417 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3599000000000001e-05 2024-09-11T06:51:36.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.417 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.418 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528515+0000", 2024-09-11T06:51:36.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.677e-05 2024-09-11T06:51:36.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.418 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.418 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.528652+0000", 2024-09-11T06:51:36.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000136546 2024-09-11T06:51:36.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.418 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.418 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:35.531174+0000", 2024-09-11T06:51:36.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0025221050000000002 2024-09-11T06:51:36.418 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.418 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.419 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.419 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4828.0:15 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-11T06:51:36.419 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:36.253730+0000", 2024-09-11T06:51:36.419 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 60.095804018000003, 2024-09-11T06:51:36.419 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0025257589999999998, 2024-09-11T06:51:36.419 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.419 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.419 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.419 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.419 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4828", 2024-09-11T06:51:36.420 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2410288650", 2024-09-11T06:51:36.420 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 15 2024-09-11T06:51:36.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.420 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.420 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.420 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253730+0000", 2024-09-11T06:51:36.420 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.420 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.421 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253730+0000", 2024-09-11T06:51:36.421 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.421 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.421 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.421 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253732+0000", 2024-09-11T06:51:36.421 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2400000000000002e-06 2024-09-11T06:51:36.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.421 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.421 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.421 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253744+0000", 2024-09-11T06:51:36.421 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1316e-05 2024-09-11T06:51:36.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.422 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.422 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253771+0000", 2024-09-11T06:51:36.422 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.724e-05 2024-09-11T06:51:36.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.422 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.422 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253785+0000", 2024-09-11T06:51:36.422 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3786e-05 2024-09-11T06:51:36.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.422 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.423 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.253886+0000", 2024-09-11T06:51:36.423 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000100981 2024-09-11T06:51:36.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.423 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.423 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.254029+0000", 2024-09-11T06:51:36.423 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014311900000000001 2024-09-11T06:51:36.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.423 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:36.424 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.254317+0000", 2024-09-11T06:51:36.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00028809300000000001 2024-09-11T06:51:36.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.424 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.424 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.255403+0000", 2024-09-11T06:51:36.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001085639 2024-09-11T06:51:36.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.424 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.425 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.256107+0000", 2024-09-11T06:51:36.425 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070468799999999997 2024-09-11T06:51:36.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.425 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.425 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.256191+0000", 2024-09-11T06:51:36.425 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3714999999999994e-05 2024-09-11T06:51:36.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.425 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.425 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:36.256256+0000", 2024-09-11T06:51:36.425 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4942000000000003e-05 2024-09-11T06:51:36.426 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.426 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.426 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.426 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.426 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5313.0:26 2.6 2:7b712441:ns::rbd_object_map.11adc335555f:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:51:36.426 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.260230+0000", 2024-09-11T06:51:36.426 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 57.089304609999999, 2024-09-11T06:51:36.426 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002457201, 2024-09-11T06:51:36.426 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.426 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.426 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.427 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.427 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5313", 2024-09-11T06:51:36.427 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/624314229", 2024-09-11T06:51:36.427 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:51:36.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.427 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.427 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.428 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260230+0000", 2024-09-11T06:51:36.428 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.428 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.428 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260230+0000", 2024-09-11T06:51:36.428 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.428 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.428 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260232+0000", 2024-09-11T06:51:36.428 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.956e-06 2024-09-11T06:51:36.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.429 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.429 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260243+0000", 2024-09-11T06:51:36.429 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1687e-05 2024-09-11T06:51:36.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.429 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.429 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260269+0000", 2024-09-11T06:51:36.429 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6211999999999999e-05 2024-09-11T06:51:36.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.429 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.429 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260284+0000", 2024-09-11T06:51:36.429 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4053000000000001e-05 2024-09-11T06:51:36.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.430 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.430 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260391+0000", 2024-09-11T06:51:36.430 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000107278 2024-09-11T06:51:36.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.430 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.430 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260541+0000", 2024-09-11T06:51:36.430 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015033000000000001 2024-09-11T06:51:36.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.430 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 3", 2024-09-11T06:51:36.430 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.260951+0000", 2024-09-11T06:51:36.431 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040981299999999997 2024-09-11T06:51:36.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.431 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.431 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262399+0000", 2024-09-11T06:51:36.431 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014475930000000001 2024-09-11T06:51:36.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.431 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.431 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262531+0000", 2024-09-11T06:51:36.431 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013239 2024-09-11T06:51:36.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.431 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.432 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262623+0000", 2024-09-11T06:51:36.432 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2361000000000006e-05 2024-09-11T06:51:36.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.432 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.432 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.262687+0000", 2024-09-11T06:51:36.432 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3528000000000006e-05 2024-09-11T06:51:36.432 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.432 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.432 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.432 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5313.0:31 2.6 2:7b712441:ns::rbd_object_map.11adc335555f:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-11T06:51:36.433 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:39.266495+0000", 2024-09-11T06:51:36.433 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 57.083038836, 2024-09-11T06:51:36.433 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0026763870000000001, 2024-09-11T06:51:36.433 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.433 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.433 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.433 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.433 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5313", 2024-09-11T06:51:36.433 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/624314229", 2024-09-11T06:51:36.433 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 31 2024-09-11T06:51:36.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.433 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.433 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.433 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.434 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266495+0000", 2024-09-11T06:51:36.434 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.434 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.434 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266495+0000", 2024-09-11T06:51:36.434 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.434 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.434 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266498+0000", 2024-09-11T06:51:36.434 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2369999999999999e-06 2024-09-11T06:51:36.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.435 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.435 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.435 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266509+0000", 2024-09-11T06:51:36.435 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1459e-05 2024-09-11T06:51:36.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.435 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.435 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.435 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266540+0000", 2024-09-11T06:51:36.435 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1189999999999998e-05 2024-09-11T06:51:36.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.435 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.435 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.435 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266563+0000", 2024-09-11T06:51:36.435 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2529000000000001e-05 2024-09-11T06:51:36.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.436 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.436 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.436 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266708+0000", 2024-09-11T06:51:36.436 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014511 2024-09-11T06:51:36.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.436 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.436 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.436 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.266962+0000", 2024-09-11T06:51:36.436 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000253897 2024-09-11T06:51:36.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.436 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.436 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 3", 2024-09-11T06:51:36.436 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.267430+0000", 2024-09-11T06:51:36.436 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00046799899999999998 2024-09-11T06:51:36.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.437 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.437 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.268765+0000", 2024-09-11T06:51:36.437 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013350359999999999 2024-09-11T06:51:36.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.437 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.437 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.269019+0000", 2024-09-11T06:51:36.437 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025397000000000001 2024-09-11T06:51:36.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.438 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.438 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.269105+0000", 2024-09-11T06:51:36.438 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5952000000000006e-05 2024-09-11T06:51:36.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.438 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.438 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.438 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:39.269172+0000", 2024-09-11T06:51:36.438 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7008000000000006e-05 2024-09-11T06:51:36.438 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.438 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.438 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.439 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.439 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:5 3.4 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:36.439 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.588582+0000", 2024-09-11T06:51:36.439 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 50.760952682999999, 2024-09-11T06:51:36.439 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002889857, 2024-09-11T06:51:36.439 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.439 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.439 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.439 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.439 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:51:36.439 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:51:36.439 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 5 2024-09-11T06:51:36.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.439 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.439 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.440 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.440 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588582+0000", 2024-09-11T06:51:36.440 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.440 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.440 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588582+0000", 2024-09-11T06:51:36.440 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.440 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.441 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588583+0000", 2024-09-11T06:51:36.441 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.223e-06 2024-09-11T06:51:36.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.441 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.441 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.441 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588588+0000", 2024-09-11T06:51:36.441 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4299999999999997e-06 2024-09-11T06:51:36.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.441 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.441 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.441 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588606+0000", 2024-09-11T06:51:36.442 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7652999999999999e-05 2024-09-11T06:51:36.442 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.442 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.442 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.442 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588615+0000", 2024-09-11T06:51:36.442 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4399999999999994e-06 2024-09-11T06:51:36.442 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.442 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.442 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.442 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.588725+0000", 2024-09-11T06:51:36.442 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010945799999999999 2024-09-11T06:51:36.442 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.442 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.442 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.589031+0000", 2024-09-11T06:51:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030590000000000001 2024-09-11T06:51:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.589521+0000", 2024-09-11T06:51:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00049061299999999999 2024-09-11T06:51:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.590792+0000", 2024-09-11T06:51:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012708039999999999 2024-09-11T06:51:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.444 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.444 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591339+0000", 2024-09-11T06:51:36.444 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00054710499999999999 2024-09-11T06:51:36.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.444 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.444 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591412+0000", 2024-09-11T06:51:36.444 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2811999999999993e-05 2024-09-11T06:51:36.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.444 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.444 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.591471+0000", 2024-09-11T06:51:36.444 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9419000000000001e-05 2024-09-11T06:51:36.445 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.445 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.445 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.445 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:8 3.4 3:3f0de396:::rbd_header.1415bf940188:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:36.445 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.596120+0000", 2024-09-11T06:51:36.445 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 50.753413799, 2024-09-11T06:51:36.445 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027911020000000002, 2024-09-11T06:51:36.445 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.445 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.445 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.445 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.446 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:51:36.446 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:51:36.446 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 8 2024-09-11T06:51:36.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.446 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.446 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.446 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.446 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596120+0000", 2024-09-11T06:51:36.446 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.446 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.446 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.446 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596120+0000", 2024-09-11T06:51:36.446 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.447 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.447 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.447 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.447 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596122+0000", 2024-09-11T06:51:36.447 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.513e-06 2024-09-11T06:51:36.447 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.447 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.447 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.447 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596130+0000", 2024-09-11T06:51:36.447 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4999999999999999e-06 2024-09-11T06:51:36.447 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.447 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.447 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.447 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596152+0000", 2024-09-11T06:51:36.447 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1846000000000001e-05 2024-09-11T06:51:36.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.448 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.448 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.448 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596163+0000", 2024-09-11T06:51:36.448 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1029999999999999e-05 2024-09-11T06:51:36.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.448 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.448 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.448 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596242+0000", 2024-09-11T06:51:36.448 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8275000000000005e-05 2024-09-11T06:51:36.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.449 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.449 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596439+0000", 2024-09-11T06:51:36.449 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000196999 2024-09-11T06:51:36.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.449 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:36.449 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.596755+0000", 2024-09-11T06:51:36.449 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00031626100000000003 2024-09-11T06:51:36.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.450 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.450 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.598043+0000", 2024-09-11T06:51:36.450 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001288389 2024-09-11T06:51:36.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.450 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.450 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.598774+0000", 2024-09-11T06:51:36.450 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00073080499999999995 2024-09-11T06:51:36.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.450 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.450 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.598855+0000", 2024-09-11T06:51:36.450 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0615e-05 2024-09-11T06:51:36.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.451 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.451 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.451 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.598912+0000", 2024-09-11T06:51:36.451 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6869e-05 2024-09-11T06:51:36.451 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.451 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.451 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.451 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.452 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5141.0:9 3.0 3:00834abc:::rbd_object_map.1415bf940188:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:36.452 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.599161+0000", 2024-09-11T06:51:36.452 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 50.750372781999999, 2024-09-11T06:51:36.452 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002826051, 2024-09-11T06:51:36.452 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.452 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.452 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.452 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.452 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5141", 2024-09-11T06:51:36.452 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2175334233", 2024-09-11T06:51:36.452 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 9 2024-09-11T06:51:36.452 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.453 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.453 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.453 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.453 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599161+0000", 2024-09-11T06:51:36.453 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.453 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.453 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.453 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599161+0000", 2024-09-11T06:51:36.453 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.454 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.454 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.454 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599163+0000", 2024-09-11T06:51:36.454 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8700000000000001e-06 2024-09-11T06:51:36.454 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.454 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.454 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.454 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599174+0000", 2024-09-11T06:51:36.454 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0416e-05 2024-09-11T06:51:36.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.455 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.455 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599202+0000", 2024-09-11T06:51:36.455 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7946e-05 2024-09-11T06:51:36.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.455 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.456 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599216+0000", 2024-09-11T06:51:36.456 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3934000000000001e-05 2024-09-11T06:51:36.456 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.456 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.456 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.456 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599345+0000", 2024-09-11T06:51:36.456 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012954000000000001 2024-09-11T06:51:36.456 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.456 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.456 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.456 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599540+0000", 2024-09-11T06:51:36.456 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019516299999999999 2024-09-11T06:51:36.457 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.457 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.457 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:36.457 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.599796+0000", 2024-09-11T06:51:36.457 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025556500000000002 2024-09-11T06:51:36.457 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.457 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.457 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.457 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.601071+0000", 2024-09-11T06:51:36.457 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001275396 2024-09-11T06:51:36.457 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.457 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.458 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.458 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.601852+0000", 2024-09-11T06:51:36.458 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000780524 2024-09-11T06:51:36.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.458 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.458 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.458 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.601925+0000", 2024-09-11T06:51:36.458 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3574999999999999e-05 2024-09-11T06:51:36.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.459 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.459 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.459 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.601988+0000", 2024-09-11T06:51:36.459 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2121999999999999e-05 2024-09-11T06:51:36.459 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.459 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.459 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.459 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.459 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5038.0:12 3.4 3:3f0de396:::rbd_header.1415bf940188:head [watch watch cookie 140693532068064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-11T06:51:36.459 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:45.718024+0000", 2024-09-11T06:51:36.459 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 50.631509844, 2024-09-11T06:51:36.459 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002724583, 2024-09-11T06:51:36.459 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.459 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.460 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.460 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.460 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5038", 2024-09-11T06:51:36.460 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/945190275", 2024-09-11T06:51:36.460 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:51:36.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.460 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.460 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.460 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.460 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718024+0000", 2024-09-11T06:51:36.460 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.460 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.461 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.461 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718024+0000", 2024-09-11T06:51:36.461 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.461 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.461 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718027+0000", 2024-09-11T06:51:36.461 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1430000000000001e-06 2024-09-11T06:51:36.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.461 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.461 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718037+0000", 2024-09-11T06:51:36.461 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0253e-05 2024-09-11T06:51:36.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718068+0000", 2024-09-11T06:51:36.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1050000000000003e-05 2024-09-11T06:51:36.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718086+0000", 2024-09-11T06:51:36.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8629000000000001e-05 2024-09-11T06:51:36.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718206+0000", 2024-09-11T06:51:36.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000119358 2024-09-11T06:51:36.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718425+0000", 2024-09-11T06:51:36.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000218975 2024-09-11T06:51:36.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:36.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.718830+0000", 2024-09-11T06:51:36.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040538199999999998 2024-09-11T06:51:36.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.719849+0000", 2024-09-11T06:51:36.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010187589999999999 2024-09-11T06:51:36.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.464 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.720548+0000", 2024-09-11T06:51:36.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000699112 2024-09-11T06:51:36.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.464 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.720632+0000", 2024-09-11T06:51:36.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4314999999999995e-05 2024-09-11T06:51:36.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.465 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:45.720749+0000", 2024-09-11T06:51:36.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116607 2024-09-11T06:51:36.465 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.465 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.465 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.465 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:47 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:36.465 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.553228+0000", 2024-09-11T06:51:36.465 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 47.796305754000002, 2024-09-11T06:51:36.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040683100000000003, 2024-09-11T06:51:36.465 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.465 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.466 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.466 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.466 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:36.466 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:36.466 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 47 2024-09-11T06:51:36.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.466 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.466 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.466 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553228+0000", 2024-09-11T06:51:36.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553228+0000", 2024-09-11T06:51:36.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553231+0000", 2024-09-11T06:51:36.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3130000000000001e-06 2024-09-11T06:51:36.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553242+0000", 2024-09-11T06:51:36.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1345999999999999e-05 2024-09-11T06:51:36.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553272+0000", 2024-09-11T06:51:36.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0179999999999999e-05 2024-09-11T06:51:36.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.469 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553287+0000", 2024-09-11T06:51:36.469 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4283e-05 2024-09-11T06:51:36.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.469 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553415+0000", 2024-09-11T06:51:36.469 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000128491 2024-09-11T06:51:36.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.469 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.553651+0000", 2024-09-11T06:51:36.470 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000236328 2024-09-11T06:51:36.470 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.470 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.470 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:36.470 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.554356+0000", 2024-09-11T06:51:36.470 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070436299999999995 2024-09-11T06:51:36.470 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.470 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.470 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.470 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.556064+0000", 2024-09-11T06:51:36.470 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0017084439999999999 2024-09-11T06:51:36.470 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.470 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.471 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.471 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.556993+0000", 2024-09-11T06:51:36.471 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00092912400000000003 2024-09-11T06:51:36.471 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.471 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.471 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.471 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.557173+0000", 2024-09-11T06:51:36.471 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017940400000000001 2024-09-11T06:51:36.471 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.471 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.471 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.471 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.557297+0000", 2024-09-11T06:51:36.471 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000124034 2024-09-11T06:51:36.472 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.472 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.472 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.472 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.472 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.472 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:66 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:36.472 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.589465+0000", 2024-09-11T06:51:36.472 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 47.760068949000001, 2024-09-11T06:51:36.472 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0039855489999999997, 2024-09-11T06:51:36.472 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.472 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.472 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.473 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.473 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:36.473 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:36.473 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 66 2024-09-11T06:51:36.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.473 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.473 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.473 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.473 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589465+0000", 2024-09-11T06:51:36.473 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.473 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.473 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.473 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589465+0000", 2024-09-11T06:51:36.474 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.474 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.474 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.474 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589468+0000", 2024-09-11T06:51:36.474 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3539999999999998e-06 2024-09-11T06:51:36.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.474 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.474 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.474 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589480+0000", 2024-09-11T06:51:36.474 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1946e-05 2024-09-11T06:51:36.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.474 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.474 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.475 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589512+0000", 2024-09-11T06:51:36.475 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2712999999999997e-05 2024-09-11T06:51:36.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.475 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.475 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589527+0000", 2024-09-11T06:51:36.475 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4686e-05 2024-09-11T06:51:36.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.476 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.476 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589673+0000", 2024-09-11T06:51:36.476 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014630700000000001 2024-09-11T06:51:36.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.476 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.476 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.476 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.589936+0000", 2024-09-11T06:51:36.476 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026234499999999998 2024-09-11T06:51:36.477 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.477 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:36.477 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.591001+0000", 2024-09-11T06:51:36.477 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001065023 2024-09-11T06:51:36.477 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.477 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.477 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.592757+0000", 2024-09-11T06:51:36.477 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0017560849999999999 2024-09-11T06:51:36.477 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.477 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.478 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.593297+0000", 2024-09-11T06:51:36.478 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00054007600000000001 2024-09-11T06:51:36.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.478 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.478 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.478 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.593385+0000", 2024-09-11T06:51:36.478 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8327999999999999e-05 2024-09-11T06:51:36.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.478 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.478 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.478 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.593451+0000", 2024-09-11T06:51:36.478 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5685999999999994e-05 2024-09-11T06:51:36.478 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.479 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.479 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.479 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.479 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5315.0:79 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-11T06:51:36.479 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:07.271557+0000", 2024-09-11T06:51:36.479 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 29.077977148999999, 2024-09-11T06:51:36.479 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0023454019999999999, 2024-09-11T06:51:36.479 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.479 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.479 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.479 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.479 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5315", 2024-09-11T06:51:36.480 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/2243844038", 2024-09-11T06:51:36.480 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 79 2024-09-11T06:51:36.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.480 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.480 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.480 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.480 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271557+0000", 2024-09-11T06:51:36.480 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.481 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.481 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.481 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271557+0000", 2024-09-11T06:51:36.481 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.481 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.481 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.481 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271560+0000", 2024-09-11T06:51:36.481 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3560000000000001e-06 2024-09-11T06:51:36.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.481 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.481 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.482 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271596+0000", 2024-09-11T06:51:36.482 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5593000000000003e-05 2024-09-11T06:51:36.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.482 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.482 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271640+0000", 2024-09-11T06:51:36.482 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4125999999999997e-05 2024-09-11T06:51:36.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.482 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.482 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271652+0000", 2024-09-11T06:51:36.482 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1853e-05 2024-09-11T06:51:36.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.483 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.483 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.483 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271785+0000", 2024-09-11T06:51:36.483 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013291299999999999 2024-09-11T06:51:36.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.483 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.483 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.483 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.271948+0000", 2024-09-11T06:51:36.483 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000163191 2024-09-11T06:51:36.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.483 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.484 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:36.484 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.272260+0000", 2024-09-11T06:51:36.484 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00031196700000000002 2024-09-11T06:51:36.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.484 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.484 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.484 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.273293+0000", 2024-09-11T06:51:36.484 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0010324139999999999 2024-09-11T06:51:36.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.484 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.484 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.484 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.273805+0000", 2024-09-11T06:51:36.485 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00051249399999999995 2024-09-11T06:51:36.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.485 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.485 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.485 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.273858+0000", 2024-09-11T06:51:36.485 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2975e-05 2024-09-11T06:51:36.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.485 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.485 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.485 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:07.273902+0000", 2024-09-11T06:51:36.486 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4520000000000001e-05 2024-09-11T06:51:36.486 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.486 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.486 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.486 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.486 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.486 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5239.0:52 2.4 2:27499dce:::rbd_object_map.1314f0a2bfa2:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-11T06:51:36.486 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:08.176534+0000", 2024-09-11T06:51:36.486 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 28.173000223999999, 2024-09-11T06:51:36.486 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0022800960000000001, 2024-09-11T06:51:36.486 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.486 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.486 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.487 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.487 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5239", 2024-09-11T06:51:36.487 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3565361812", 2024-09-11T06:51:36.487 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 52 2024-09-11T06:51:36.487 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.487 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.487 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.487 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.487 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176534+0000", 2024-09-11T06:51:36.487 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.487 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.487 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.487 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.488 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176534+0000", 2024-09-11T06:51:36.488 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.488 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.488 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.488 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.488 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176537+0000", 2024-09-11T06:51:36.488 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5600000000000001e-06 2024-09-11T06:51:36.488 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.488 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.488 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.488 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176551+0000", 2024-09-11T06:51:36.488 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3926e-05 2024-09-11T06:51:36.488 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.488 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.489 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.489 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176584+0000", 2024-09-11T06:51:36.489 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3105999999999998e-05 2024-09-11T06:51:36.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.489 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.489 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.489 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176600+0000", 2024-09-11T06:51:36.489 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6327000000000001e-05 2024-09-11T06:51:36.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.489 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.489 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.489 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176753+0000", 2024-09-11T06:51:36.489 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015280900000000001 2024-09-11T06:51:36.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.490 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.490 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.490 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.176911+0000", 2024-09-11T06:51:36.490 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000158145 2024-09-11T06:51:36.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.490 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.490 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:36.490 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.177216+0000", 2024-09-11T06:51:36.490 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030545000000000002 2024-09-11T06:51:36.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.490 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.490 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.490 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.178461+0000", 2024-09-11T06:51:36.490 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012451210000000001 2024-09-11T06:51:36.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.491 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.491 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.178706+0000", 2024-09-11T06:51:36.491 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024469999999999998 2024-09-11T06:51:36.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.491 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.491 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.178763+0000", 2024-09-11T06:51:36.491 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7339000000000001e-05 2024-09-11T06:51:36.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.491 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.491 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:08.178814+0000", 2024-09-11T06:51:36.492 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0612999999999998e-05 2024-09-11T06:51:36.492 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.492 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.492 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.492 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.492 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.492 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:36.494 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T06:51:36.613 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:36.613 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:36.614 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-11T06:51:36.614 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:36.626 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T06:51:36.749 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:36.749 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-11T06:51:36.749 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-11T06:51:36.749 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-11T06:51:36.749 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:36.763 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T06:51:36.881 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:36.881 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-11T06:51:36.881 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-11T06:51:36.881 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-11T06:51:36.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.881 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4868.0:26 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 140250278007648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:36.881 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:51:36.882 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 66.149718707999995, 2024-09-11T06:51:36.882 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034645140000000001, 2024-09-11T06:51:36.882 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.882 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.882 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.882 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.882 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4868", 2024-09-11T06:51:36.882 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3285024586", 2024-09-11T06:51:36.882 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:51:36.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.882 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.882 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.882 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:51:36.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.883 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.883 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729633+0000", 2024-09-11T06:51:36.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.883 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.883 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729634+0000", 2024-09-11T06:51:36.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0470000000000001e-06 2024-09-11T06:51:36.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.883 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.883 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729640+0000", 2024-09-11T06:51:36.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8860000000000001e-06 2024-09-11T06:51:36.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.884 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.884 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729654+0000", 2024-09-11T06:51:36.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4636e-05 2024-09-11T06:51:36.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.884 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.884 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.729664+0000", 2024-09-11T06:51:36.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.594e-06 2024-09-11T06:51:36.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.884 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.884 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730422+0000", 2024-09-11T06:51:36.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000758648 2024-09-11T06:51:36.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730571+0000", 2024-09-11T06:51:36.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000148932 2024-09-11T06:51:36.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:36.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.730936+0000", 2024-09-11T06:51:36.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036510900000000002 2024-09-11T06:51:36.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732159+0000", 2024-09-11T06:51:36.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001222302 2024-09-11T06:51:36.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732763+0000", 2024-09-11T06:51:36.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00060421599999999998 2024-09-11T06:51:36.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.732916+0000", 2024-09-11T06:51:36.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015264 2024-09-11T06:51:36.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.887 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.887 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.733097+0000", 2024-09-11T06:51:36.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000181504 2024-09-11T06:51:36.887 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.887 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.887 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.887 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5175.0:26 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 139740787512064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:36.887 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:51:36.887 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 65.987034983000001, 2024-09-11T06:51:36.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003613733, 2024-09-11T06:51:36.887 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.887 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.887 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.888 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.888 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5175", 2024-09-11T06:51:36.888 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3826853399", 2024-09-11T06:51:36.888 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:51:36.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.888 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.888 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.888 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:51:36.888 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.888 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.889 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892316+0000", 2024-09-11T06:51:36.889 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.889 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.889 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892318+0000", 2024-09-11T06:51:36.889 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.623e-06 2024-09-11T06:51:36.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.889 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.889 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892325+0000", 2024-09-11T06:51:36.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1600000000000001e-06 2024-09-11T06:51:36.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892346+0000", 2024-09-11T06:51:36.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1246e-05 2024-09-11T06:51:36.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892362+0000", 2024-09-11T06:51:36.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5409000000000001e-05 2024-09-11T06:51:36.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.892883+0000", 2024-09-11T06:51:36.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00052087599999999998 2024-09-11T06:51:36.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893123+0000", 2024-09-11T06:51:36.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000240769 2024-09-11T06:51:36.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:36.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893526+0000", 2024-09-11T06:51:36.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040300200000000002 2024-09-11T06:51:36.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894820+0000", 2024-09-11T06:51:36.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012934159999999999 2024-09-11T06:51:36.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895608+0000", 2024-09-11T06:51:36.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00078788100000000004 2024-09-11T06:51:36.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895724+0000", 2024-09-11T06:51:36.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011595999999999999 2024-09-11T06:51:36.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.893 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.893 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.895930+0000", 2024-09-11T06:51:36.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000206391 2024-09-11T06:51:36.893 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.893 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.893 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.894 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.894 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4744.0:133 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-11T06:51:36.894 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:51:36.894 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 65.986236966000007, 2024-09-11T06:51:36.894 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035017360000000001, 2024-09-11T06:51:36.894 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.894 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.894 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:36.894 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.894 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4744", 2024-09-11T06:51:36.894 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1913872305", 2024-09-11T06:51:36.894 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 133 2024-09-11T06:51:36.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.894 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:51:36.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893114+0000", 2024-09-11T06:51:36.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893116+0000", 2024-09-11T06:51:36.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1629999999999999e-06 2024-09-11T06:51:36.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.896 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.896 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893128+0000", 2024-09-11T06:51:36.896 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.11e-05 2024-09-11T06:51:36.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.896 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.896 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893156+0000", 2024-09-11T06:51:36.896 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8266e-05 2024-09-11T06:51:36.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.896 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.896 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.893171+0000", 2024-09-11T06:51:36.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5426000000000001e-05 2024-09-11T06:51:36.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894561+0000", 2024-09-11T06:51:36.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013899959999999999 2024-09-11T06:51:36.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:36.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.894781+0000", 2024-09-11T06:51:36.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021987100000000001 2024-09-11T06:51:36.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.898 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896349+0000", 2024-09-11T06:51:36.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015681779999999999 2024-09-11T06:51:36.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.898 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.898 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896458+0000", 2024-09-11T06:51:36.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000108187 2024-09-11T06:51:36.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.898 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.898 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:30.896616+0000", 2024-09-11T06:51:36.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015854899999999999 2024-09-11T06:51:36.898 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.898 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.899 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.899 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5553.0:24 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 139650593197904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:36.899 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.566567+0000", 2024-09-11T06:51:36.899 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 48.312784458000003, 2024-09-11T06:51:36.899 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034350790000000002, 2024-09-11T06:51:36.899 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.899 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.899 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.899 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.900 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5553", 2024-09-11T06:51:36.900 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3590511604", 2024-09-11T06:51:36.900 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 24 2024-09-11T06:51:36.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.900 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566567+0000", 2024-09-11T06:51:36.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566567+0000", 2024-09-11T06:51:36.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566569+0000", 2024-09-11T06:51:36.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.973e-06 2024-09-11T06:51:36.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566578+0000", 2024-09-11T06:51:36.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8170000000000005e-06 2024-09-11T06:51:36.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.902 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566602+0000", 2024-09-11T06:51:36.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4596e-05 2024-09-11T06:51:36.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.902 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.902 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.566619+0000", 2024-09-11T06:51:36.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7099999999999999e-05 2024-09-11T06:51:36.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.902 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.902 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.567321+0000", 2024-09-11T06:51:36.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070168200000000002 2024-09-11T06:51:36.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.903 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.903 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.567454+0000", 2024-09-11T06:51:36.903 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013286400000000001 2024-09-11T06:51:36.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.903 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:36.903 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.567674+0000", 2024-09-11T06:51:36.903 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000219679 2024-09-11T06:51:36.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.903 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.903 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.569014+0000", 2024-09-11T06:51:36.903 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013404129999999999 2024-09-11T06:51:36.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.904 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.904 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.569634+0000", 2024-09-11T06:51:36.904 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00062029200000000004 2024-09-11T06:51:36.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.904 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.904 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.569805+0000", 2024-09-11T06:51:36.904 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000170366 2024-09-11T06:51:36.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.904 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.904 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.570002+0000", 2024-09-11T06:51:36.904 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019729700000000001 2024-09-11T06:51:36.905 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.905 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.905 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.905 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:82 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch reconnect cookie 139888024456048 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:36.905 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.690185+0000", 2024-09-11T06:51:36.905 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 48.189166456000002, 2024-09-11T06:51:36.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0045884799999999998, 2024-09-11T06:51:36.905 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.905 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.905 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:36.905 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.905 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:36.905 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:36.906 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 82 2024-09-11T06:51:36.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.906 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.906 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690185+0000", 2024-09-11T06:51:36.906 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.906 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690185+0000", 2024-09-11T06:51:36.906 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.907 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690186+0000", 2024-09-11T06:51:36.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8400000000000002e-07 2024-09-11T06:51:36.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.907 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.907 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690190+0000", 2024-09-11T06:51:36.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5499999999999996e-06 2024-09-11T06:51:36.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.907 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.907 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690203+0000", 2024-09-11T06:51:36.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2228999999999999e-05 2024-09-11T06:51:36.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.907 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690228+0000", 2024-09-11T06:51:36.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5587000000000001e-05 2024-09-11T06:51:36.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.691298+0000", 2024-09-11T06:51:36.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001070218 2024-09-11T06:51:36.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.691511+0000", 2024-09-11T06:51:36.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021301299999999999 2024-09-11T06:51:36.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.909 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.694773+0000", 2024-09-11T06:51:36.909 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003261899 2024-09-11T06:51:36.909 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.909 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.909 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.909 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5544.0:84 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify cookie 139887823033168] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:36.909 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:51:36.909 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 48.189074679999997, 2024-09-11T06:51:36.909 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035622620000000001, 2024-09-11T06:51:36.909 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.910 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.910 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:36.910 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.910 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5544", 2024-09-11T06:51:36.910 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:36.910 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 84 2024-09-11T06:51:36.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.910 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.910 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.910 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:51:36.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.911 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.911 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:51:36.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.911 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.911 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690277+0000", 2024-09-11T06:51:36.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5300000000000003e-07 2024-09-11T06:51:36.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.911 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.911 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690281+0000", 2024-09-11T06:51:36.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9330000000000003e-06 2024-09-11T06:51:36.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.912 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.912 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.912 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690291+0000", 2024-09-11T06:51:36.912 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7359999999999996e-06 2024-09-11T06:51:36.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.912 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.912 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.912 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.690298+0000", 2024-09-11T06:51:36.912 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3699999999999997e-06 2024-09-11T06:51:36.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.912 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.912 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.913 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693380+0000", 2024-09-11T06:51:36.913 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0030811060000000001 2024-09-11T06:51:36.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.913 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693599+0000", 2024-09-11T06:51:36.913 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021947199999999999 2024-09-11T06:51:36.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.913 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.693839+0000", 2024-09-11T06:51:36.913 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023989200000000001 2024-09-11T06:51:36.913 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.913 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.914 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.914 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.914 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5098.0:25 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 140617967560800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:36.914 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:50:48.697727+0000", 2024-09-11T06:51:36.914 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 48.181624104000001, 2024-09-11T06:51:36.914 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035341330000000001, 2024-09-11T06:51:36.914 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.914 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.914 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.914 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.914 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5098", 2024-09-11T06:51:36.914 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/844541331", 2024-09-11T06:51:36.914 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 25 2024-09-11T06:51:36.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.915 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.915 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.915 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697727+0000", 2024-09-11T06:51:36.915 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.915 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.915 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.915 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697727+0000", 2024-09-11T06:51:36.915 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.915 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.915 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.915 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697729+0000", 2024-09-11T06:51:36.916 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.33e-06 2024-09-11T06:51:36.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.916 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.916 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.916 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697734+0000", 2024-09-11T06:51:36.916 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6590000000000004e-06 2024-09-11T06:51:36.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.916 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.916 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.916 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697749+0000", 2024-09-11T06:51:36.916 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4867e-05 2024-09-11T06:51:36.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.916 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.917 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.917 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.697759+0000", 2024-09-11T06:51:36.917 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0073e-05 2024-09-11T06:51:36.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.917 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.917 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.917 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.698715+0000", 2024-09-11T06:51:36.917 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00095545799999999996 2024-09-11T06:51:36.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.917 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.917 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.917 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.698850+0000", 2024-09-11T06:51:36.917 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000135814 2024-09-11T06:51:36.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.918 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:36.918 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.699085+0000", 2024-09-11T06:51:36.918 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023507199999999999 2024-09-11T06:51:36.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.918 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.918 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.700258+0000", 2024-09-11T06:51:36.918 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011726600000000001 2024-09-11T06:51:36.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.918 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.918 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.701061+0000", 2024-09-11T06:51:36.918 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00080256100000000003 2024-09-11T06:51:36.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.919 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.919 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.919 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.701144+0000", 2024-09-11T06:51:36.919 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3188000000000004e-05 2024-09-11T06:51:36.919 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.919 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.919 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.919 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:50:48.701261+0000", 2024-09-11T06:51:36.919 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000117451 2024-09-11T06:51:36.919 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.919 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.919 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.919 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.919 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.920 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.6105.0:12 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b,call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b,call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b,call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-11T06:51:36.920 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:20.630296+0000", 2024-09-11T06:51:36.920 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 16.249055178999999, 2024-09-11T06:51:36.920 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003874653, 2024-09-11T06:51:36.920 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.920 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.920 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:36.920 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.920 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.6105", 2024-09-11T06:51:36.920 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/722227396", 2024-09-11T06:51:36.920 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 12 2024-09-11T06:51:36.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.921 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.921 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.921 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.921 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.630296+0000", 2024-09-11T06:51:36.921 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.921 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.921 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.921 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.630296+0000", 2024-09-11T06:51:36.921 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.921 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.921 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.922 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.630298+0000", 2024-09-11T06:51:36.922 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2500000000000001e-06 2024-09-11T06:51:36.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.922 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.922 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.922 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.630317+0000", 2024-09-11T06:51:36.922 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9043e-05 2024-09-11T06:51:36.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.922 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.922 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.922 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.630349+0000", 2024-09-11T06:51:36.922 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1890000000000001e-05 2024-09-11T06:51:36.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.922 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.923 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.923 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.630368+0000", 2024-09-11T06:51:36.923 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8533e-05 2024-09-11T06:51:36.923 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.923 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.923 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.923 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.630500+0000", 2024-09-11T06:51:36.923 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013175900000000001 2024-09-11T06:51:36.923 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.923 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.923 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.923 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.630806+0000", 2024-09-11T06:51:36.923 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030631700000000002 2024-09-11T06:51:36.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.924 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.924 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.924 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.634171+0000", 2024-09-11T06:51:36.924 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033648609999999998 2024-09-11T06:51:36.924 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.924 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.924 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.924 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.924 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.6105.0:20 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.metadata_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-11T06:51:36.924 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:20.648867+0000", 2024-09-11T06:51:36.924 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 16.230484167, 2024-09-11T06:51:36.924 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035862189999999999, 2024-09-11T06:51:36.925 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.925 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.925 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.925 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.925 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.6105", 2024-09-11T06:51:36.925 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/722227396", 2024-09-11T06:51:36.925 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 20 2024-09-11T06:51:36.925 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.925 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.925 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.925 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.925 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.648867+0000", 2024-09-11T06:51:36.925 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.925 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.925 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.926 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.926 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.648867+0000", 2024-09-11T06:51:36.926 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.926 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.926 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.926 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.648868+0000", 2024-09-11T06:51:36.926 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5000000000000001e-07 2024-09-11T06:51:36.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.926 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.926 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.926 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.648873+0000", 2024-09-11T06:51:36.926 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2970000000000003e-06 2024-09-11T06:51:36.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.927 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.927 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.927 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.648889+0000", 2024-09-11T06:51:36.927 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5549999999999999e-05 2024-09-11T06:51:36.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.927 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.927 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.927 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.648899+0000", 2024-09-11T06:51:36.927 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0516000000000001e-05 2024-09-11T06:51:36.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.927 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.927 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.927 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.649514+0000", 2024-09-11T06:51:36.927 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00061461800000000004 2024-09-11T06:51:36.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.928 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.928 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.649816+0000", 2024-09-11T06:51:36.928 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030191700000000002 2024-09-11T06:51:36.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.928 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:36.928 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.650295+0000", 2024-09-11T06:51:36.928 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00047892799999999999 2024-09-11T06:51:36.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.928 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.929 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.651725+0000", 2024-09-11T06:51:36.929 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0014304630000000001 2024-09-11T06:51:36.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.929 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.929 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.652187+0000", 2024-09-11T06:51:36.929 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00046161299999999999 2024-09-11T06:51:36.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.929 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.929 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.652279+0000", 2024-09-11T06:51:36.929 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2025000000000001e-05 2024-09-11T06:51:36.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.930 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.930 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.652453+0000", 2024-09-11T06:51:36.930 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017434199999999999 2024-09-11T06:51:36.930 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.930 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.930 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.930 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.6105.0:21 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-11T06:51:36.930 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:20.649631+0000", 2024-09-11T06:51:36.930 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 16.229719824, 2024-09-11T06:51:36.930 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034058339999999999, 2024-09-11T06:51:36.931 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.931 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.931 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:36.931 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.931 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.6105", 2024-09-11T06:51:36.931 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/722227396", 2024-09-11T06:51:36.931 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 21 2024-09-11T06:51:36.931 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.931 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.931 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.931 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.931 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.649631+0000", 2024-09-11T06:51:36.931 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.931 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.932 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.932 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.649631+0000", 2024-09-11T06:51:36.932 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.932 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.932 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.649634+0000", 2024-09-11T06:51:36.932 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.04e-06 2024-09-11T06:51:36.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.932 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.932 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.649644+0000", 2024-09-11T06:51:36.932 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.061e-05 2024-09-11T06:51:36.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.933 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.933 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.649671+0000", 2024-09-11T06:51:36.933 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6679e-05 2024-09-11T06:51:36.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.933 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.933 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.649690+0000", 2024-09-11T06:51:36.933 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9573999999999999e-05 2024-09-11T06:51:36.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.933 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.933 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.651104+0000", 2024-09-11T06:51:36.934 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001413739 2024-09-11T06:51:36.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.934 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:36.934 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.651336+0000", 2024-09-11T06:51:36.934 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023153799999999999 2024-09-11T06:51:36.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.934 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.934 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.652555+0000", 2024-09-11T06:51:36.934 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001219765 2024-09-11T06:51:36.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.934 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.935 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.652705+0000", 2024-09-11T06:51:36.935 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000149445 2024-09-11T06:51:36.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.935 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.935 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.935 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.653037+0000", 2024-09-11T06:51:36.935 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033244399999999999 2024-09-11T06:51:36.935 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.935 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.935 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.935 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.935 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.6105.0:33 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [watch unwatch cookie 140610048726272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-11T06:51:36.935 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:20.671124+0000", 2024-09-11T06:51:36.936 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 16.208227776000001, 2024-09-11T06:51:36.936 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035064290000000001, 2024-09-11T06:51:36.936 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.936 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.936 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.936 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.936 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.6105", 2024-09-11T06:51:36.936 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/722227396", 2024-09-11T06:51:36.936 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 33 2024-09-11T06:51:36.936 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.936 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.936 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.936 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.671124+0000", 2024-09-11T06:51:36.937 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.937 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.937 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.937 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.671124+0000", 2024-09-11T06:51:36.937 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.937 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.937 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.937 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.671124+0000", 2024-09-11T06:51:36.937 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7600000000000006e-07 2024-09-11T06:51:36.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.937 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.937 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.938 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.671129+0000", 2024-09-11T06:51:36.938 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7330000000000003e-06 2024-09-11T06:51:36.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.938 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.938 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.671145+0000", 2024-09-11T06:51:36.938 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5367e-05 2024-09-11T06:51:36.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.938 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.938 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.671154+0000", 2024-09-11T06:51:36.938 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.363e-06 2024-09-11T06:51:36.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.939 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.939 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.671797+0000", 2024-09-11T06:51:36.939 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00064351000000000002 2024-09-11T06:51:36.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.939 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.939 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.939 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.671943+0000", 2024-09-11T06:51:36.939 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000145699 2024-09-11T06:51:36.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.939 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.939 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:36.939 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.672240+0000", 2024-09-11T06:51:36.939 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00029651100000000001 2024-09-11T06:51:36.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.939 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.940 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.940 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.673194+0000", 2024-09-11T06:51:36.940 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00095448699999999996 2024-09-11T06:51:36.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.940 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.940 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.674221+0000", 2024-09-11T06:51:36.940 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001026714 2024-09-11T06:51:36.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.940 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.940 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.674327+0000", 2024-09-11T06:51:36.940 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000105732 2024-09-11T06:51:36.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.941 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.941 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.941 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.674630+0000", 2024-09-11T06:51:36.941 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030333699999999999 2024-09-11T06:51:36.941 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.941 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.941 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.941 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.941 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.941 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5542.0:3 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-11T06:51:36.941 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:20.994530+0000", 2024-09-11T06:51:36.941 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.884821327999999, 2024-09-11T06:51:36.941 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.99603794099999998, 2024-09-11T06:51:36.941 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.942 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.942 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:36.942 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.942 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5542", 2024-09-11T06:51:36.942 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/1546942093", 2024-09-11T06:51:36.942 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 3 2024-09-11T06:51:36.942 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.942 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.942 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.942 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.942 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.994530+0000", 2024-09-11T06:51:36.942 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.942 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.942 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.942 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.943 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.994530+0000", 2024-09-11T06:51:36.943 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.943 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.943 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.943 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.994532+0000", 2024-09-11T06:51:36.943 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9740000000000001e-06 2024-09-11T06:51:36.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.943 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.943 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.943 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.994543+0000", 2024-09-11T06:51:36.943 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1066999999999999e-05 2024-09-11T06:51:36.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.943 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.944 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.944 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.994558+0000", 2024-09-11T06:51:36.944 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4987999999999999e-05 2024-09-11T06:51:36.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.944 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.944 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.994572+0000", 2024-09-11T06:51:36.944 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3974e-05 2024-09-11T06:51:36.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.944 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.944 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.994659+0000", 2024-09-11T06:51:36.944 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7491000000000006e-05 2024-09-11T06:51:36.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.945 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.945 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-11T06:51:36.945 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:20.994675+0000", 2024-09-11T06:51:36.945 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.506e-05 2024-09-11T06:51:36.945 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.945 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.945 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.945 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:21.989532+0000", 2024-09-11T06:51:36.945 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.99485760400000001 2024-09-11T06:51:36.945 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.945 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.945 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.945 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:21.989833+0000", 2024-09-11T06:51:36.945 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00030123699999999999 2024-09-11T06:51:36.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.946 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.946 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:21.990568+0000", 2024-09-11T06:51:36.946 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00073454599999999998 2024-09-11T06:51:36.946 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.946 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.946 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.946 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5660.0:26 2.1 2:9bfa7d91:::rbd_header.1314f0a2bfa2:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e107)", 2024-09-11T06:51:36.946 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:30.104530+0000", 2024-09-11T06:51:36.946 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.7748214640000004, 2024-09-11T06:51:36.946 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034986409999999998, 2024-09-11T06:51:36.946 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.947 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.947 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.947 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.947 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5660", 2024-09-11T06:51:36.947 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3214439167", 2024-09-11T06:51:36.947 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 26 2024-09-11T06:51:36.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.947 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.947 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.947 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.947 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:30.104530+0000", 2024-09-11T06:51:36.947 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.947 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.948 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.948 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:30.104530+0000", 2024-09-11T06:51:36.948 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.948 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.948 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:30.104532+0000", 2024-09-11T06:51:36.948 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7990000000000001e-06 2024-09-11T06:51:36.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.948 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.948 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:30.104539+0000", 2024-09-11T06:51:36.948 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4109999999999999e-06 2024-09-11T06:51:36.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.949 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.949 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:30.104553+0000", 2024-09-11T06:51:36.949 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4228e-05 2024-09-11T06:51:36.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.949 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.949 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:30.104562+0000", 2024-09-11T06:51:36.949 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7830000000000004e-06 2024-09-11T06:51:36.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.949 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.949 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:30.105488+0000", 2024-09-11T06:51:36.949 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00092615299999999998 2024-09-11T06:51:36.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.950 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.950 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:30.105588+0000", 2024-09-11T06:51:36.950 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010027299999999999 2024-09-11T06:51:36.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.950 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:36.950 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:30.105962+0000", 2024-09-11T06:51:36.950 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00037397299999999999 2024-09-11T06:51:36.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.950 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.950 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:30.107613+0000", 2024-09-11T06:51:36.950 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0016503119999999999 2024-09-11T06:51:36.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.951 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.951 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:30.107903+0000", 2024-09-11T06:51:36.951 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00028990999999999999 2024-09-11T06:51:36.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.951 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.951 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:30.107975+0000", 2024-09-11T06:51:36.951 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2014999999999994e-05 2024-09-11T06:51:36.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.951 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.951 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:30.108028+0000", 2024-09-11T06:51:36.951 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3783999999999999e-05 2024-09-11T06:51:36.952 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.952 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.952 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.952 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.952 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.952 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5723.0:83 2.1 2:86c3c9db:ns::rbd_data.11adc335555f.0000000000000002:head [delete] snapc 4=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e111)", 2024-09-11T06:51:36.952 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:34.490836+0000", 2024-09-11T06:51:36.952 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 2.3885151119999999, 2024-09-11T06:51:36.952 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.015120638, 2024-09-11T06:51:36.952 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.952 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.952 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.952 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.953 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5723", 2024-09-11T06:51:36.953 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3725019196", 2024-09-11T06:51:36.953 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 83 2024-09-11T06:51:36.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.953 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.953 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.953 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.490836+0000", 2024-09-11T06:51:36.953 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.953 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.953 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.490836+0000", 2024-09-11T06:51:36.953 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.954 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.954 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.954 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.490839+0000", 2024-09-11T06:51:36.954 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.43e-06 2024-09-11T06:51:36.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.954 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.954 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.954 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.490848+0000", 2024-09-11T06:51:36.954 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8670000000000006e-06 2024-09-11T06:51:36.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.954 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.954 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.954 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.490870+0000", 2024-09-11T06:51:36.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1466000000000001e-05 2024-09-11T06:51:36.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.955 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.955 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.490886+0000", 2024-09-11T06:51:36.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5926999999999998e-05 2024-09-11T06:51:36.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.955 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.955 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.490988+0000", 2024-09-11T06:51:36.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000101754 2024-09-11T06:51:36.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.955 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.956 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.491161+0000", 2024-09-11T06:51:36.956 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000173003 2024-09-11T06:51:36.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.956 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:36.956 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.491283+0000", 2024-09-11T06:51:36.956 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000122559 2024-09-11T06:51:36.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.956 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.956 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.505825+0000", 2024-09-11T06:51:36.957 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.014541909 2024-09-11T06:51:36.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.957 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.957 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.505869+0000", 2024-09-11T06:51:36.957 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3667000000000001e-05 2024-09-11T06:51:36.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.957 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.505915+0000", 2024-09-11T06:51:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6468000000000003e-05 2024-09-11T06:51:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.505957+0000", 2024-09-11T06:51:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1588000000000002e-05 2024-09-11T06:51:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5723.0:85 2.1 2:8c2f6822:ns::rbd_data.11adc335555f.0000000000000004:head [delete] snapc 4=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e111)", 2024-09-11T06:51:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:34.490928+0000", 2024-09-11T06:51:36.958 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 2.3884233990000001, 2024-09-11T06:51:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.015455247, 2024-09-11T06:51:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5723", 2024-09-11T06:51:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3725019196", 2024-09-11T06:51:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 85 2024-09-11T06:51:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.490928+0000", 2024-09-11T06:51:36.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.960 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.960 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.490928+0000", 2024-09-11T06:51:36.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.960 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.960 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.490929+0000", 2024-09-11T06:51:36.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9700000000000005e-07 2024-09-11T06:51:36.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.961 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.961 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.490933+0000", 2024-09-11T06:51:36.961 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.296e-06 2024-09-11T06:51:36.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.961 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.961 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.961 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.490948+0000", 2024-09-11T06:51:36.961 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4960000000000001e-05 2024-09-11T06:51:36.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.961 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.961 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.961 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.490957+0000", 2024-09-11T06:51:36.961 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1030000000000008e-06 2024-09-11T06:51:36.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.492957+0000", 2024-09-11T06:51:36.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0019997050000000001 2024-09-11T06:51:36.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.493367+0000", 2024-09-11T06:51:36.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000410073 2024-09-11T06:51:36.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:36.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.493639+0000", 2024-09-11T06:51:36.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00027174300000000002 2024-09-11T06:51:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.506078+0000", 2024-09-11T06:51:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.012439323 2024-09-11T06:51:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.506117+0000", 2024-09-11T06:51:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8578999999999998e-05 2024-09-11T06:51:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.506265+0000", 2024-09-11T06:51:36.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014865599999999999 2024-09-11T06:51:36.964 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.964 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.964 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.964 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.506383+0000", 2024-09-11T06:51:36.964 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011791200000000001 2024-09-11T06:51:36.964 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.964 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.964 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.964 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.964 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.964 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5723.0:86 2.1 2:92890cf1:ns::rbd_data.11adc335555f.0000000000000005:head [delete] snapc 4=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e111)", 2024-09-11T06:51:36.964 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:34.490972+0000", 2024-09-11T06:51:36.964 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 2.3883792169999998, 2024-09-11T06:51:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.015532865999999999, 2024-09-11T06:51:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5723", 2024-09-11T06:51:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3725019196", 2024-09-11T06:51:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 86 2024-09-11T06:51:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.490972+0000", 2024-09-11T06:51:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.490972+0000", 2024-09-11T06:51:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.490973+0000", 2024-09-11T06:51:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9999999999999997e-07 2024-09-11T06:51:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.490976+0000", 2024-09-11T06:51:36.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.304e-06 2024-09-11T06:51:36.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.967 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.967 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.490987+0000", 2024-09-11T06:51:36.967 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0443e-05 2024-09-11T06:51:36.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.967 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.967 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.490994+0000", 2024-09-11T06:51:36.967 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7999999999999999e-06 2024-09-11T06:51:36.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.967 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.967 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.495972+0000", 2024-09-11T06:51:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0049777900000000002 2024-09-11T06:51:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.496332+0000", 2024-09-11T06:51:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00035952999999999999 2024-09-11T06:51:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.496675+0000", 2024-09-11T06:51:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00034299999999999999 2024-09-11T06:51:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.968 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.506405+0000", 2024-09-11T06:51:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0097308110000000007 2024-09-11T06:51:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.506438+0000", 2024-09-11T06:51:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2835e-05 2024-09-11T06:51:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.506476+0000", 2024-09-11T06:51:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8096000000000003e-05 2024-09-11T06:51:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.970 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.970 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.506505+0000", 2024-09-11T06:51:36.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8557000000000001e-05 2024-09-11T06:51:36.970 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.970 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.970 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.970 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.5723.0:92 2.1 2:8616a778:ns::rbd_data.11adc335555f.000000000000000b:head [delete] snapc 4=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e111)", 2024-09-11T06:51:36.970 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:34.491005+0000", 2024-09-11T06:51:36.970 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 2.3883461329999998, 2024-09-11T06:51:36.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.015904232000000001, 2024-09-11T06:51:36.970 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.970 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-11T06:51:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.5723", 2024-09-11T06:51:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.176:0/3725019196", 2024-09-11T06:51:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 92 2024-09-11T06:51:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.491005+0000", 2024-09-11T06:51:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.491005+0000", 2024-09-11T06:51:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.491006+0000", 2024-09-11T06:51:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7000000000000004e-07 2024-09-11T06:51:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.491009+0000", 2024-09-11T06:51:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9100000000000001e-06 2024-09-11T06:51:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.491019+0000", 2024-09-11T06:51:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7829999999999993e-06 2024-09-11T06:51:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.491045+0000", 2024-09-11T06:51:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6132999999999999e-05 2024-09-11T06:51:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.501080+0000", 2024-09-11T06:51:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.010035162 2024-09-11T06:51:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.501512+0000", 2024-09-11T06:51:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00043243600000000002 2024-09-11T06:51:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.501877+0000", 2024-09-11T06:51:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00036459300000000002 2024-09-11T06:51:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.506608+0000", 2024-09-11T06:51:36.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.004730911 2024-09-11T06:51:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-11T06:51:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.506648+0000", 2024-09-11T06:51:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0114000000000003e-05 2024-09-11T06:51:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-11T06:51:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.506794+0000", 2024-09-11T06:51:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014576500000000001 2024-09-11T06:51:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.506909+0000", 2024-09-11T06:51:36.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000115655 2024-09-11T06:51:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop_reply(client.5723.0:83 2.1 e111/102 ondisk, result = 0)", 2024-09-11T06:51:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:34.499253+0000", 2024-09-11T06:51:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 2.3800979209999999, 2024-09-11T06:51:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0067099830000000001, 2024-09-11T06:51:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.976 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.499253+0000", 2024-09-11T06:51:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.499253+0000", 2024-09-11T06:51:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.499255+0000", 2024-09-11T06:51:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8700000000000001e-06 2024-09-11T06:51:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.499266+0000", 2024-09-11T06:51:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.029e-05 2024-09-11T06:51:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.499310+0000", 2024-09-11T06:51:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4443000000000001e-05 2024-09-11T06:51:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.499331+0000", 2024-09-11T06:51:36.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0829e-05 2024-09-11T06:51:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.505855+0000", 2024-09-11T06:51:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.006523732 2024-09-11T06:51:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.505863+0000", 2024-09-11T06:51:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7169999999999997e-06 2024-09-11T06:51:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.979 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.505963+0000", 2024-09-11T06:51:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000100102 2024-09-11T06:51:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop_reply(client.5723.0:85 2.1 e111/102 ondisk, result = 0)", 2024-09-11T06:51:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:34.501973+0000", 2024-09-11T06:51:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 2.3773779299999998, 2024-09-11T06:51:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0044368910000000001, 2024-09-11T06:51:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.980 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.501973+0000", 2024-09-11T06:51:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.501973+0000", 2024-09-11T06:51:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.981 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.501975+0000", 2024-09-11T06:51:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4759999999999999e-06 2024-09-11T06:51:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.501983+0000", 2024-09-11T06:51:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3629999999999994e-06 2024-09-11T06:51:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.502003+0000", 2024-09-11T06:51:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0134000000000001e-05 2024-09-11T06:51:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.982 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.502016+0000", 2024-09-11T06:51:36.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.222e-05 2024-09-11T06:51:36.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.983 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.506038+0000", 2024-09-11T06:51:36.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040223180000000004 2024-09-11T06:51:36.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.983 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.506061+0000", 2024-09-11T06:51:36.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2840000000000002e-05 2024-09-11T06:51:36.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.506410+0000", 2024-09-11T06:51:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00034954000000000002 2024-09-11T06:51:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop_reply(client.5723.0:86 2.1 e111/102 ondisk, result = 0)", 2024-09-11T06:51:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-11T06:51:34.502267+0000", 2024-09-11T06:51:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 2.3770844499999999, 2024-09-11T06:51:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0042433719999999996, 2024-09-11T06:51:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-11T06:51:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-11T06:51:36.984 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-11T06:51:36.985 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-11T06:51:36.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-11T06:51:36.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.502267+0000", 2024-09-11T06:51:36.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-11T06:51:36.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.502267+0000", 2024-09-11T06:51:36.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-11T06:51:36.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-11T06:51:36.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.502268+0000", 2024-09-11T06:51:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.66e-06 2024-09-11T06:51:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-11T06:51:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.502278+0000", 2024-09-11T06:51:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4429999999999992e-06 2024-09-11T06:51:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-11T06:51:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.502310+0000", 2024-09-11T06:51:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2076999999999998e-05 2024-09-11T06:51:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-11T06:51:36.986 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.502328+0000", 2024-09-11T06:51:36.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8136000000000001e-05 2024-09-11T06:51:36.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.987 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-11T06:51:36.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.506427+0000", 2024-09-11T06:51:36.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040987569999999997 2024-09-11T06:51:36.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.987 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-11T06:51:36.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.506432+0000", 2024-09-11T06:51:36.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4979999999999997e-06 2024-09-11T06:51:36.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T06:51:36.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T06:51:36.987 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-11T06:51:36.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-11T06:51:34.506510+0000", 2024-09-11T06:51:36.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7800999999999996e-05 2024-09-11T06:51:36.988 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.988 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.988 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.988 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T06:51:36.988 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T06:51:36.988 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:36.989 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T06:51:36.990 INFO:teuthology.orchestra.run.smithi071.stderr:marked in osd.1. 2024-09-11T06:51:36.990 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-11T06:51:37.122 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:37.122 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:37.122 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:51:37.122 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:37.137 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T06:51:37.254 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:37.254 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:37.254 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:51:37.255 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:51:37.255 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:37.268 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T06:51:37.395 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:37.395 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:51:37.395 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:51:37.396 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:51:37.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.396 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:37.396 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:51:37.396 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 150.94971400700001, 2024-09-11T06:51:37.396 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014944941, 2024-09-11T06:51:37.397 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.397 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.397 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:37.397 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:37.398 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:37.399 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:37.399 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 111 2024-09-11T06:51:37.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.399 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.399 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.399 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.399 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:51:37.399 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.399 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.399 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.399 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440746+0000", 2024-09-11T06:51:37.399 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.400 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.400 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.400 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440748+0000", 2024-09-11T06:51:37.400 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:51:37.400 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.400 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.400 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.400 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440760+0000", 2024-09-11T06:51:37.400 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1897e-05 2024-09-11T06:51:37.400 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.400 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.400 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.400 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440782+0000", 2024-09-11T06:51:37.401 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2552999999999999e-05 2024-09-11T06:51:37.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.401 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.401 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440793+0000", 2024-09-11T06:51:37.401 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0643e-05 2024-09-11T06:51:37.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.401 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.401 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449755+0000", 2024-09-11T06:51:37.401 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0089622620000000004 2024-09-11T06:51:37.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.402 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.402 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.449918+0000", 2024-09-11T06:51:37.402 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163086 2024-09-11T06:51:37.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.402 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:37.402 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.450150+0000", 2024-09-11T06:51:37.402 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023204500000000001 2024-09-11T06:51:37.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.402 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:37.402 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454533+0000", 2024-09-11T06:51:37.402 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0043831349999999998 2024-09-11T06:51:37.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.403 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.403 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:37.403 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455511+0000", 2024-09-11T06:51:37.403 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0009776050000000001 2024-09-11T06:51:37.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.403 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.403 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:37.403 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455597+0000", 2024-09-11T06:51:37.403 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6446000000000005e-05 2024-09-11T06:51:37.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.403 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.403 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.403 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455691+0000", 2024-09-11T06:51:37.404 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.3832000000000006e-05 2024-09-11T06:51:37.404 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.404 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.404 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.404 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.404 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.404 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:37.404 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:51:37.404 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 150.94964424400001, 2024-09-11T06:51:37.404 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018885411000000001, 2024-09-11T06:51:37.404 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.404 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.404 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:37.404 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:37.405 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:37.405 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:37.405 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 112 2024-09-11T06:51:37.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.405 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.405 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.405 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.405 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:51:37.405 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.406 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.406 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.406 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.406 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440816+0000", 2024-09-11T06:51:37.406 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.406 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.406 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.406 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.406 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.440817+0000", 2024-09-11T06:51:37.406 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4300000000000002e-07 2024-09-11T06:51:37.406 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.407 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.407 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.407 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441003+0000", 2024-09-11T06:51:37.407 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018661200000000001 2024-09-11T06:51:37.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.407 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.407 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.407 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441027+0000", 2024-09-11T06:51:37.407 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3920000000000001e-05 2024-09-11T06:51:37.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.407 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.407 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.408 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441062+0000", 2024-09-11T06:51:37.408 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4663000000000003e-05 2024-09-11T06:51:37.408 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.408 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.408 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.408 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.451981+0000", 2024-09-11T06:51:37.408 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010918926000000001 2024-09-11T06:51:37.409 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.409 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.409 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.409 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452111+0000", 2024-09-11T06:51:37.409 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129529 2024-09-11T06:51:37.409 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.409 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.409 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:37.409 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.452311+0000", 2024-09-11T06:51:37.409 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020013799999999999 2024-09-11T06:51:37.409 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.409 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.410 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:37.410 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455809+0000", 2024-09-11T06:51:37.410 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003498443 2024-09-11T06:51:37.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.410 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.410 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:37.410 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459566+0000", 2024-09-11T06:51:37.410 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003757271 2024-09-11T06:51:37.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.410 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.410 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:37.410 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459640+0000", 2024-09-11T06:51:37.410 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3949000000000006e-05 2024-09-11T06:51:37.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.411 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.411 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.411 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459701+0000", 2024-09-11T06:51:37.411 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1117000000000005e-05 2024-09-11T06:51:37.411 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.411 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.411 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.411 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.411 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:37.411 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:51:37.411 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 150.94936539299999, 2024-09-11T06:51:37.411 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018760915, 2024-09-11T06:51:37.411 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.412 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.412 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:37.412 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:37.412 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:37.412 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:37.412 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 113 2024-09-11T06:51:37.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.412 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.412 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.412 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.412 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:51:37.412 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.413 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.413 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441095+0000", 2024-09-11T06:51:37.413 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.413 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.413 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441096+0000", 2024-09-11T06:51:37.413 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4130000000000001e-06 2024-09-11T06:51:37.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.413 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.413 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441110+0000", 2024-09-11T06:51:37.414 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3900000000000001e-05 2024-09-11T06:51:37.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.414 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.414 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.414 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441131+0000", 2024-09-11T06:51:37.414 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0636999999999999e-05 2024-09-11T06:51:37.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.414 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.414 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.414 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441147+0000", 2024-09-11T06:51:37.414 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6512999999999999e-05 2024-09-11T06:51:37.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.415 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.415 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.415 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453274+0000", 2024-09-11T06:51:37.415 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012126764999999999 2024-09-11T06:51:37.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.415 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.415 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.415 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453498+0000", 2024-09-11T06:51:37.415 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000223568 2024-09-11T06:51:37.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.415 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.416 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:37.416 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.453872+0000", 2024-09-11T06:51:37.416 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037467099999999998 2024-09-11T06:51:37.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.416 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.416 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:37.416 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458709+0000", 2024-09-11T06:51:37.416 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0048365680000000003 2024-09-11T06:51:37.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.416 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.416 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:37.416 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459716+0000", 2024-09-11T06:51:37.416 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010075240000000001 2024-09-11T06:51:37.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.417 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.417 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:37.417 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459779+0000", 2024-09-11T06:51:37.417 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.2336999999999996e-05 2024-09-11T06:51:37.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.417 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.417 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.417 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459856+0000", 2024-09-11T06:51:37.417 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7019000000000002e-05 2024-09-11T06:51:37.417 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.417 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.417 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.418 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.418 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:37.418 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:51:37.418 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 150.94928865399999, 2024-09-11T06:51:37.418 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.018963114, 2024-09-11T06:51:37.418 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.418 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.418 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:37.418 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:37.418 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:37.418 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:37.418 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 114 2024-09-11T06:51:37.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.419 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.419 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.419 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.419 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:51:37.419 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.419 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.419 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.419 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:51:37.419 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.419 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.419 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.419 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441172+0000", 2024-09-11T06:51:37.420 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9299999999999996e-07 2024-09-11T06:51:37.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.420 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.420 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.420 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441180+0000", 2024-09-11T06:51:37.420 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.7200000000000006e-06 2024-09-11T06:51:37.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.420 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.420 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.420 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441265+0000", 2024-09-11T06:51:37.420 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4753000000000007e-05 2024-09-11T06:51:37.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.420 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.421 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.421 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441288+0000", 2024-09-11T06:51:37.421 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2860000000000001e-05 2024-09-11T06:51:37.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.421 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.421 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.421 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454585+0000", 2024-09-11T06:51:37.421 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013297286 2024-09-11T06:51:37.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.421 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.421 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.421 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454714+0000", 2024-09-11T06:51:37.421 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000128465 2024-09-11T06:51:37.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.422 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.422 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:37.422 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.454909+0000", 2024-09-11T06:51:37.422 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019522499999999999 2024-09-11T06:51:37.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.422 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.422 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:37.422 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459868+0000", 2024-09-11T06:51:37.422 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0049591239999999997 2024-09-11T06:51:37.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.422 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.422 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:37.423 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459988+0000", 2024-09-11T06:51:37.423 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000120532 2024-09-11T06:51:37.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.423 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.423 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:37.423 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460071+0000", 2024-09-11T06:51:37.423 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2256000000000007e-05 2024-09-11T06:51:37.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.423 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.423 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.423 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460135+0000", 2024-09-11T06:51:37.423 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3999999999999997e-05 2024-09-11T06:51:37.424 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.424 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.424 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.424 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.424 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.424 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:37.424 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:51:37.424 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 150.94913102500001, 2024-09-11T06:51:37.424 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.019036849000000002, 2024-09-11T06:51:37.424 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.424 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.424 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:37.424 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:37.424 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:37.425 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:37.425 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 115 2024-09-11T06:51:37.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.425 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.425 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.425 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.425 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:51:37.425 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.425 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.425 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.425 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441329+0000", 2024-09-11T06:51:37.425 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.426 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.426 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.426 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441330+0000", 2024-09-11T06:51:37.426 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1930000000000001e-06 2024-09-11T06:51:37.426 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.426 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.426 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.426 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441544+0000", 2024-09-11T06:51:37.426 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021405799999999999 2024-09-11T06:51:37.426 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.426 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.426 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.427 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441572+0000", 2024-09-11T06:51:37.427 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7317e-05 2024-09-11T06:51:37.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.427 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.427 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.427 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441593+0000", 2024-09-11T06:51:37.427 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1702999999999999e-05 2024-09-11T06:51:37.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.427 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.427 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.427 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.455891+0000", 2024-09-11T06:51:37.427 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01429708 2024-09-11T06:51:37.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.427 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.428 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.428 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456046+0000", 2024-09-11T06:51:37.428 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000155682 2024-09-11T06:51:37.428 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.428 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.428 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:37.428 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.456242+0000", 2024-09-11T06:51:37.428 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000195692 2024-09-11T06:51:37.428 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.428 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.428 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:37.428 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459876+0000", 2024-09-11T06:51:37.428 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036336480000000002 2024-09-11T06:51:37.429 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.429 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.429 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:37.429 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460245+0000", 2024-09-11T06:51:37.429 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036901700000000002 2024-09-11T06:51:37.429 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.429 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.429 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:37.429 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460316+0000", 2024-09-11T06:51:37.429 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1846000000000003e-05 2024-09-11T06:51:37.429 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.429 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.429 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.430 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460366+0000", 2024-09-11T06:51:37.430 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9613000000000001e-05 2024-09-11T06:51:37.430 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.430 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.430 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.430 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.430 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.430 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:37.430 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:51:37.430 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 150.948808945, 2024-09-11T06:51:37.430 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01904432, 2024-09-11T06:51:37.430 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.430 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.430 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:37.430 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:37.431 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:37.431 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:37.431 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 116 2024-09-11T06:51:37.431 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.431 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.431 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.431 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.431 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:51:37.431 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.431 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.431 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.431 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.431 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441651+0000", 2024-09-11T06:51:37.431 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.432 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.432 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.432 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441653+0000", 2024-09-11T06:51:37.432 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.567e-06 2024-09-11T06:51:37.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.432 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.432 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.432 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441665+0000", 2024-09-11T06:51:37.432 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1653e-05 2024-09-11T06:51:37.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.433 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.433 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.433 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441684+0000", 2024-09-11T06:51:37.433 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9490000000000001e-05 2024-09-11T06:51:37.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.433 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.433 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.433 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441699+0000", 2024-09-11T06:51:37.433 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5044e-05 2024-09-11T06:51:37.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.433 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.433 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.433 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457297+0000", 2024-09-11T06:51:37.433 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015598241000000001 2024-09-11T06:51:37.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.434 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.434 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.434 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457577+0000", 2024-09-11T06:51:37.434 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027930799999999998 2024-09-11T06:51:37.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.434 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.434 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:37.434 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.457957+0000", 2024-09-11T06:51:37.434 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00038088 2024-09-11T06:51:37.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.435 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.435 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:37.435 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460432+0000", 2024-09-11T06:51:37.435 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024743349999999998 2024-09-11T06:51:37.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.435 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.435 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:37.435 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460584+0000", 2024-09-11T06:51:37.435 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015232900000000001 2024-09-11T06:51:37.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.435 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.435 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:37.435 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460646+0000", 2024-09-11T06:51:37.435 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1883000000000004e-05 2024-09-11T06:51:37.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.436 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.436 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.436 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.460696+0000", 2024-09-11T06:51:37.436 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9589999999999998e-05 2024-09-11T06:51:37.436 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.436 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.436 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.436 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.437 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:37.437 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:51:37.437 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 150.94873687899999, 2024-09-11T06:51:37.437 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.021834473, 2024-09-11T06:51:37.437 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.437 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.437 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:37.437 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:37.437 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:37.438 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:37.438 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 117 2024-09-11T06:51:37.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.438 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.438 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.438 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.438 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:51:37.438 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.438 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.438 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.438 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441723+0000", 2024-09-11T06:51:37.438 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.439 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.439 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.439 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441725+0000", 2024-09-11T06:51:37.439 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5069999999999999e-06 2024-09-11T06:51:37.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.439 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.439 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.439 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441738+0000", 2024-09-11T06:51:37.439 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2813000000000001e-05 2024-09-11T06:51:37.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.439 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.440 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.440 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441759+0000", 2024-09-11T06:51:37.440 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1246999999999998e-05 2024-09-11T06:51:37.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.440 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.440 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.440 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.441779+0000", 2024-09-11T06:51:37.440 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0392999999999999e-05 2024-09-11T06:51:37.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.440 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.440 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.440 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458804+0000", 2024-09-11T06:51:37.441 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.017024182999999998 2024-09-11T06:51:37.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.441 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.441 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.441 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.458977+0000", 2024-09-11T06:51:37.441 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000173926 2024-09-11T06:51:37.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.441 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.441 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:37.441 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.459170+0000", 2024-09-11T06:51:37.441 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000192184 2024-09-11T06:51:37.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.442 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.442 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:37.442 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.462300+0000", 2024-09-11T06:51:37.442 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031304969999999999 2024-09-11T06:51:37.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.442 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.442 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:37.442 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463408+0000", 2024-09-11T06:51:37.442 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011077839999999999 2024-09-11T06:51:37.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.442 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.442 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:37.442 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463497+0000", 2024-09-11T06:51:37.442 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.8828999999999999e-05 2024-09-11T06:51:37.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.443 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.443 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.463558+0000", 2024-09-11T06:51:37.443 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.1110000000000003e-05 2024-09-11T06:51:37.443 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.443 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.443 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.443 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:37.444 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:51:37.444 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 150.57209537700001, 2024-09-11T06:51:37.444 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013136297, 2024-09-11T06:51:37.444 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.444 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.444 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:37.444 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:37.444 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:37.444 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:37.444 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 138 2024-09-11T06:51:37.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.444 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.444 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.444 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.445 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:51:37.445 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.445 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.445 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:51:37.445 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.445 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.445 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818365+0000", 2024-09-11T06:51:37.445 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.89e-07 2024-09-11T06:51:37.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.446 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.446 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818369+0000", 2024-09-11T06:51:37.446 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4170000000000001e-06 2024-09-11T06:51:37.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.446 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.446 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.446 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818376+0000", 2024-09-11T06:51:37.446 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4270000000000002e-06 2024-09-11T06:51:37.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.446 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.446 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.446 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818383+0000", 2024-09-11T06:51:37.446 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3060000000000003e-06 2024-09-11T06:51:37.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.447 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.447 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.824783+0000", 2024-09-11T06:51:37.447 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064003020000000001 2024-09-11T06:51:37.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.447 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.447 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825013+0000", 2024-09-11T06:51:37.447 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023039499999999999 2024-09-11T06:51:37.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.447 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:37.447 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.825484+0000", 2024-09-11T06:51:37.448 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000470562 2024-09-11T06:51:37.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.448 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:37.448 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828633+0000", 2024-09-11T06:51:37.448 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031496689999999999 2024-09-11T06:51:37.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.448 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:37.448 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831413+0000", 2024-09-11T06:51:37.448 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027793219999999999 2024-09-11T06:51:37.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.448 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:37.449 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831450+0000", 2024-09-11T06:51:37.449 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7370999999999998e-05 2024-09-11T06:51:37.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.449 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.449 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831501+0000", 2024-09-11T06:51:37.449 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1036999999999999e-05 2024-09-11T06:51:37.449 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.449 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.449 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.450 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.450 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:37.450 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:51:37.450 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 150.572070897, 2024-09-11T06:51:37.450 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013223011, 2024-09-11T06:51:37.450 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.450 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.450 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:37.450 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:37.450 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:37.450 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:37.450 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 139 2024-09-11T06:51:37.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.450 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.451 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.451 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.451 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:51:37.451 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.451 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.451 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.451 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818389+0000", 2024-09-11T06:51:37.451 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.451 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.451 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.451 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818390+0000", 2024-09-11T06:51:37.451 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5600000000000001e-07 2024-09-11T06:51:37.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.452 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.452 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818394+0000", 2024-09-11T06:51:37.452 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5929999999999999e-06 2024-09-11T06:51:37.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.452 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.452 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818399+0000", 2024-09-11T06:51:37.452 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7740000000000004e-06 2024-09-11T06:51:37.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.452 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.452 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818412+0000", 2024-09-11T06:51:37.452 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2625e-05 2024-09-11T06:51:37.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.453 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.453 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826205+0000", 2024-09-11T06:51:37.453 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0077930730000000002 2024-09-11T06:51:37.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.453 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.453 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826281+0000", 2024-09-11T06:51:37.453 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5876000000000003e-05 2024-09-11T06:51:37.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.453 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:37.453 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.826384+0000", 2024-09-11T06:51:37.454 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103838 2024-09-11T06:51:37.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.454 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:37.454 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.830642+0000", 2024-09-11T06:51:37.454 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004257083 2024-09-11T06:51:37.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.454 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:37.454 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831509+0000", 2024-09-11T06:51:37.454 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00086730900000000003 2024-09-11T06:51:37.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.455 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:37.455 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831547+0000", 2024-09-11T06:51:37.455 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7793000000000002e-05 2024-09-11T06:51:37.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.455 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.455 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831612+0000", 2024-09-11T06:51:37.455 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5691000000000005e-05 2024-09-11T06:51:37.455 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.455 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.455 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.456 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:37.456 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:51:37.456 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 150.572042203, 2024-09-11T06:51:37.456 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016098024999999998, 2024-09-11T06:51:37.456 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.456 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.456 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:37.456 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:37.456 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:37.456 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:37.456 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 140 2024-09-11T06:51:37.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.456 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.457 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.457 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.457 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:51:37.457 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.457 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.457 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.457 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:51:37.457 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.457 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.457 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.457 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818418+0000", 2024-09-11T06:51:37.457 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5199999999999998e-07 2024-09-11T06:51:37.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.458 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.458 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.458 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818479+0000", 2024-09-11T06:51:37.458 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0368000000000002e-05 2024-09-11T06:51:37.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.458 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.459 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.459 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818487+0000", 2024-09-11T06:51:37.459 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2339999999999994e-06 2024-09-11T06:51:37.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.459 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.459 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818491+0000", 2024-09-11T06:51:37.459 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4329999999999997e-06 2024-09-11T06:51:37.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.460 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.460 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828080+0000", 2024-09-11T06:51:37.460 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0095890449999999992 2024-09-11T06:51:37.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.460 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.460 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.460 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828130+0000", 2024-09-11T06:51:37.460 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9239000000000001e-05 2024-09-11T06:51:37.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.460 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.460 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:37.460 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828208+0000", 2024-09-11T06:51:37.460 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.852e-05 2024-09-11T06:51:37.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.461 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.461 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:37.461 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831625+0000", 2024-09-11T06:51:37.461 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034169410000000002 2024-09-11T06:51:37.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.461 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.461 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:37.461 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834411+0000", 2024-09-11T06:51:37.461 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002786111 2024-09-11T06:51:37.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.461 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.461 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:37.461 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834457+0000", 2024-09-11T06:51:37.462 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6041000000000002e-05 2024-09-11T06:51:37.462 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.462 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.462 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.462 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834516+0000", 2024-09-11T06:51:37.462 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8740999999999999e-05 2024-09-11T06:51:37.462 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.462 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.462 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.462 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.462 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.462 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4165.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-11T06:51:37.462 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:51:37.462 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 150.571961136, 2024-09-11T06:51:37.463 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016588772000000002, 2024-09-11T06:51:37.463 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.463 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.463 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:37.463 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:37.463 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4165", 2024-09-11T06:51:37.463 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.89:0/2875175839", 2024-09-11T06:51:37.463 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 141 2024-09-11T06:51:37.463 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.463 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.463 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.463 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.463 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:51:37.463 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.463 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.464 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.464 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.464 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818499+0000", 2024-09-11T06:51:37.464 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.464 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.464 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.464 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.464 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818500+0000", 2024-09-11T06:51:37.464 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0800000000000005e-07 2024-09-11T06:51:37.464 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.464 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.464 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.464 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818505+0000", 2024-09-11T06:51:37.464 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2429999999999996e-06 2024-09-11T06:51:37.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.465 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.465 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.465 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818510+0000", 2024-09-11T06:51:37.465 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9339999999999997e-06 2024-09-11T06:51:37.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.465 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.465 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.465 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.818514+0000", 2024-09-11T06:51:37.465 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7270000000000001e-06 2024-09-11T06:51:37.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.465 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.465 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.465 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828711+0000", 2024-09-11T06:51:37.465 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010197031 2024-09-11T06:51:37.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.466 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.466 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.466 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.828853+0000", 2024-09-11T06:51:37.466 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014201200000000001 2024-09-11T06:51:37.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.466 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.466 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:37.466 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.829115+0000", 2024-09-11T06:51:37.466 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026257500000000003 2024-09-11T06:51:37.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.466 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.466 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:37.466 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.831629+0000", 2024-09-11T06:51:37.467 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025138169999999998 2024-09-11T06:51:37.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.467 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.467 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:37.467 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.834991+0000", 2024-09-11T06:51:37.467 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033624459999999998 2024-09-11T06:51:37.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.467 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.467 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:37.467 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835032+0000", 2024-09-11T06:51:37.467 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1001000000000003e-05 2024-09-11T06:51:37.468 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.468 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.468 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.468 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:06.835088+0000", 2024-09-11T06:51:37.468 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5478e-05 2024-09-11T06:51:37.468 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.468 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.468 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.468 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.468 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.468 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5235.0:1 2.5 2:b2cc3336:::rbd_id.testrbdnbd21607:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-11T06:51:37.468 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:51:37.468 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 62.831159114000002, 2024-09-11T06:51:37.468 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.68353462499999995, 2024-09-11T06:51:37.469 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.469 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.469 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:37.469 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:37.469 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5235", 2024-09-11T06:51:37.469 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2008977164", 2024-09-11T06:51:37.469 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:51:37.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.469 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.469 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.469 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.469 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:51:37.469 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.470 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.470 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.470 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559301+0000", 2024-09-11T06:51:37.470 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.470 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.470 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.470 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559303+0000", 2024-09-11T06:51:37.470 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1830000000000001e-06 2024-09-11T06:51:37.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.470 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.470 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.470 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559314+0000", 2024-09-11T06:51:37.470 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0396e-05 2024-09-11T06:51:37.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.471 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.471 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.471 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559335+0000", 2024-09-11T06:51:37.471 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1183e-05 2024-09-11T06:51:37.471 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.471 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.471 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.471 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559354+0000", 2024-09-11T06:51:37.471 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9225999999999999e-05 2024-09-11T06:51:37.471 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.471 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.471 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.471 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559522+0000", 2024-09-11T06:51:37.471 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000168021 2024-09-11T06:51:37.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.472 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.472 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:51:37.472 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:34.559543+0000", 2024-09-11T06:51:37.472 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0616999999999999e-05 2024-09-11T06:51:37.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.472 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.472 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.472 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.238584+0000", 2024-09-11T06:51:37.472 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.67904107499999999 2024-09-11T06:51:37.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.472 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.473 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for missing object", 2024-09-11T06:51:37.473 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.238933+0000", 2024-09-11T06:51:37.473 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034879600000000001 2024-09-11T06:51:37.473 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.473 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.473 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.473 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.241968+0000", 2024-09-11T06:51:37.473 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030350830000000001 2024-09-11T06:51:37.473 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.473 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.473 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.473 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.242281+0000", 2024-09-11T06:51:37.473 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00031299499999999999 2024-09-11T06:51:37.473 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.474 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.474 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.474 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.242836+0000", 2024-09-11T06:51:37.474 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00055504999999999997 2024-09-11T06:51:37.474 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.474 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.474 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.474 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.474 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5235.0:18 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 0~1024 [fadvise_sequential]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T06:51:37.474 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:51:37.474 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 62.128666674999998, 2024-09-11T06:51:37.475 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.048993760999999997, 2024-09-11T06:51:37.475 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.475 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.475 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:37.475 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:37.475 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5235", 2024-09-11T06:51:37.475 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2008977164", 2024-09-11T06:51:37.475 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 18 2024-09-11T06:51:37.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.475 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.475 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.475 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:51:37.475 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.476 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.476 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261794+0000", 2024-09-11T06:51:37.476 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.476 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.476 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261797+0000", 2024-09-11T06:51:37.476 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2059999999999999e-06 2024-09-11T06:51:37.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.476 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.476 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261812+0000", 2024-09-11T06:51:37.477 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5159e-05 2024-09-11T06:51:37.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.477 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.477 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261852+0000", 2024-09-11T06:51:37.477 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.9906e-05 2024-09-11T06:51:37.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.477 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.477 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.261865+0000", 2024-09-11T06:51:37.477 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.312e-05 2024-09-11T06:51:37.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.478 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.263166+0000", 2024-09-11T06:51:37.478 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001301134 2024-09-11T06:51:37.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.478 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for missing object", 2024-09-11T06:51:37.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.263529+0000", 2024-09-11T06:51:37.478 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036279900000000002 2024-09-11T06:51:37.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.478 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.300567+0000", 2024-09-11T06:51:37.478 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.037038326000000003 2024-09-11T06:51:37.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.479 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:37.479 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.300800+0000", 2024-09-11T06:51:37.479 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00023323300000000001 2024-09-11T06:51:37.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.479 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.479 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.479 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.306709+0000", 2024-09-11T06:51:37.479 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0059087690000000003 2024-09-11T06:51:37.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.479 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.479 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.480 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.306850+0000", 2024-09-11T06:51:37.480 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014106200000000001 2024-09-11T06:51:37.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.480 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:37.480 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.307121+0000", 2024-09-11T06:51:37.480 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027083599999999999 2024-09-11T06:51:37.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.481 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:37.481 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.308809+0000", 2024-09-11T06:51:37.481 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0016880759999999999 2024-09-11T06:51:37.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.481 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:37.481 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310382+0000", 2024-09-11T06:51:37.481 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0015732840000000001 2024-09-11T06:51:37.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.481 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:37.481 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310501+0000", 2024-09-11T06:51:37.481 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000118276 2024-09-11T06:51:37.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.482 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.482 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.310787+0000", 2024-09-11T06:51:37.482 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00028657500000000001 2024-09-11T06:51:37.482 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.482 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.482 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.482 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:71 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 524288~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:37.482 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596272+0000", 2024-09-11T06:51:37.482 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 48.79418802, 2024-09-11T06:51:37.482 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.013390681999999999, 2024-09-11T06:51:37.482 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.482 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.483 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:37.483 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:37.483 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:37.483 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:37.483 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 71 2024-09-11T06:51:37.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.483 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.483 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.483 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.483 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596272+0000", 2024-09-11T06:51:37.483 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.483 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.483 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.484 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596272+0000", 2024-09-11T06:51:37.484 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.484 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.484 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.484 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596273+0000", 2024-09-11T06:51:37.484 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.9999999999999996e-07 2024-09-11T06:51:37.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.484 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.484 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.484 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596395+0000", 2024-09-11T06:51:37.485 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012204799999999999 2024-09-11T06:51:37.485 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.485 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.485 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.485 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596411+0000", 2024-09-11T06:51:37.485 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5746000000000001e-05 2024-09-11T06:51:37.485 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.485 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.485 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.485 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596419+0000", 2024-09-11T06:51:37.485 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2600000000000005e-06 2024-09-11T06:51:37.485 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.485 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.485 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.486 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.605311+0000", 2024-09-11T06:51:37.486 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0088915179999999993 2024-09-11T06:51:37.486 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.486 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.486 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.486 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.605436+0000", 2024-09-11T06:51:37.486 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000125203 2024-09-11T06:51:37.486 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.486 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.486 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:37.486 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.605625+0000", 2024-09-11T06:51:37.486 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018961299999999999 2024-09-11T06:51:37.486 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.486 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.487 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:37.487 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609477+0000", 2024-09-11T06:51:37.487 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0038513549999999999 2024-09-11T06:51:37.487 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.487 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.487 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:37.487 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609499+0000", 2024-09-11T06:51:37.487 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2215e-05 2024-09-11T06:51:37.487 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.487 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.487 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:37.487 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609589+0000", 2024-09-11T06:51:37.487 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0029000000000001e-05 2024-09-11T06:51:37.487 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.488 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.488 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.488 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609663+0000", 2024-09-11T06:51:37.488 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3894999999999996e-05 2024-09-11T06:51:37.488 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.488 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.488 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.488 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.488 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.488 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:72 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 655360~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:37.488 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596432+0000", 2024-09-11T06:51:37.488 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 48.794028533000002, 2024-09-11T06:51:37.488 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.01414727, 2024-09-11T06:51:37.488 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.489 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.489 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:37.489 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:37.489 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:37.489 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:37.489 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 72 2024-09-11T06:51:37.489 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.489 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.489 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.489 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.489 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596432+0000", 2024-09-11T06:51:37.489 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.489 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.489 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.490 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.490 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596432+0000", 2024-09-11T06:51:37.490 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.490 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.490 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.490 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.490 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596433+0000", 2024-09-11T06:51:37.490 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.1999999999999998e-07 2024-09-11T06:51:37.490 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.490 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.490 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.490 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596556+0000", 2024-09-11T06:51:37.490 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012379000000000001 2024-09-11T06:51:37.490 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.490 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.491 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.491 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596571+0000", 2024-09-11T06:51:37.491 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4749999999999999e-05 2024-09-11T06:51:37.491 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.491 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.491 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.491 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596579+0000", 2024-09-11T06:51:37.491 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2500000000000006e-06 2024-09-11T06:51:37.491 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.491 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.491 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.491 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.606784+0000", 2024-09-11T06:51:37.491 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010204757 2024-09-11T06:51:37.492 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.492 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.492 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.492 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.606950+0000", 2024-09-11T06:51:37.492 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000166229 2024-09-11T06:51:37.492 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.492 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.492 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:37.492 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.607176+0000", 2024-09-11T06:51:37.492 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022549899999999999 2024-09-11T06:51:37.492 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.492 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.492 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:37.492 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610363+0000", 2024-09-11T06:51:37.493 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031874690000000001 2024-09-11T06:51:37.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.493 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.493 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:37.493 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610490+0000", 2024-09-11T06:51:37.493 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012639899999999999 2024-09-11T06:51:37.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.493 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.493 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:37.493 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610529+0000", 2024-09-11T06:51:37.493 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.8936000000000001e-05 2024-09-11T06:51:37.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.493 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.493 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.493 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610579+0000", 2024-09-11T06:51:37.494 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0371000000000003e-05 2024-09-11T06:51:37.494 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.494 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.494 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.494 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.494 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:73 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 786432~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:37.494 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596592+0000", 2024-09-11T06:51:37.494 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 48.793868279999998, 2024-09-11T06:51:37.494 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.014914779, 2024-09-11T06:51:37.494 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.494 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.494 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:37.494 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:37.495 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:37.495 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:37.495 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 73 2024-09-11T06:51:37.495 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.495 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.495 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.495 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.495 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596592+0000", 2024-09-11T06:51:37.495 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.495 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.495 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.495 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.495 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596592+0000", 2024-09-11T06:51:37.495 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.496 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.496 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.496 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596593+0000", 2024-09-11T06:51:37.496 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.0699999999999996e-07 2024-09-11T06:51:37.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.496 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.496 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.496 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596786+0000", 2024-09-11T06:51:37.496 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019278200000000001 2024-09-11T06:51:37.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.496 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.496 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.496 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596801+0000", 2024-09-11T06:51:37.497 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5690000000000001e-05 2024-09-11T06:51:37.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.497 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.497 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.497 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596815+0000", 2024-09-11T06:51:37.497 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3753e-05 2024-09-11T06:51:37.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.497 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.497 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.497 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608442+0000", 2024-09-11T06:51:37.497 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.011627385 2024-09-11T06:51:37.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.497 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.497 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.498 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608570+0000", 2024-09-11T06:51:37.498 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012740399999999999 2024-09-11T06:51:37.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.498 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.498 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:37.498 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.608754+0000", 2024-09-11T06:51:37.498 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018392899999999999 2024-09-11T06:51:37.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.499 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.499 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:37.499 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610370+0000", 2024-09-11T06:51:37.499 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001615998 2024-09-11T06:51:37.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.499 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.499 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:37.499 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611420+0000", 2024-09-11T06:51:37.500 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0010502790000000001 2024-09-11T06:51:37.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.500 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.500 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:37.500 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611458+0000", 2024-09-11T06:51:37.500 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.7749000000000001e-05 2024-09-11T06:51:37.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.500 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.500 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.500 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611507+0000", 2024-09-11T06:51:37.500 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9002999999999998e-05 2024-09-11T06:51:37.500 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.500 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.500 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.501 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.501 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.501 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:74 2.5 2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head [write 917504~131072] snapc 5=[5,3] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:37.501 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.596849+0000", 2024-09-11T06:51:37.501 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 48.793611642000002, 2024-09-11T06:51:37.501 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.015202963999999999, 2024-09-11T06:51:37.501 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.501 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.501 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:37.501 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:37.502 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:37.502 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:37.502 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 74 2024-09-11T06:51:37.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.502 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.502 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.502 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.502 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596849+0000", 2024-09-11T06:51:37.502 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.502 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.502 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.502 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596849+0000", 2024-09-11T06:51:37.502 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.503 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.503 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.596850+0000", 2024-09-11T06:51:37.503 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3170000000000001e-06 2024-09-11T06:51:37.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.503 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.503 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.597118+0000", 2024-09-11T06:51:37.503 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026772999999999999 2024-09-11T06:51:37.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.503 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.503 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.597136+0000", 2024-09-11T06:51:37.503 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8467000000000001e-05 2024-09-11T06:51:37.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.504 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.504 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.504 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.597148+0000", 2024-09-11T06:51:37.504 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1486e-05 2024-09-11T06:51:37.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.504 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.504 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.504 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609714+0000", 2024-09-11T06:51:37.504 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012566124 2024-09-11T06:51:37.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.504 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.504 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.504 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609792+0000", 2024-09-11T06:51:37.505 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.8579999999999996e-05 2024-09-11T06:51:37.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.505 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:37.505 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.609899+0000", 2024-09-11T06:51:37.505 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00010716499999999999 2024-09-11T06:51:37.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.505 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:37.505 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.610865+0000", 2024-09-11T06:51:37.505 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00096540299999999999 2024-09-11T06:51:37.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.505 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:37.506 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.611970+0000", 2024-09-11T06:51:37.506 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001105118 2024-09-11T06:51:37.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.506 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.506 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:37.506 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.612005+0000", 2024-09-11T06:51:37.506 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4789000000000002e-05 2024-09-11T06:51:37.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.506 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.506 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.506 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.612052+0000", 2024-09-11T06:51:37.506 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6785e-05 2024-09-11T06:51:37.506 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.506 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.506 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.507 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.507 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.507 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5723.0:84 2.5 2:bca04221:ns::rbd_data.11adc335555f.0000000000000003:head [delete] snapc 4=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e111)", 2024-09-11T06:51:37.507 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:34.287960+0000", 2024-09-11T06:51:37.507 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 3.102500445, 2024-09-11T06:51:37.507 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016172883999999998, 2024-09-11T06:51:37.507 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.507 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.507 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:37.507 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:37.507 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5723", 2024-09-11T06:51:37.508 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3725019196", 2024-09-11T06:51:37.508 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 84 2024-09-11T06:51:37.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.508 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.508 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.508 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.508 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.287960+0000", 2024-09-11T06:51:37.508 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.508 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.508 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.508 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.287960+0000", 2024-09-11T06:51:37.508 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.509 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.509 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.509 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.287961+0000", 2024-09-11T06:51:37.509 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4399999999999999e-07 2024-09-11T06:51:37.509 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.509 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.509 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.509 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.287965+0000", 2024-09-11T06:51:37.509 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3540000000000002e-06 2024-09-11T06:51:37.509 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.509 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.510 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.510 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.287978+0000", 2024-09-11T06:51:37.510 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2754e-05 2024-09-11T06:51:37.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.510 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.510 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.287984+0000", 2024-09-11T06:51:37.510 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7229999999999998e-06 2024-09-11T06:51:37.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.510 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.510 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.288063+0000", 2024-09-11T06:51:37.510 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.8083000000000004e-05 2024-09-11T06:51:37.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.511 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.511 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.511 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.288163+0000", 2024-09-11T06:51:37.511 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000100461 2024-09-11T06:51:37.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.511 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.511 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:37.511 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.288298+0000", 2024-09-11T06:51:37.511 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000135291 2024-09-11T06:51:37.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.511 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.511 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:37.511 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.291637+0000", 2024-09-11T06:51:37.512 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033391499999999999 2024-09-11T06:51:37.512 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.512 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.512 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:37.512 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.304062+0000", 2024-09-11T06:51:37.512 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.012424162000000001 2024-09-11T06:51:37.512 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.512 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.512 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:37.512 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.304101+0000", 2024-09-11T06:51:37.512 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.8912999999999998e-05 2024-09-11T06:51:37.512 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.512 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.512 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.512 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.304133+0000", 2024-09-11T06:51:37.513 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2149e-05 2024-09-11T06:51:37.513 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.513 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.513 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.513 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.513 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.513 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5723.0:87 2.5 2:b515ded6:ns::rbd_data.11adc335555f.0000000000000006:head [delete] snapc 4=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e111)", 2024-09-11T06:51:37.513 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:34.288003+0000", 2024-09-11T06:51:37.513 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 3.1024574989999998, 2024-09-11T06:51:37.513 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016283750999999999, 2024-09-11T06:51:37.513 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.513 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.513 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:37.513 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:37.514 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5723", 2024-09-11T06:51:37.514 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3725019196", 2024-09-11T06:51:37.514 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 87 2024-09-11T06:51:37.514 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.514 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.514 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.514 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.514 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.288003+0000", 2024-09-11T06:51:37.514 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.514 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.514 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.514 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.514 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.288003+0000", 2024-09-11T06:51:37.514 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.514 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.515 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.515 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.515 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.288003+0000", 2024-09-11T06:51:37.515 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.3799999999999997e-07 2024-09-11T06:51:37.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.515 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.515 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.515 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.288006+0000", 2024-09-11T06:51:37.515 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0130000000000002e-06 2024-09-11T06:51:37.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.515 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.515 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.515 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.288017+0000", 2024-09-11T06:51:37.516 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0599e-05 2024-09-11T06:51:37.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.516 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.516 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.516 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.288042+0000", 2024-09-11T06:51:37.516 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5327e-05 2024-09-11T06:51:37.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.516 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.516 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.516 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.290594+0000", 2024-09-11T06:51:37.516 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002551382 2024-09-11T06:51:37.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.516 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.516 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.517 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.290645+0000", 2024-09-11T06:51:37.517 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0902999999999997e-05 2024-09-11T06:51:37.517 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.517 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.517 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:37.517 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.290710+0000", 2024-09-11T06:51:37.517 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5145000000000001e-05 2024-09-11T06:51:37.517 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.517 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.517 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:37.517 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.293395+0000", 2024-09-11T06:51:37.517 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0026852740000000001 2024-09-11T06:51:37.517 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.517 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.518 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:37.518 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.304185+0000", 2024-09-11T06:51:37.518 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010789953 2024-09-11T06:51:37.518 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.518 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.518 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:37.518 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.304243+0000", 2024-09-11T06:51:37.518 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8301000000000003e-05 2024-09-11T06:51:37.518 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.518 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.518 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.518 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.304287+0000", 2024-09-11T06:51:37.518 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3316000000000003e-05 2024-09-11T06:51:37.518 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.519 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.519 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.519 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.519 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.519 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5723.0:90 2.5 2:b0f71fdb:ns::rbd_data.11adc335555f.0000000000000009:head [delete] snapc 4=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e111)", 2024-09-11T06:51:37.519 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:34.288141+0000", 2024-09-11T06:51:37.519 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 3.1023188409999998, 2024-09-11T06:51:37.519 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.016266619999999999, 2024-09-11T06:51:37.519 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.519 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.520 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:37.520 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:37.520 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5723", 2024-09-11T06:51:37.520 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3725019196", 2024-09-11T06:51:37.520 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 90 2024-09-11T06:51:37.520 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.520 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.520 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.520 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.520 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.288141+0000", 2024-09-11T06:51:37.520 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.520 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.520 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.520 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.521 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.288141+0000", 2024-09-11T06:51:37.521 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.521 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.521 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.521 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.521 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.288142+0000", 2024-09-11T06:51:37.521 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.5199999999999996e-07 2024-09-11T06:51:37.521 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.521 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.521 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.521 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.288145+0000", 2024-09-11T06:51:37.521 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7269999999999999e-06 2024-09-11T06:51:37.521 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.522 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.522 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.522 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.288152+0000", 2024-09-11T06:51:37.522 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.8959999999999997e-06 2024-09-11T06:51:37.522 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.522 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.522 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.522 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.288157+0000", 2024-09-11T06:51:37.522 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5339999999999999e-06 2024-09-11T06:51:37.522 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.522 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.522 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.522 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.291664+0000", 2024-09-11T06:51:37.522 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00350622 2024-09-11T06:51:37.523 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.523 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.523 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.523 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.291748+0000", 2024-09-11T06:51:37.523 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4413000000000006e-05 2024-09-11T06:51:37.523 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.523 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.523 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:37.523 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.291839+0000", 2024-09-11T06:51:37.523 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.1181000000000007e-05 2024-09-11T06:51:37.523 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.523 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.524 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:37.524 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.293688+0000", 2024-09-11T06:51:37.524 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001848677 2024-09-11T06:51:37.524 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.524 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.524 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:37.524 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.304349+0000", 2024-09-11T06:51:37.524 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010661412 2024-09-11T06:51:37.524 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.524 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.524 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:37.524 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.304385+0000", 2024-09-11T06:51:37.524 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5706e-05 2024-09-11T06:51:37.525 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.525 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.525 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.525 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.304408+0000", 2024-09-11T06:51:37.525 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3102e-05 2024-09-11T06:51:37.525 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.525 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.525 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.525 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.525 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.525 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:37.526 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T06:51:37.653 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:37.653 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:37.653 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:51:37.653 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:37.668 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T06:51:37.788 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:37.788 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:37.788 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:51:37.788 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:51:37.788 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:37.802 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T06:51:37.921 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:37.921 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:51:37.921 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:51:37.921 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:51:37.921 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.921 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4558.0:1 3.3 3:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-11T06:51:37.921 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:51:37.921 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 80.828317709999993, 2024-09-11T06:51:37.921 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0085520600000000002, 2024-09-11T06:51:37.922 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.922 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.922 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:37.922 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:37.922 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4558", 2024-09-11T06:51:37.922 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1478388898", 2024-09-11T06:51:37.922 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:51:37.922 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.922 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.922 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.922 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.922 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:51:37.923 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.923 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.923 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.923 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.923 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090529+0000", 2024-09-11T06:51:37.923 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.924 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.924 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.924 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.924 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090533+0000", 2024-09-11T06:51:37.925 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2629999999999999e-06 2024-09-11T06:51:37.925 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.925 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.925 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.925 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090546+0000", 2024-09-11T06:51:37.925 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2918999999999999e-05 2024-09-11T06:51:37.926 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.926 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.926 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.926 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090572+0000", 2024-09-11T06:51:37.926 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6412000000000001e-05 2024-09-11T06:51:37.926 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.927 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.927 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.927 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090594+0000", 2024-09-11T06:51:37.927 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2476e-05 2024-09-11T06:51:37.927 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.927 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.927 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.928 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.090938+0000", 2024-09-11T06:51:37.928 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034325099999999998 2024-09-11T06:51:37.928 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.928 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.928 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.928 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.091149+0000", 2024-09-11T06:51:37.929 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021151300000000001 2024-09-11T06:51:37.929 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.929 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.929 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:37.929 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.091377+0000", 2024-09-11T06:51:37.929 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022785399999999999 2024-09-11T06:51:37.929 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.930 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.930 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:37.930 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.092912+0000", 2024-09-11T06:51:37.930 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0015346089999999999 2024-09-11T06:51:37.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.930 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.931 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:37.931 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.098919+0000", 2024-09-11T06:51:37.931 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0060076799999999996 2024-09-11T06:51:37.931 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.932 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.932 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:37.932 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.099009+0000", 2024-09-11T06:51:37.932 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.0056000000000006e-05 2024-09-11T06:51:37.932 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.932 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.933 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.933 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:17.099081+0000", 2024-09-11T06:51:37.933 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.2026999999999993e-05 2024-09-11T06:51:37.933 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.933 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.933 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.934 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.934 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.934 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:51:37.934 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:51:37.934 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 74.786828630000002, 2024-09-11T06:51:37.934 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0071924390000000001, 2024-09-11T06:51:37.935 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.935 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.935 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:37.935 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.935 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.936 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.936 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:51:37.936 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.936 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.936 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.936 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.937 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132018+0000", 2024-09-11T06:51:37.937 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.937 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.937 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.938 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.938 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.132020+0000", 2024-09-11T06:51:37.938 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.846e-06 2024-09-11T06:51:37.938 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.938 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.938 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.938 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135688+0000", 2024-09-11T06:51:37.938 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036677350000000001 2024-09-11T06:51:37.938 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.938 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.938 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.938 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135747+0000", 2024-09-11T06:51:37.938 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.9314999999999997e-05 2024-09-11T06:51:37.938 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.939 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.939 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.939 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.135797+0000", 2024-09-11T06:51:37.939 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.9984000000000001e-05 2024-09-11T06:51:37.939 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.939 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.939 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.939 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136143+0000", 2024-09-11T06:51:37.939 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034594299999999998 2024-09-11T06:51:37.939 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.939 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.939 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.940 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136179+0000", 2024-09-11T06:51:37.940 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.5636000000000002e-05 2024-09-11T06:51:37.940 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.940 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.940 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.940 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.139211+0000", 2024-09-11T06:51:37.940 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0030319800000000001 2024-09-11T06:51:37.940 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.940 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.940 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.940 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.940 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.941 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:51:37.941 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:51:37.941 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 74.784765375999996, 2024-09-11T06:51:37.941 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064954139999999997, 2024-09-11T06:51:37.941 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.941 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.941 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:37.941 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.941 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.941 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.941 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:51:37.942 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.942 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.942 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.942 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.942 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134082+0000", 2024-09-11T06:51:37.942 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.942 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.942 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.942 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.942 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.134084+0000", 2024-09-11T06:51:37.942 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.894e-06 2024-09-11T06:51:37.942 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.943 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.943 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.943 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136513+0000", 2024-09-11T06:51:37.943 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024290969999999999 2024-09-11T06:51:37.943 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.943 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.943 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.943 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136554+0000", 2024-09-11T06:51:37.943 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.0932e-05 2024-09-11T06:51:37.943 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.943 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.944 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.944 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136584+0000", 2024-09-11T06:51:37.944 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0056000000000002e-05 2024-09-11T06:51:37.944 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.944 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.944 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.944 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136805+0000", 2024-09-11T06:51:37.944 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022086100000000001 2024-09-11T06:51:37.944 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.944 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.945 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.945 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.136864+0000", 2024-09-11T06:51:37.945 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.9907000000000001e-05 2024-09-11T06:51:37.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.945 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.945 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.945 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.140577+0000", 2024-09-11T06:51:37.945 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0037126669999999998 2024-09-11T06:51:37.945 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.945 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.945 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.946 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.946 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:37.946 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:51:37.946 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 74.776474575999998, 2024-09-11T06:51:37.946 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035763180000000002, 2024-09-11T06:51:37.946 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.946 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.946 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:37.946 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.946 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.946 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.946 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:51:37.947 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.947 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.947 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.947 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.947 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142373+0000", 2024-09-11T06:51:37.947 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.947 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.947 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.948 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.948 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.142374+0000", 2024-09-11T06:51:37.948 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5460000000000001e-06 2024-09-11T06:51:37.948 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.948 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.948 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.948 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143902+0000", 2024-09-11T06:51:37.948 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00152808 2024-09-11T06:51:37.948 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.948 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.949 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.949 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143946+0000", 2024-09-11T06:51:37.949 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3874e-05 2024-09-11T06:51:37.949 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.949 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.949 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.949 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.143978+0000", 2024-09-11T06:51:37.949 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2425999999999998e-05 2024-09-11T06:51:37.949 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.949 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.949 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.950 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.144177+0000", 2024-09-11T06:51:37.950 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019837500000000001 2024-09-11T06:51:37.950 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.950 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.950 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.950 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.144198+0000", 2024-09-11T06:51:37.950 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1639000000000001e-05 2024-09-11T06:51:37.950 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.950 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.950 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.950 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.145949+0000", 2024-09-11T06:51:37.950 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0017503779999999999 2024-09-11T06:51:37.951 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.951 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.951 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.951 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.951 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.951 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:37.951 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:51:37.951 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 74.772158873999999, 2024-09-11T06:51:37.951 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0057693220000000003, 2024-09-11T06:51:37.951 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.951 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.952 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:37.952 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.952 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.952 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.952 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:51:37.952 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.952 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.952 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.952 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.952 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146688+0000", 2024-09-11T06:51:37.952 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.952 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.952 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.952 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.953 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.146690+0000", 2024-09-11T06:51:37.953 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.607e-06 2024-09-11T06:51:37.953 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.953 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.953 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.953 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148693+0000", 2024-09-11T06:51:37.953 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0020029739999999998 2024-09-11T06:51:37.953 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.953 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.953 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.953 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148735+0000", 2024-09-11T06:51:37.953 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1958e-05 2024-09-11T06:51:37.953 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.954 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.954 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.954 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.148764+0000", 2024-09-11T06:51:37.954 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8999000000000001e-05 2024-09-11T06:51:37.954 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.954 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.954 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.954 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149069+0000", 2024-09-11T06:51:37.954 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000305032 2024-09-11T06:51:37.954 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.954 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.954 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.954 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149204+0000", 2024-09-11T06:51:37.955 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013495 2024-09-11T06:51:37.955 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.955 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.955 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.955 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.152458+0000", 2024-09-11T06:51:37.955 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0032538020000000001 2024-09-11T06:51:37.955 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.955 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.955 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.955 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.955 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.955 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:37.955 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:51:37.955 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 74.771260333000001, 2024-09-11T06:51:37.955 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0042425329999999997, 2024-09-11T06:51:37.956 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.956 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.956 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:37.956 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.956 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.956 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.956 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:51:37.956 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.956 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.956 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.957 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.957 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147587+0000", 2024-09-11T06:51:37.957 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.957 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.957 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.957 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.957 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.147588+0000", 2024-09-11T06:51:37.957 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.277e-06 2024-09-11T06:51:37.957 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.957 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.958 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.958 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149051+0000", 2024-09-11T06:51:37.958 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001462669 2024-09-11T06:51:37.958 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.958 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.958 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.958 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149090+0000", 2024-09-11T06:51:37.958 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.8967999999999997e-05 2024-09-11T06:51:37.959 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.959 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.959 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.959 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149120+0000", 2024-09-11T06:51:37.959 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0766000000000003e-05 2024-09-11T06:51:37.959 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.959 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.959 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.959 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149395+0000", 2024-09-11T06:51:37.959 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027484900000000002 2024-09-11T06:51:37.959 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.959 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.959 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.959 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.149439+0000", 2024-09-11T06:51:37.960 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.3940999999999998e-05 2024-09-11T06:51:37.960 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.960 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.960 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.960 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.151829+0000", 2024-09-11T06:51:37.960 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002390063 2024-09-11T06:51:37.960 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.960 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.960 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.960 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.960 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.960 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.6 61/60 [PushOp(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head, version: 54'2, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:37.960 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:51:37.961 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 74.765119702000007, 2024-09-11T06:51:37.961 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0046568219999999997, 2024-09-11T06:51:37.961 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.961 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.961 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:37.961 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.961 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.961 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.961 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:51:37.962 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.962 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.962 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.962 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.962 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153727+0000", 2024-09-11T06:51:37.962 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.962 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.962 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.962 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.962 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.153729+0000", 2024-09-11T06:51:37.962 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5629999999999999e-06 2024-09-11T06:51:37.962 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.963 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.963 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.963 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155179+0000", 2024-09-11T06:51:37.963 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001450469 2024-09-11T06:51:37.963 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.963 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.963 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.963 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155221+0000", 2024-09-11T06:51:37.963 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.1448e-05 2024-09-11T06:51:37.963 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.963 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.963 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.963 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155249+0000", 2024-09-11T06:51:37.963 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8433e-05 2024-09-11T06:51:37.964 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.964 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.964 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.964 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155420+0000", 2024-09-11T06:51:37.964 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017070599999999999 2024-09-11T06:51:37.964 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.964 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.964 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.964 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155445+0000", 2024-09-11T06:51:37.964 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4813000000000001e-05 2024-09-11T06:51:37.964 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.964 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.965 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.965 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.158384+0000", 2024-09-11T06:51:37.965 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0029393900000000001 2024-09-11T06:51:37.965 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.965 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.965 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.965 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.965 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.965 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.5 61/60 [PushOp(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head, version: 54'2, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:b92e2369:::rbd_data.1314f0a2bfa2.0000000000000000:head@54'2, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:37.965 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:51:37.965 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 74.763092952999997, 2024-09-11T06:51:37.965 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0046826020000000001, 2024-09-11T06:51:37.965 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.966 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.966 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:37.966 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.966 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.966 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.966 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:51:37.966 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.966 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.966 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.966 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.966 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155754+0000", 2024-09-11T06:51:37.966 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.966 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.966 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.967 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.967 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.155756+0000", 2024-09-11T06:51:37.967 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.384e-06 2024-09-11T06:51:37.967 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.967 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.967 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.967 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157205+0000", 2024-09-11T06:51:37.967 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014496260000000001 2024-09-11T06:51:37.967 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.967 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.967 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.967 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157252+0000", 2024-09-11T06:51:37.967 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.6368000000000001e-05 2024-09-11T06:51:37.967 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.968 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.968 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.968 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157286+0000", 2024-09-11T06:51:37.968 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4944999999999998e-05 2024-09-11T06:51:37.968 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.968 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.968 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.968 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157540+0000", 2024-09-11T06:51:37.968 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00025307999999999998 2024-09-11T06:51:37.968 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.969 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.969 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.969 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.157576+0000", 2024-09-11T06:51:37.969 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.6878999999999997e-05 2024-09-11T06:51:37.969 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.969 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.969 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.969 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:23.160437+0000", 2024-09-11T06:51:37.969 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00286032 2024-09-11T06:51:37.969 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.969 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.969 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.969 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.969 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.970 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:51:37.970 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.280679+0000", 2024-09-11T06:51:37.970 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 62.638167963999997, 2024-09-11T06:51:37.970 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0040934309999999998, 2024-09-11T06:51:37.970 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.970 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.970 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:51:37.970 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.970 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.970 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.970 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280679+0000", 2024-09-11T06:51:37.970 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.970 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.971 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.971 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.971 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280679+0000", 2024-09-11T06:51:37.971 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.971 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.971 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.971 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.971 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280681+0000", 2024-09-11T06:51:37.971 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2529999999999999e-06 2024-09-11T06:51:37.971 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.971 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.971 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.971 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280713+0000", 2024-09-11T06:51:37.971 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.1936000000000001e-05 2024-09-11T06:51:37.972 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.972 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.972 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.972 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280758+0000", 2024-09-11T06:51:37.972 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5124999999999999e-05 2024-09-11T06:51:37.972 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.972 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.973 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.973 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280779+0000", 2024-09-11T06:51:37.973 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0936000000000001e-05 2024-09-11T06:51:37.973 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.973 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.973 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.973 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.280932+0000", 2024-09-11T06:51:37.973 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015220899999999999 2024-09-11T06:51:37.973 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.973 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.973 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.973 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.284773+0000", 2024-09-11T06:51:37.974 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0038409719999999998 2024-09-11T06:51:37.974 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.974 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.974 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.974 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.974 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.974 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPull(2.5 e70/69 cost 1048576)", 2024-09-11T06:51:37.974 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:35.293438+0000", 2024-09-11T06:51:37.974 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 62.625409228000002, 2024-09-11T06:51:37.974 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0061479350000000002, 2024-09-11T06:51:37.974 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.974 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.974 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "reached pg", 2024-09-11T06:51:37.975 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.975 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.975 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.975 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293438+0000", 2024-09-11T06:51:37.975 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.975 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.975 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.975 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.975 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293438+0000", 2024-09-11T06:51:37.975 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.975 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.975 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.976 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.976 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293439+0000", 2024-09-11T06:51:37.976 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.437e-06 2024-09-11T06:51:37.976 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.976 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.976 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.976 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293459+0000", 2024-09-11T06:51:37.976 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9922e-05 2024-09-11T06:51:37.976 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.976 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.976 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.977 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293482+0000", 2024-09-11T06:51:37.977 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2969e-05 2024-09-11T06:51:37.977 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.977 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.977 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.977 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.293494+0000", 2024-09-11T06:51:37.977 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.131e-05 2024-09-11T06:51:37.977 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.977 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.977 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.977 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.295901+0000", 2024-09-11T06:51:37.978 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0024077949999999999 2024-09-11T06:51:37.978 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.978 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.978 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.978 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:35.299586+0000", 2024-09-11T06:51:37.978 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0036845020000000001 2024-09-11T06:51:37.978 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.978 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.978 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.979 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.979 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.979 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(client.5239.0:37 2.7 e91/69 2:eab0161e:::rbd_data.1314f0a2bfa2.0000000000000001:head v 91'134, mlcod=91'130)", 2024-09-11T06:51:37.979 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:08.068368+0000", 2024-09-11T06:51:37.979 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 29.850479149999998, 2024-09-11T06:51:37.979 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003850118, 2024-09-11T06:51:37.979 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.979 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.979 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:37.979 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.979 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.979 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.980 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068368+0000", 2024-09-11T06:51:37.980 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.980 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.980 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.980 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.980 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068368+0000", 2024-09-11T06:51:37.980 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.980 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.980 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.980 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.980 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068369+0000", 2024-09-11T06:51:37.981 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2699999999999998e-07 2024-09-11T06:51:37.981 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.981 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.981 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.981 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068458+0000", 2024-09-11T06:51:37.981 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.9075999999999999e-05 2024-09-11T06:51:37.981 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.981 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.981 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.981 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068471+0000", 2024-09-11T06:51:37.981 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.269e-05 2024-09-11T06:51:37.982 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.982 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.982 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.982 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.068480+0000", 2024-09-11T06:51:37.982 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.6199999999999994e-06 2024-09-11T06:51:37.982 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.982 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.982 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.982 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.069523+0000", 2024-09-11T06:51:37.982 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001042465 2024-09-11T06:51:37.982 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.982 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.983 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.983 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.069542+0000", 2024-09-11T06:51:37.983 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9386e-05 2024-09-11T06:51:37.983 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.983 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.983 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:37.983 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072122+0000", 2024-09-11T06:51:37.983 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0025799540000000002 2024-09-11T06:51:37.983 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.983 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.983 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.983 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072218+0000", 2024-09-11T06:51:37.984 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.6100000000000005e-05 2024-09-11T06:51:37.984 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.984 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.984 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.984 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.984 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.984 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(client.5239.0:41 2.7 e91/69 2:eab0161e:::rbd_data.1314f0a2bfa2.0000000000000001:head v 91'138, mlcod=91'132)", 2024-09-11T06:51:37.984 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:08.071899+0000", 2024-09-11T06:51:37.984 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 29.846948372, 2024-09-11T06:51:37.984 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0037511240000000002, 2024-09-11T06:51:37.984 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.985 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.985 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:37.985 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.985 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.985 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.985 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.071899+0000", 2024-09-11T06:51:37.985 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.985 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.985 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.985 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.985 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.071899+0000", 2024-09-11T06:51:37.986 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.986 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.986 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.986 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.986 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.071900+0000", 2024-09-11T06:51:37.986 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3500000000000005e-07 2024-09-11T06:51:37.986 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.986 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.986 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.986 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072027+0000", 2024-09-11T06:51:37.986 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012720299999999999 2024-09-11T06:51:37.986 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.987 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.987 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.987 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072043+0000", 2024-09-11T06:51:37.987 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6475999999999999e-05 2024-09-11T06:51:37.987 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.987 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.987 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.987 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072053+0000", 2024-09-11T06:51:37.987 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2860000000000005e-06 2024-09-11T06:51:37.987 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.987 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.987 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.988 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072256+0000", 2024-09-11T06:51:37.988 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000203876 2024-09-11T06:51:37.988 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.988 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.988 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.988 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.072282+0000", 2024-09-11T06:51:37.988 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5811000000000001e-05 2024-09-11T06:51:37.988 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.988 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.989 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:37.989 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.075544+0000", 2024-09-11T06:51:37.989 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0032621500000000001 2024-09-11T06:51:37.989 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.989 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.989 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.989 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:08.075650+0000", 2024-09-11T06:51:37.989 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000105487 2024-09-11T06:51:37.989 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.989 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.990 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.990 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.990 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.990 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.4 101/100 [PushOp(2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head, version: 54'23, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head@54'23, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T06:51:37.990 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.525431+0000", 2024-09-11T06:51:37.990 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.39341581, 2024-09-11T06:51:37.990 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0042697100000000003, 2024-09-11T06:51:37.990 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.990 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.990 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:37.990 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.990 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.990 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.991 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.525431+0000", 2024-09-11T06:51:37.991 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.991 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.991 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.991 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.991 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.525431+0000", 2024-09-11T06:51:37.991 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.991 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.991 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.991 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.991 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.525433+0000", 2024-09-11T06:51:37.991 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5829999999999999e-06 2024-09-11T06:51:37.991 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.992 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.992 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.992 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.527430+0000", 2024-09-11T06:51:37.992 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0019966350000000001 2024-09-11T06:51:37.992 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.992 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.992 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.992 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.527477+0000", 2024-09-11T06:51:37.992 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7681e-05 2024-09-11T06:51:37.992 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.992 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.992 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.992 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.527525+0000", 2024-09-11T06:51:37.992 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.8244000000000001e-05 2024-09-11T06:51:37.993 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.993 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.993 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.993 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.527795+0000", 2024-09-11T06:51:37.993 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00026935199999999999 2024-09-11T06:51:37.993 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.993 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.993 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.993 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.527816+0000", 2024-09-11T06:51:37.993 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1112e-05 2024-09-11T06:51:37.993 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.993 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.993 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.994 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.529701+0000", 2024-09-11T06:51:37.994 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001885103 2024-09-11T06:51:37.994 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.994 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.994 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.994 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.994 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.994 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.4 101/100 [PushOp(2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head, version: 54'23, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head@54'23, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:37.994 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.540925+0000", 2024-09-11T06:51:37.994 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.377922414, 2024-09-11T06:51:37.994 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0040908539999999997, 2024-09-11T06:51:37.994 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.994 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.994 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:37.995 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.995 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.995 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.995 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.540925+0000", 2024-09-11T06:51:37.995 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.995 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.995 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.995 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.995 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.540925+0000", 2024-09-11T06:51:37.995 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.995 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.995 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.995 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:37.995 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.540926+0000", 2024-09-11T06:51:37.995 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.813e-06 2024-09-11T06:51:37.996 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.996 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.996 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:37.996 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542405+0000", 2024-09-11T06:51:37.996 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001478011 2024-09-11T06:51:37.996 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.996 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.996 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:37.996 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542453+0000", 2024-09-11T06:51:37.996 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.8677999999999997e-05 2024-09-11T06:51:37.996 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.996 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.996 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:37.997 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542483+0000", 2024-09-11T06:51:37.997 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9921999999999999e-05 2024-09-11T06:51:37.997 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.997 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.997 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:37.997 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542758+0000", 2024-09-11T06:51:37.997 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00027447499999999999 2024-09-11T06:51:37.997 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.997 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.997 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:37.997 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542802+0000", 2024-09-11T06:51:37.997 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4288e-05 2024-09-11T06:51:37.997 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.997 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.997 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:37.998 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.545016+0000", 2024-09-11T06:51:37.998 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0022136669999999999 2024-09-11T06:51:37.998 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.998 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:37.998 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:37.998 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.998 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.998 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.4 101/100 [PushOp(2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head, version: 54'23, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head@54'23, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:37.998 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.547946+0000", 2024-09-11T06:51:37.998 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.370900927000001, 2024-09-11T06:51:37.998 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035631679999999998, 2024-09-11T06:51:37.998 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:37.998 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:37.999 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:37.999 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:37.999 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.999 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:37.999 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.547946+0000", 2024-09-11T06:51:37.999 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.999 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:37.999 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:37.999 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:37.999 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.547946+0000", 2024-09-11T06:51:37.999 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:37.999 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.000 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.000 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:38.000 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.547948+0000", 2024-09-11T06:51:38.000 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4529999999999999e-06 2024-09-11T06:51:38.000 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.000 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.000 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:38.001 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549382+0000", 2024-09-11T06:51:38.001 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0014345759999999999 2024-09-11T06:51:38.001 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.001 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.001 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:38.001 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549425+0000", 2024-09-11T06:51:38.001 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.2528999999999999e-05 2024-09-11T06:51:38.002 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.002 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.002 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:38.002 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549453+0000", 2024-09-11T06:51:38.002 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7980999999999999e-05 2024-09-11T06:51:38.002 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.002 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.002 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.002 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549623+0000", 2024-09-11T06:51:38.002 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000170753 2024-09-11T06:51:38.003 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.003 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.003 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:38.003 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549644+0000", 2024-09-11T06:51:38.003 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0628999999999998e-05 2024-09-11T06:51:38.003 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.003 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.003 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:38.003 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.551509+0000", 2024-09-11T06:51:38.003 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001865247 2024-09-11T06:51:38.004 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.004 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:38.004 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.004 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.004 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.004 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(client.6081.0:37 2.4 e101/100 2:27499dce:::rbd_object_map.1314f0a2bfa2:head v 101'172, mlcod=54'22)", 2024-09-11T06:51:38.004 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.549499+0000", 2024-09-11T06:51:38.004 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.36934806, 2024-09-11T06:51:38.004 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0087992420000000005, 2024-09-11T06:51:38.004 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:38.005 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:38.005 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:38.005 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:38.005 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.005 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:38.005 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549499+0000", 2024-09-11T06:51:38.005 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.005 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.005 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.005 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:38.005 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549499+0000", 2024-09-11T06:51:38.006 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.006 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.006 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.006 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:38.006 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549500+0000", 2024-09-11T06:51:38.006 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.217e-06 2024-09-11T06:51:38.006 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.006 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.006 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:38.006 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549511+0000", 2024-09-11T06:51:38.006 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.1143e-05 2024-09-11T06:51:38.007 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.007 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.007 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:38.007 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549532+0000", 2024-09-11T06:51:38.007 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0513000000000002e-05 2024-09-11T06:51:38.007 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.007 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.007 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:38.007 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.549545+0000", 2024-09-11T06:51:38.007 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3209e-05 2024-09-11T06:51:38.007 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.008 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.008 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.008 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.551649+0000", 2024-09-11T06:51:38.008 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002103696 2024-09-11T06:51:38.008 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.008 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.008 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:38.008 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.551694+0000", 2024-09-11T06:51:38.008 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5674999999999999e-05 2024-09-11T06:51:38.008 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.008 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.008 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:38.009 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.558099+0000", 2024-09-11T06:51:38.009 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064040679999999997 2024-09-11T06:51:38.009 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.009 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.009 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:38.009 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.558298+0000", 2024-09-11T06:51:38.009 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019972100000000001 2024-09-11T06:51:38.009 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.009 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:38.009 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.009 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.010 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.010 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "MOSDPGPush(2.4 101/100 [PushOp(2:272abc33:ns::rbd_data.11adc335555f.000000000000000c:head, version: 54'24, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(2:272abc33:ns::rbd_data.11adc335555f.000000000000000c:head@54'24, size: 4194304, copy_subset: [(0, 4194304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-11T06:51:38.010 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.572958+0000", 2024-09-11T06:51:38.010 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.345888634000001, 2024-09-11T06:51:38.010 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035425080000000002, 2024-09-11T06:51:38.010 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:38.010 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:38.010 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:38.010 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:38.010 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.011 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:38.011 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.572958+0000", 2024-09-11T06:51:38.011 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.011 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.011 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.011 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:38.011 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.572958+0000", 2024-09-11T06:51:38.011 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.011 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.011 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.011 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:38.011 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.572960+0000", 2024-09-11T06:51:38.012 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.333e-06 2024-09-11T06:51:38.012 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.012 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.012 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:38.012 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.574357+0000", 2024-09-11T06:51:38.012 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0013975540000000001 2024-09-11T06:51:38.012 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.012 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.012 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:38.012 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.574398+0000", 2024-09-11T06:51:38.012 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.0908000000000002e-05 2024-09-11T06:51:38.013 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.013 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.013 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:38.013 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.574425+0000", 2024-09-11T06:51:38.013 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6899000000000001e-05 2024-09-11T06:51:38.013 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.013 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.013 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.013 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.574604+0000", 2024-09-11T06:51:38.013 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000178359 2024-09-11T06:51:38.013 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.013 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.014 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:38.014 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.574626+0000", 2024-09-11T06:51:38.014 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2768999999999998e-05 2024-09-11T06:51:38.014 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.014 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.014 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:38.014 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.576501+0000", 2024-09-11T06:51:38.014 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001874686 2024-09-11T06:51:38.014 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.014 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:38.014 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.015 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.015 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.015 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_repop(osd.3.0:263 2.6 e107/99 2:65e41ba8:::rbd_data.1314f0a2bfa2.0000000000000006:6 v 107'47, mlcod=74'43)", 2024-09-11T06:51:38.015 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:30.106025+0000", 2024-09-11T06:51:38.015 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 7.8128224949999998, 2024-09-11T06:51:38.015 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0038585270000000001, 2024-09-11T06:51:38.015 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:38.015 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:38.015 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:38.015 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:38.015 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.015 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:38.016 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.106025+0000", 2024-09-11T06:51:38.016 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.016 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.016 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.016 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:38.016 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.106025+0000", 2024-09-11T06:51:38.016 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.016 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.016 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.016 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:38.016 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.106026+0000", 2024-09-11T06:51:38.016 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7399999999999991e-07 2024-09-11T06:51:38.017 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.017 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.017 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:38.017 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.106038+0000", 2024-09-11T06:51:38.017 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2696e-05 2024-09-11T06:51:38.017 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.017 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.017 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:38.017 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.106102+0000", 2024-09-11T06:51:38.017 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3627000000000006e-05 2024-09-11T06:51:38.017 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.017 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.017 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:38.018 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.106126+0000", 2024-09-11T06:51:38.018 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3676000000000002e-05 2024-09-11T06:51:38.018 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.018 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.018 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.018 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.106278+0000", 2024-09-11T06:51:38.018 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00015224800000000001 2024-09-11T06:51:38.018 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.018 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.018 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:38.018 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.106316+0000", 2024-09-11T06:51:38.018 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.8418999999999999e-05 2024-09-11T06:51:38.018 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.018 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.019 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:38.019 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.109781+0000", 2024-09-11T06:51:38.019 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034645040000000002 2024-09-11T06:51:38.019 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.019 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.019 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:38.019 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.109883+0000", 2024-09-11T06:51:38.019 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000102383 2024-09-11T06:51:38.019 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.019 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:38.019 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.019 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.019 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.020 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5723.0:82 2.4 2:33197ea5:ns::rbd_data.11adc335555f.0000000000000001:head [delete] snapc 4=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e111)", 2024-09-11T06:51:38.020 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:34.289044+0000", 2024-09-11T06:51:38.020 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 3.6298027980000001, 2024-09-11T06:51:38.020 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0064595520000000003, 2024-09-11T06:51:38.020 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:38.020 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:38.020 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:38.020 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:38.020 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5723", 2024-09-11T06:51:38.020 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3725019196", 2024-09-11T06:51:38.020 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 82 2024-09-11T06:51:38.020 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.020 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:38.021 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.021 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:38.021 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.289044+0000", 2024-09-11T06:51:38.021 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.021 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.021 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.021 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:38.021 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.289044+0000", 2024-09-11T06:51:38.021 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.021 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.021 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.021 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:38.021 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.289046+0000", 2024-09-11T06:51:38.021 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.903e-06 2024-09-11T06:51:38.022 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.022 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.022 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:38.022 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.289052+0000", 2024-09-11T06:51:38.022 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7969999999999997e-06 2024-09-11T06:51:38.022 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.022 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.022 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:38.022 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.289066+0000", 2024-09-11T06:51:38.022 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3521e-05 2024-09-11T06:51:38.022 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.022 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.022 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:38.023 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.289073+0000", 2024-09-11T06:51:38.023 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.8180000000000004e-06 2024-09-11T06:51:38.023 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.023 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.023 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.023 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.289184+0000", 2024-09-11T06:51:38.023 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000110561 2024-09-11T06:51:38.023 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.023 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.023 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:38.023 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.289375+0000", 2024-09-11T06:51:38.023 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019082100000000001 2024-09-11T06:51:38.023 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.023 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.023 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:38.024 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.289557+0000", 2024-09-11T06:51:38.024 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00018186999999999999 2024-09-11T06:51:38.024 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.024 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.024 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:38.024 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.292237+0000", 2024-09-11T06:51:38.024 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0026806849999999999 2024-09-11T06:51:38.024 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.024 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.024 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:38.024 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.295362+0000", 2024-09-11T06:51:38.024 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0031244549999999999 2024-09-11T06:51:38.024 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.025 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.025 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:38.025 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.295444+0000", 2024-09-11T06:51:38.025 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2074000000000001e-05 2024-09-11T06:51:38.025 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.025 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.025 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:38.025 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.295504+0000", 2024-09-11T06:51:38.025 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0047000000000003e-05 2024-09-11T06:51:38.025 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.025 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:38.025 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.025 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.025 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.026 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5723.0:93 2.4 2:272abc33:ns::rbd_data.11adc335555f.000000000000000c:head [delete] snapc 4=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e111)", 2024-09-11T06:51:38.026 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:34.295765+0000", 2024-09-11T06:51:38.026 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 3.6230816689999998, 2024-09-11T06:51:38.026 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052823560000000002, 2024-09-11T06:51:38.026 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:38.026 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:38.026 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:38.026 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:38.026 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5723", 2024-09-11T06:51:38.026 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3725019196", 2024-09-11T06:51:38.026 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 93 2024-09-11T06:51:38.026 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.026 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:38.027 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.027 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:38.027 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.295765+0000", 2024-09-11T06:51:38.027 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.027 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.027 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.027 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:38.027 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.295765+0000", 2024-09-11T06:51:38.027 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.027 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.027 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.028 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:38.028 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.295767+0000", 2024-09-11T06:51:38.028 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2860000000000001e-06 2024-09-11T06:51:38.028 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.028 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.028 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:38.028 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.295772+0000", 2024-09-11T06:51:38.028 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.378e-06 2024-09-11T06:51:38.028 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.028 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.028 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:38.029 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.295786+0000", 2024-09-11T06:51:38.029 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3543999999999999e-05 2024-09-11T06:51:38.029 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.029 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.029 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:38.029 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.295792+0000", 2024-09-11T06:51:38.029 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5509999999999996e-06 2024-09-11T06:51:38.029 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.029 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.029 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.029 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.295851+0000", 2024-09-11T06:51:38.029 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8649e-05 2024-09-11T06:51:38.029 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.030 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.030 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:38.030 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.296018+0000", 2024-09-11T06:51:38.030 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016670399999999999 2024-09-11T06:51:38.030 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.030 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.030 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:38.030 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.296167+0000", 2024-09-11T06:51:38.030 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014995300000000001 2024-09-11T06:51:38.031 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.031 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.031 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:38.031 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298223+0000", 2024-09-11T06:51:38.031 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0020557430000000001 2024-09-11T06:51:38.031 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.031 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.031 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:38.031 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.300936+0000", 2024-09-11T06:51:38.031 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0027126670000000002 2024-09-11T06:51:38.031 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.031 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.031 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:38.032 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.300982+0000", 2024-09-11T06:51:38.032 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5745999999999999e-05 2024-09-11T06:51:38.032 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.032 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.032 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:38.032 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.301048+0000", 2024-09-11T06:51:38.032 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.6135000000000003e-05 2024-09-11T06:51:38.032 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.032 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:38.032 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.032 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.032 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:38.032 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:38.034 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T06:51:38.163 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:38.164 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:38.164 INFO:teuthology.orchestra.run.smithi089.stdout: "num_ops": 0 2024-09-11T06:51:38.164 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:38.177 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T06:51:38.179 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T06:51:38.301 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:38.301 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [], 2024-09-11T06:51:38.301 INFO:teuthology.orchestra.run.smithi089.stdout: "complaint_time": 30, 2024-09-11T06:51:38.301 INFO:teuthology.orchestra.run.smithi089.stdout: "num_blocked_ops": 0 2024-09-11T06:51:38.301 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:38.315 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T06:51:38.442 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-11T06:51:38.443 INFO:teuthology.orchestra.run.smithi089.stdout: "size": 20, 2024-09-11T06:51:38.443 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 600, 2024-09-11T06:51:38.443 INFO:teuthology.orchestra.run.smithi089.stdout: "ops": [ 2024-09-11T06:51:38.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.443 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-11T06:51:38.443 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:51:38.443 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 147.208467606, 2024-09-11T06:51:38.443 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010553425999999999, 2024-09-11T06:51:38.443 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:38.443 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:38.443 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:38.444 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:38.444 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:51:38.444 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:51:38.444 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 1 2024-09-11T06:51:38.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.444 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:38.444 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.444 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:38.444 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:51:38.444 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.444 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.444 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:38.444 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230268+0000", 2024-09-11T06:51:38.445 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.445 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:38.445 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230274+0000", 2024-09-11T06:51:38.445 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.533e-06 2024-09-11T06:51:38.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.445 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:38.445 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230285+0000", 2024-09-11T06:51:38.445 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0540000000000001e-05 2024-09-11T06:51:38.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.445 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:38.446 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230309+0000", 2024-09-11T06:51:38.446 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.4223000000000002e-05 2024-09-11T06:51:38.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.446 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.446 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:38.446 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230327+0000", 2024-09-11T06:51:38.446 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7592999999999999e-05 2024-09-11T06:51:38.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.446 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.446 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.446 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230478+0000", 2024-09-11T06:51:38.446 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000151459 2024-09-11T06:51:38.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.446 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.447 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:38.447 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.230490+0000", 2024-09-11T06:51:38.447 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.147e-05 2024-09-11T06:51:38.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.447 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.447 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231803+0000", 2024-09-11T06:51:38.447 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001313158 2024-09-11T06:51:38.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.447 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for peered", 2024-09-11T06:51:38.447 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.231817+0000", 2024-09-11T06:51:38.447 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4197e-05 2024-09-11T06:51:38.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.448 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.448 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.235945+0000", 2024-09-11T06:51:38.448 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0041282480000000002 2024-09-11T06:51:38.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.448 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:38.448 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236312+0000", 2024-09-11T06:51:38.448 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00036652700000000001 2024-09-11T06:51:38.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.448 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:38.448 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.236689+0000", 2024-09-11T06:51:38.449 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000377117 2024-09-11T06:51:38.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.449 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:38.449 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.238460+0000", 2024-09-11T06:51:38.449 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001770918 2024-09-11T06:51:38.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.449 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:38.449 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240627+0000", 2024-09-11T06:51:38.449 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021670779999999998 2024-09-11T06:51:38.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.449 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:38.450 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240751+0000", 2024-09-11T06:51:38.450 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00012384900000000001 2024-09-11T06:51:38.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.450 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.450 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:38.450 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:11.240821+0000", 2024-09-11T06:51:38.450 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0516e-05 2024-09-11T06:51:38.450 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.450 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:38.450 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.450 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.450 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-11T06:51:38.450 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:51:38.450 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 146.20629676300001, 2024-09-11T06:51:38.451 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069904770000000001, 2024-09-11T06:51:38.451 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:38.451 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:38.451 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:38.451 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:38.451 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:51:38.451 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:51:38.451 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 5 2024-09-11T06:51:38.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.451 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:38.451 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.451 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:38.451 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:51:38.451 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.452 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:38.452 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232439+0000", 2024-09-11T06:51:38.452 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.452 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:38.452 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232442+0000", 2024-09-11T06:51:38.452 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.044e-06 2024-09-11T06:51:38.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.452 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:38.452 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232460+0000", 2024-09-11T06:51:38.453 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7863000000000002e-05 2024-09-11T06:51:38.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.453 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:38.453 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232492+0000", 2024-09-11T06:51:38.453 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2812999999999999e-05 2024-09-11T06:51:38.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.453 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:38.453 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232510+0000", 2024-09-11T06:51:38.453 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7552999999999999e-05 2024-09-11T06:51:38.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.453 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.454 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232640+0000", 2024-09-11T06:51:38.454 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013029900000000001 2024-09-11T06:51:38.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.454 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:38.454 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.232650+0000", 2024-09-11T06:51:38.454 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0023999999999999e-05 2024-09-11T06:51:38.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.454 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.454 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235641+0000", 2024-09-11T06:51:38.454 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00299089 2024-09-11T06:51:38.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.455 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:38.455 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.235980+0000", 2024-09-11T06:51:38.455 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000338907 2024-09-11T06:51:38.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.455 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:38.455 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.236351+0000", 2024-09-11T06:51:38.455 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00037087699999999998 2024-09-11T06:51:38.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.455 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:38.455 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.238074+0000", 2024-09-11T06:51:38.456 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001722859 2024-09-11T06:51:38.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.456 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:38.456 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239160+0000", 2024-09-11T06:51:38.456 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001086074 2024-09-11T06:51:38.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.456 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:38.456 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239299+0000", 2024-09-11T06:51:38.456 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013933500000000001 2024-09-11T06:51:38.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.457 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.457 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:38.457 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:12.239429+0000", 2024-09-11T06:51:38.457 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000129939 2024-09-11T06:51:38.457 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.457 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:38.457 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.457 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.457 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.4230.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-11T06:51:38.457 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:51:38.458 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 145.192208811, 2024-09-11T06:51:38.458 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0069253250000000004, 2024-09-11T06:51:38.458 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:38.458 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:38.458 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:38.458 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:38.458 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.4230", 2024-09-11T06:51:38.458 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.71:0/3403166027", 2024-09-11T06:51:38.458 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 7 2024-09-11T06:51:38.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.458 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:38.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.459 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:38.459 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:51:38.459 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.459 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:38.459 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246527+0000", 2024-09-11T06:51:38.459 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.459 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:38.460 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246530+0000", 2024-09-11T06:51:38.460 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9469999999999999e-06 2024-09-11T06:51:38.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.460 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.460 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:38.460 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246545+0000", 2024-09-11T06:51:38.460 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5973000000000001e-05 2024-09-11T06:51:38.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.460 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.460 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:38.460 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246578+0000", 2024-09-11T06:51:38.460 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2759999999999998e-05 2024-09-11T06:51:38.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.461 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.461 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:38.461 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246595+0000", 2024-09-11T06:51:38.461 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6939000000000002e-05 2024-09-11T06:51:38.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.461 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.461 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.461 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246770+0000", 2024-09-11T06:51:38.461 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000174723 2024-09-11T06:51:38.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.462 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.462 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:38.462 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.246789+0000", 2024-09-11T06:51:38.462 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9159000000000001e-05 2024-09-11T06:51:38.462 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.462 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.462 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.462 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250337+0000", 2024-09-11T06:51:38.462 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035480759999999998 2024-09-11T06:51:38.462 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.462 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.463 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:38.463 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250549+0000", 2024-09-11T06:51:38.463 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00021192199999999999 2024-09-11T06:51:38.463 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.463 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.463 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 1", 2024-09-11T06:51:38.463 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.250750+0000", 2024-09-11T06:51:38.463 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00020104199999999999 2024-09-11T06:51:38.463 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.463 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.463 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:38.463 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.252455+0000", 2024-09-11T06:51:38.464 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001705249 2024-09-11T06:51:38.464 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.464 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.464 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:38.464 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253305+0000", 2024-09-11T06:51:38.464 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00084941200000000002 2024-09-11T06:51:38.464 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.464 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.464 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:38.464 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253388+0000", 2024-09-11T06:51:38.465 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.3642999999999999e-05 2024-09-11T06:51:38.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.465 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.465 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:38.465 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:49:13.253452+0000", 2024-09-11T06:51:38.465 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.3479999999999995e-05 2024-09-11T06:51:38.465 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.465 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:38.465 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.465 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.465 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5544.0:57 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-11T06:51:38.466 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:50:48.568529+0000", 2024-09-11T06:51:38.466 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 49.870205890999998, 2024-09-11T06:51:38.466 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.004228229, 2024-09-11T06:51:38.466 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:38.466 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:38.466 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:38.466 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:38.466 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5544", 2024-09-11T06:51:38.466 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3502894973", 2024-09-11T06:51:38.466 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 57 2024-09-11T06:51:38.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.466 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:38.467 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.467 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:38.467 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568529+0000", 2024-09-11T06:51:38.467 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.467 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.467 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:38.467 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568529+0000", 2024-09-11T06:51:38.467 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.467 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.468 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:38.468 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568532+0000", 2024-09-11T06:51:38.468 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.0000000000000001e-06 2024-09-11T06:51:38.468 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.468 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.468 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:38.468 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568547+0000", 2024-09-11T06:51:38.468 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4219999999999999e-05 2024-09-11T06:51:38.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.469 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.469 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:38.469 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568580+0000", 2024-09-11T06:51:38.469 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.3328999999999999e-05 2024-09-11T06:51:38.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.469 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.469 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:38.469 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568596+0000", 2024-09-11T06:51:38.469 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5990000000000001e-05 2024-09-11T06:51:38.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.469 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.470 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.470 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568728+0000", 2024-09-11T06:51:38.470 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013175400000000001 2024-09-11T06:51:38.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.470 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.470 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:38.470 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.568738+0000", 2024-09-11T06:51:38.470 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0196e-05 2024-09-11T06:51:38.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.470 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.470 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.471 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.572464+0000", 2024-09-11T06:51:38.471 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0037264160000000002 2024-09-11T06:51:38.471 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.471 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.471 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:38.471 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.572584+0000", 2024-09-11T06:51:38.471 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000119438 2024-09-11T06:51:38.471 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.471 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.471 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:38.471 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:50:48.572758+0000", 2024-09-11T06:51:38.472 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017388599999999999 2024-09-11T06:51:38.472 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.472 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:38.472 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.472 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.472 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5386.0:17 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [watch unwatch cookie 94226507267584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:38.472 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:12.563293+0000", 2024-09-11T06:51:38.472 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 25.875442149000001, 2024-09-11T06:51:38.472 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0028623239999999999, 2024-09-11T06:51:38.473 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:38.473 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:38.473 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:38.473 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:38.473 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5386", 2024-09-11T06:51:38.473 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3377056437", 2024-09-11T06:51:38.473 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 17 2024-09-11T06:51:38.473 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.473 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:38.473 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.474 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:38.474 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563293+0000", 2024-09-11T06:51:38.474 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.474 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.474 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:38.474 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563293+0000", 2024-09-11T06:51:38.474 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.474 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.474 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:38.475 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563294+0000", 2024-09-11T06:51:38.475 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.5300000000000004e-07 2024-09-11T06:51:38.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.475 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:38.475 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563296+0000", 2024-09-11T06:51:38.475 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6079999999999998e-06 2024-09-11T06:51:38.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.475 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:38.475 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563302+0000", 2024-09-11T06:51:38.475 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2549999999999997e-06 2024-09-11T06:51:38.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.476 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:38.476 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563305+0000", 2024-09-11T06:51:38.476 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2949999999999998e-06 2024-09-11T06:51:38.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.476 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.476 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563792+0000", 2024-09-11T06:51:38.476 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00048731399999999998 2024-09-11T06:51:38.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.476 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:38.476 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563899+0000", 2024-09-11T06:51:38.476 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000106977 2024-09-11T06:51:38.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.477 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:38.477 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.564145+0000", 2024-09-11T06:51:38.477 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00024587600000000002 2024-09-11T06:51:38.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.477 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:38.477 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.564760+0000", 2024-09-11T06:51:38.477 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00061473099999999998 2024-09-11T06:51:38.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.477 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:38.477 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566046+0000", 2024-09-11T06:51:38.478 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012856810000000001 2024-09-11T06:51:38.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.478 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:38.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566085+0000", 2024-09-11T06:51:38.478 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.9499000000000002e-05 2024-09-11T06:51:38.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.478 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:38.478 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566156+0000", 2024-09-11T06:51:38.478 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.0434999999999999e-05 2024-09-11T06:51:38.478 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.478 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:38.478 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.479 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.479 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5826.0:131 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.client_list in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-11T06:51:38.479 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:12.563833+0000", 2024-09-11T06:51:38.479 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 25.874902593000002, 2024-09-11T06:51:38.479 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0028764329999999999, 2024-09-11T06:51:38.479 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:38.479 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:38.479 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:38.479 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:38.479 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5826", 2024-09-11T06:51:38.479 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/2236190565", 2024-09-11T06:51:38.479 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 131 2024-09-11T06:51:38.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.480 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:38.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.480 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:38.480 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563833+0000", 2024-09-11T06:51:38.480 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.480 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:38.480 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563833+0000", 2024-09-11T06:51:38.480 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.480 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:38.481 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563835+0000", 2024-09-11T06:51:38.481 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8109999999999999e-06 2024-09-11T06:51:38.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.481 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:38.481 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563840+0000", 2024-09-11T06:51:38.481 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7599999999999999e-06 2024-09-11T06:51:38.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.481 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:38.481 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563854+0000", 2024-09-11T06:51:38.481 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3332999999999999e-05 2024-09-11T06:51:38.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.482 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:38.482 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.563874+0000", 2024-09-11T06:51:38.482 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0630999999999999e-05 2024-09-11T06:51:38.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.482 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.482 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.564553+0000", 2024-09-11T06:51:38.482 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00067821199999999998 2024-09-11T06:51:38.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.482 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for rw locks", 2024-09-11T06:51:38.482 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.564610+0000", 2024-09-11T06:51:38.483 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.7494999999999998e-05 2024-09-11T06:51:38.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.483 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.483 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.483 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566240+0000", 2024-09-11T06:51:38.483 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0016295940000000001 2024-09-11T06:51:38.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.483 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.483 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:38.483 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566365+0000", 2024-09-11T06:51:38.483 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000124876 2024-09-11T06:51:38.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.483 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.484 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:38.484 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:12.566709+0000", 2024-09-11T06:51:38.484 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00034472099999999998 2024-09-11T06:51:38.484 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.484 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:38.484 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.484 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.484 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5447.0:10 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T06:51:38.484 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:13.624024+0000", 2024-09-11T06:51:38.484 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 24.8147117, 2024-09-11T06:51:38.484 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0038665190000000001, 2024-09-11T06:51:38.484 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:38.485 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:38.485 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:38.485 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:38.485 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5447", 2024-09-11T06:51:38.485 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3862443986", 2024-09-11T06:51:38.485 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 10 2024-09-11T06:51:38.485 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.485 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:38.485 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.485 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:38.485 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624024+0000", 2024-09-11T06:51:38.485 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.485 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.485 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.485 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:38.486 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624024+0000", 2024-09-11T06:51:38.486 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.486 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.486 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.486 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:38.486 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624026+0000", 2024-09-11T06:51:38.486 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8090000000000001e-06 2024-09-11T06:51:38.486 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.486 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.486 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:38.486 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624037+0000", 2024-09-11T06:51:38.486 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.061e-05 2024-09-11T06:51:38.486 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.486 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.487 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:38.487 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624059+0000", 2024-09-11T06:51:38.487 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.2269e-05 2024-09-11T06:51:38.487 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.487 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.487 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:38.487 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624074+0000", 2024-09-11T06:51:38.487 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4982e-05 2024-09-11T06:51:38.487 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.487 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.487 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.487 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624238+0000", 2024-09-11T06:51:38.487 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000163191 2024-09-11T06:51:38.487 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.488 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.488 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:38.488 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.624261+0000", 2024-09-11T06:51:38.488 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3779000000000001e-05 2024-09-11T06:51:38.488 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.488 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.488 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.488 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.627590+0000", 2024-09-11T06:51:38.488 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033287450000000001 2024-09-11T06:51:38.488 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.488 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.489 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:38.489 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.627723+0000", 2024-09-11T06:51:38.489 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013311599999999999 2024-09-11T06:51:38.489 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.489 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.489 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:38.489 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:13.627890+0000", 2024-09-11T06:51:38.489 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016701800000000001 2024-09-11T06:51:38.489 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.489 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:38.490 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.490 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.490 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.490 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.6081.0:32 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.set_minimum_set in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-11T06:51:38.490 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.541248+0000", 2024-09-11T06:51:38.490 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.897487373000001, 2024-09-11T06:51:38.490 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0028300869999999998, 2024-09-11T06:51:38.490 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:38.491 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:38.491 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:38.491 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:38.491 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.6081", 2024-09-11T06:51:38.491 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3800899964", 2024-09-11T06:51:38.491 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 32 2024-09-11T06:51:38.491 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.491 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:38.491 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.491 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:38.491 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541248+0000", 2024-09-11T06:51:38.491 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.492 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.492 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.492 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:38.492 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541248+0000", 2024-09-11T06:51:38.492 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.492 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.492 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.492 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:38.492 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541251+0000", 2024-09-11T06:51:38.492 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.5299999999999999e-06 2024-09-11T06:51:38.492 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.492 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.492 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:38.493 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541266+0000", 2024-09-11T06:51:38.493 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5139e-05 2024-09-11T06:51:38.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.493 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.493 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:38.493 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541298+0000", 2024-09-11T06:51:38.493 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2152e-05 2024-09-11T06:51:38.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.493 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.493 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:38.493 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541313+0000", 2024-09-11T06:51:38.493 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5532999999999998e-05 2024-09-11T06:51:38.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.494 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.494 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.494 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541554+0000", 2024-09-11T06:51:38.494 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000240306 2024-09-11T06:51:38.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.494 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.494 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:38.494 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541957+0000", 2024-09-11T06:51:38.494 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00040357799999999998 2024-09-11T06:51:38.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.494 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.494 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:38.494 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.544078+0000", 2024-09-11T06:51:38.495 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021208490000000002 2024-09-11T06:51:38.495 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.495 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:38.495 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.495 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.495 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.495 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.6081.0:33 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-11T06:51:38.495 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:20.541352+0000", 2024-09-11T06:51:38.495 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 17.897383248000001, 2024-09-11T06:51:38.495 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034084850000000002, 2024-09-11T06:51:38.495 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:38.495 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:38.495 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:38.495 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:38.496 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.6081", 2024-09-11T06:51:38.496 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3800899964", 2024-09-11T06:51:38.496 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 33 2024-09-11T06:51:38.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.496 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:38.496 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.496 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:38.496 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541352+0000", 2024-09-11T06:51:38.496 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.496 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.496 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:38.496 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541352+0000", 2024-09-11T06:51:38.497 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.497 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.497 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:38.497 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541353+0000", 2024-09-11T06:51:38.497 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.7000000000000003e-07 2024-09-11T06:51:38.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.497 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.497 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:38.497 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541358+0000", 2024-09-11T06:51:38.497 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.7400000000000004e-06 2024-09-11T06:51:38.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.497 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.498 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:38.498 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541374+0000", 2024-09-11T06:51:38.498 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.5670000000000001e-05 2024-09-11T06:51:38.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.498 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.498 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:38.498 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.541386+0000", 2024-09-11T06:51:38.498 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2359e-05 2024-09-11T06:51:38.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.498 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.498 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.498 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542684+0000", 2024-09-11T06:51:38.498 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012984590000000001 2024-09-11T06:51:38.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.499 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.499 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:38.499 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542820+0000", 2024-09-11T06:51:38.499 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013536700000000001 2024-09-11T06:51:38.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.499 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.499 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:38.499 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.542972+0000", 2024-09-11T06:51:38.499 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000151843 2024-09-11T06:51:38.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.499 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.499 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:38.499 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.544110+0000", 2024-09-11T06:51:38.500 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011381060000000001 2024-09-11T06:51:38.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.500 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.500 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:38.500 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.544510+0000", 2024-09-11T06:51:38.500 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00040019799999999999 2024-09-11T06:51:38.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.500 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.500 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:38.500 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.544643+0000", 2024-09-11T06:51:38.500 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013313499999999999 2024-09-11T06:51:38.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.500 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.501 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:38.501 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:20.544761+0000", 2024-09-11T06:51:38.501 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000117638 2024-09-11T06:51:38.501 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.501 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:38.501 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.501 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.501 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.501 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5542.0:10 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-11T06:51:38.501 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:22.002114+0000", 2024-09-11T06:51:38.501 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 16.436621799000001, 2024-09-11T06:51:38.501 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00564235, 2024-09-11T06:51:38.501 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:38.501 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:38.502 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:38.502 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:38.502 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5542", 2024-09-11T06:51:38.502 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1546942093", 2024-09-11T06:51:38.502 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 10 2024-09-11T06:51:38.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.502 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:38.502 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.502 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:38.502 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002114+0000", 2024-09-11T06:51:38.502 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.502 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.502 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:38.503 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002114+0000", 2024-09-11T06:51:38.503 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.503 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:38.503 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002115+0000", 2024-09-11T06:51:38.503 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.872e-06 2024-09-11T06:51:38.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.503 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:38.503 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002123+0000", 2024-09-11T06:51:38.503 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.3309999999999998e-06 2024-09-11T06:51:38.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.504 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:38.504 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002136+0000", 2024-09-11T06:51:38.504 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2816000000000001e-05 2024-09-11T06:51:38.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.504 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.504 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:38.504 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002145+0000", 2024-09-11T06:51:38.504 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.8570000000000007e-06 2024-09-11T06:51:38.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.504 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.504 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.504 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002214+0000", 2024-09-11T06:51:38.504 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.8832999999999996e-05 2024-09-11T06:51:38.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.505 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:38.505 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.002220+0000", 2024-09-11T06:51:38.505 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.4249999999999997e-06 2024-09-11T06:51:38.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.505 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.505 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.007445+0000", 2024-09-11T06:51:38.505 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0052251659999999998 2024-09-11T06:51:38.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.505 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:38.506 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.007586+0000", 2024-09-11T06:51:38.506 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00014103100000000001 2024-09-11T06:51:38.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.506 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.506 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:38.506 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.007756+0000", 2024-09-11T06:51:38.506 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017001900000000001 2024-09-11T06:51:38.506 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.506 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:38.506 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.506 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.506 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5542.0:21 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.create in=13b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-11T06:51:38.507 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:22.028810+0000", 2024-09-11T06:51:38.507 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 16.409925755, 2024-09-11T06:51:38.507 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0034172709999999999, 2024-09-11T06:51:38.507 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:38.507 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:38.507 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:38.507 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:38.507 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5542", 2024-09-11T06:51:38.507 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1546942093", 2024-09-11T06:51:38.507 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 21 2024-09-11T06:51:38.507 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.507 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:38.507 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.507 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:38.508 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.028810+0000", 2024-09-11T06:51:38.508 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.508 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.508 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:38.508 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.028810+0000", 2024-09-11T06:51:38.508 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.508 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.508 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:38.508 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.028812+0000", 2024-09-11T06:51:38.508 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.8100000000000002e-06 2024-09-11T06:51:38.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.509 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.509 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:38.509 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.028826+0000", 2024-09-11T06:51:38.509 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.3882e-05 2024-09-11T06:51:38.509 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.509 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.509 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:38.509 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.028859+0000", 2024-09-11T06:51:38.509 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2225999999999999e-05 2024-09-11T06:51:38.509 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.509 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.509 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:38.509 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.028880+0000", 2024-09-11T06:51:38.510 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1755000000000001e-05 2024-09-11T06:51:38.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.510 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.510 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.029048+0000", 2024-09-11T06:51:38.510 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016811699999999999 2024-09-11T06:51:38.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.510 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:38.510 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.029198+0000", 2024-09-11T06:51:38.510 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000149753 2024-09-11T06:51:38.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.510 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:38.511 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.029612+0000", 2024-09-11T06:51:38.511 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00041426899999999999 2024-09-11T06:51:38.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.511 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.511 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:38.511 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.030882+0000", 2024-09-11T06:51:38.511 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012693030000000001 2024-09-11T06:51:38.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.511 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.511 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:38.511 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.032087+0000", 2024-09-11T06:51:38.511 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0012054279999999999 2024-09-11T06:51:38.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.512 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.512 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:38.512 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.032167+0000", 2024-09-11T06:51:38.512 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.0177000000000002e-05 2024-09-11T06:51:38.512 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.512 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.512 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:38.512 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.032227+0000", 2024-09-11T06:51:38.512 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.9551e-05 2024-09-11T06:51:38.512 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.512 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:38.512 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.512 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.513 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.513 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5542.0:23 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.tag_create in=17b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-11T06:51:38.513 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:22.033580+0000", 2024-09-11T06:51:38.513 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 16.405155305000001, 2024-09-11T06:51:38.513 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.002804384, 2024-09-11T06:51:38.513 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:38.513 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:38.513 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:38.513 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:38.513 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5542", 2024-09-11T06:51:38.513 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/1546942093", 2024-09-11T06:51:38.513 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 23 2024-09-11T06:51:38.513 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.514 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:38.514 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.514 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:38.514 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033580+0000", 2024-09-11T06:51:38.514 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.514 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.514 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.514 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:38.514 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033580+0000", 2024-09-11T06:51:38.514 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.514 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.514 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.514 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:38.515 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033581+0000", 2024-09-11T06:51:38.515 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.217e-06 2024-09-11T06:51:38.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.515 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.515 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:38.515 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033591+0000", 2024-09-11T06:51:38.515 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.2660000000000007e-06 2024-09-11T06:51:38.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.515 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.515 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:38.515 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033611+0000", 2024-09-11T06:51:38.515 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0418999999999999e-05 2024-09-11T06:51:38.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.515 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.516 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:38.516 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033628+0000", 2024-09-11T06:51:38.516 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7316000000000002e-05 2024-09-11T06:51:38.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.516 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.516 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.516 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033803+0000", 2024-09-11T06:51:38.516 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00017436200000000001 2024-09-11T06:51:38.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.516 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.516 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:38.516 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.033944+0000", 2024-09-11T06:51:38.516 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000141688 2024-09-11T06:51:38.517 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.517 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.517 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:38.517 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.034475+0000", 2024-09-11T06:51:38.517 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00053032399999999999 2024-09-11T06:51:38.517 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.517 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.517 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:38.517 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.035684+0000", 2024-09-11T06:51:38.517 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001209452 2024-09-11T06:51:38.517 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.517 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.517 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:38.518 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.036254+0000", 2024-09-11T06:51:38.518 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00056982599999999995 2024-09-11T06:51:38.518 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.518 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.518 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:38.518 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.036328+0000", 2024-09-11T06:51:38.518 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.4499999999999995e-05 2024-09-11T06:51:38.518 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.518 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.518 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:38.518 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:22.036384+0000", 2024-09-11T06:51:38.518 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.6014000000000003e-05 2024-09-11T06:51:38.518 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.518 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:38.519 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.519 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.519 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.519 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5536.0:76 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-11T06:51:38.519 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:25.786870+0000", 2024-09-11T06:51:38.519 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 12.651865216999999, 2024-09-11T06:51:38.519 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0038803930000000002, 2024-09-11T06:51:38.519 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:38.519 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:38.519 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:38.519 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:38.519 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5536", 2024-09-11T06:51:38.520 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/612379138", 2024-09-11T06:51:38.520 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 76 2024-09-11T06:51:38.520 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.520 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:38.520 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.520 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:38.520 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.786870+0000", 2024-09-11T06:51:38.520 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.520 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.520 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.520 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:38.520 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.786870+0000", 2024-09-11T06:51:38.520 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.521 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.521 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.521 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:38.521 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.786873+0000", 2024-09-11T06:51:38.521 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2729999999999998e-06 2024-09-11T06:51:38.521 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.521 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.521 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:38.521 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.786891+0000", 2024-09-11T06:51:38.521 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7153e-05 2024-09-11T06:51:38.521 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.521 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.521 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:38.521 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.786925+0000", 2024-09-11T06:51:38.522 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.4227999999999998e-05 2024-09-11T06:51:38.522 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.522 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.522 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:38.522 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.786941+0000", 2024-09-11T06:51:38.522 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.6413e-05 2024-09-11T06:51:38.522 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.522 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.522 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.522 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.787108+0000", 2024-09-11T06:51:38.522 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016641 2024-09-11T06:51:38.522 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.523 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.523 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:38.523 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.787132+0000", 2024-09-11T06:51:38.523 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.3946000000000001e-05 2024-09-11T06:51:38.523 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.523 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.523 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.523 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.790458+0000", 2024-09-11T06:51:38.523 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0033267790000000002 2024-09-11T06:51:38.523 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.523 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.523 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:38.523 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.790589+0000", 2024-09-11T06:51:38.524 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013019400000000001 2024-09-11T06:51:38.524 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.524 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.524 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:38.524 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.790751+0000", 2024-09-11T06:51:38.524 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00016199699999999999 2024-09-11T06:51:38.524 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.524 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:38.524 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.524 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.524 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.524 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5536.0:99 2.3 2:c408cb7c:::journal.1314f0a2bfa2:head [call journal.tag_create in=17b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-11T06:51:38.524 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:25.830334+0000", 2024-09-11T06:51:38.525 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 12.608400913000001, 2024-09-11T06:51:38.525 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0028281019999999999, 2024-09-11T06:51:38.525 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:38.525 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:38.525 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:38.525 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:38.525 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5536", 2024-09-11T06:51:38.525 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/612379138", 2024-09-11T06:51:38.525 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 99 2024-09-11T06:51:38.525 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.525 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:38.525 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.525 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:38.526 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.830334+0000", 2024-09-11T06:51:38.526 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.526 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.526 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.526 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:38.526 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.830334+0000", 2024-09-11T06:51:38.526 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.526 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.526 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.526 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:38.526 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.830337+0000", 2024-09-11T06:51:38.526 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1100000000000001e-06 2024-09-11T06:51:38.526 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.527 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.527 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:38.527 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.830347+0000", 2024-09-11T06:51:38.527 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.075e-05 2024-09-11T06:51:38.527 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.527 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.527 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:38.527 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.830380+0000", 2024-09-11T06:51:38.527 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.2191999999999999e-05 2024-09-11T06:51:38.527 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.527 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.527 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:38.527 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.830400+0000", 2024-09-11T06:51:38.528 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.0886e-05 2024-09-11T06:51:38.528 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.528 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.528 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.528 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.830537+0000", 2024-09-11T06:51:38.528 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000136964 2024-09-11T06:51:38.528 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.528 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.528 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:38.528 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.830762+0000", 2024-09-11T06:51:38.528 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00022447100000000001 2024-09-11T06:51:38.528 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.528 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.529 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:38.529 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.831663+0000", 2024-09-11T06:51:38.529 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00090116800000000004 2024-09-11T06:51:38.529 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.529 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.529 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:38.529 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.832966+0000", 2024-09-11T06:51:38.529 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001303305 2024-09-11T06:51:38.529 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.529 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.529 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:38.529 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.832992+0000", 2024-09-11T06:51:38.529 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.6126e-05 2024-09-11T06:51:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.833096+0000", 2024-09-11T06:51:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000103082 2024-09-11T06:51:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:25.833163+0000", 2024-09-11T06:51:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7047999999999999e-05 2024-09-11T06:51:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.531 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5660.0:32 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+full_try+supports_pool_eio e107)", 2024-09-11T06:51:38.531 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:30.113546+0000", 2024-09-11T06:51:38.531 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 8.3251893530000007, 2024-09-11T06:51:38.531 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0057805510000000001, 2024-09-11T06:51:38.531 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:38.531 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:38.531 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:38.531 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:38.531 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5660", 2024-09-11T06:51:38.531 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3214439167", 2024-09-11T06:51:38.531 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 32 2024-09-11T06:51:38.531 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.531 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:38.531 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.532 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:38.532 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.113546+0000", 2024-09-11T06:51:38.532 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.532 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.532 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.532 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:38.532 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.113546+0000", 2024-09-11T06:51:38.532 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.532 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.532 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.532 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:38.532 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.113551+0000", 2024-09-11T06:51:38.532 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.8230000000000002e-06 2024-09-11T06:51:38.532 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.533 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.533 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:38.533 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.113569+0000", 2024-09-11T06:51:38.533 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.8436000000000001e-05 2024-09-11T06:51:38.533 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.533 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.533 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:38.533 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.113621+0000", 2024-09-11T06:51:38.533 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.1452000000000001e-05 2024-09-11T06:51:38.533 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.533 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.533 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:38.533 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.113648+0000", 2024-09-11T06:51:38.534 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.7379e-05 2024-09-11T06:51:38.534 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.534 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.534 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.534 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.113844+0000", 2024-09-11T06:51:38.534 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00019605600000000001 2024-09-11T06:51:38.534 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.534 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.534 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:38.534 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.113862+0000", 2024-09-11T06:51:38.534 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7917000000000001e-05 2024-09-11T06:51:38.534 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.534 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.535 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.535 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.118846+0000", 2024-09-11T06:51:38.535 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0049840209999999999 2024-09-11T06:51:38.535 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.535 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.535 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:38.535 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.119030+0000", 2024-09-11T06:51:38.535 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000184067 2024-09-11T06:51:38.535 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.535 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.535 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:38.535 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:30.119327+0000", 2024-09-11T06:51:38.535 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00029639999999999999 2024-09-11T06:51:38.536 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.536 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:38.536 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.536 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.536 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5660.0:53 2.2 2:4510a54f:::rbd_object_map.1314f0a2bfa2.0000000000000007:head [call rbd.object_map_snap_remove in=25b] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e108)", 2024-09-11T06:51:38.536 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:31.138066+0000", 2024-09-11T06:51:38.536 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 7.3006694249999997, 2024-09-11T06:51:38.536 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003789766, 2024-09-11T06:51:38.536 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:38.536 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:38.536 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:38.537 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:38.537 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5660", 2024-09-11T06:51:38.537 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3214439167", 2024-09-11T06:51:38.537 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 53 2024-09-11T06:51:38.537 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.537 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:38.537 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.537 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:38.537 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:31.138066+0000", 2024-09-11T06:51:38.537 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.537 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.537 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.537 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:38.537 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:31.138066+0000", 2024-09-11T06:51:38.538 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.538 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.538 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.538 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:38.538 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:31.138068+0000", 2024-09-11T06:51:38.538 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.9290000000000001e-06 2024-09-11T06:51:38.538 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.538 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.538 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:38.538 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:31.138075+0000", 2024-09-11T06:51:38.538 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1180000000000003e-06 2024-09-11T06:51:38.538 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.538 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.539 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:38.539 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:31.138093+0000", 2024-09-11T06:51:38.539 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.7915e-05 2024-09-11T06:51:38.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.539 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.539 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:38.539 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:31.138102+0000", 2024-09-11T06:51:38.539 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.7539999999999995e-06 2024-09-11T06:51:38.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.539 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.539 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.539 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:31.138193+0000", 2024-09-11T06:51:38.539 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 9.1221e-05 2024-09-11T06:51:38.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.540 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.540 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:38.540 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:31.138312+0000", 2024-09-11T06:51:38.540 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00011957599999999999 2024-09-11T06:51:38.540 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.540 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.540 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:38.540 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:31.141856+0000", 2024-09-11T06:51:38.540 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0035432530000000001 2024-09-11T06:51:38.540 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.540 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:38.540 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.540 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.541 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.541 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5723.0:3 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+full_try+supports_pool_eio e111)", 2024-09-11T06:51:38.541 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:34.208333+0000", 2024-09-11T06:51:38.541 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.230402411, 2024-09-11T06:51:38.541 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0029589500000000001, 2024-09-11T06:51:38.541 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:38.541 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:38.541 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "started", 2024-09-11T06:51:38.541 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:38.541 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5723", 2024-09-11T06:51:38.541 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3725019196", 2024-09-11T06:51:38.541 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 3 2024-09-11T06:51:38.541 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.541 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:38.542 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.542 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:38.542 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.208333+0000", 2024-09-11T06:51:38.542 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.542 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.542 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:38.542 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.208333+0000", 2024-09-11T06:51:38.542 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.542 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.542 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:38.542 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.208335+0000", 2024-09-11T06:51:38.543 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.827e-06 2024-09-11T06:51:38.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.543 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.543 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:38.543 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.208341+0000", 2024-09-11T06:51:38.543 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.0419999999999999e-06 2024-09-11T06:51:38.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.543 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.543 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:38.543 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.208353+0000", 2024-09-11T06:51:38.543 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.2058999999999999e-05 2024-09-11T06:51:38.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.543 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.543 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:38.544 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.208361+0000", 2024-09-11T06:51:38.544 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2840000000000006e-06 2024-09-11T06:51:38.544 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.544 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.544 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.544 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.208425+0000", 2024-09-11T06:51:38.544 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.4105999999999999e-05 2024-09-11T06:51:38.544 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.544 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.544 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op must wait for map", 2024-09-11T06:51:38.544 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.208431+0000", 2024-09-11T06:51:38.544 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.857e-06 2024-09-11T06:51:38.544 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.545 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.545 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.545 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.211055+0000", 2024-09-11T06:51:38.545 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0026241139999999999 2024-09-11T06:51:38.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.545 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.545 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:38.545 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.211192+0000", 2024-09-11T06:51:38.545 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00013648299999999999 2024-09-11T06:51:38.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.545 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.545 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:38.545 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.211292+0000", 2024-09-11T06:51:38.545 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000100178 2024-09-11T06:51:38.546 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.546 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:38.546 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.546 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.546 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.546 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5723.0:81 2.2 2:56207b30:ns::rbd_data.11adc335555f.0000000000000000:head [delete] snapc 4=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e111)", 2024-09-11T06:51:38.546 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:34.287933+0000", 2024-09-11T06:51:38.546 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.1508023510000003, 2024-09-11T06:51:38.546 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.010316603000000001, 2024-09-11T06:51:38.546 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:38.546 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:38.546 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:38.546 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:38.546 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5723", 2024-09-11T06:51:38.547 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3725019196", 2024-09-11T06:51:38.547 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 81 2024-09-11T06:51:38.547 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.547 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:38.547 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.547 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:38.547 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.287933+0000", 2024-09-11T06:51:38.547 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.547 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.547 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.547 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:38.547 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.287933+0000", 2024-09-11T06:51:38.547 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.548 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.548 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.548 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:38.548 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.287935+0000", 2024-09-11T06:51:38.548 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.823e-06 2024-09-11T06:51:38.548 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.548 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.548 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:38.548 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.287942+0000", 2024-09-11T06:51:38.548 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 7.1149999999999996e-06 2024-09-11T06:51:38.548 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.548 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.548 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:38.549 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.287964+0000", 2024-09-11T06:51:38.549 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.1747999999999999e-05 2024-09-11T06:51:38.549 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.549 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.549 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:38.549 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.287974+0000", 2024-09-11T06:51:38.549 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0387000000000001e-05 2024-09-11T06:51:38.549 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.549 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.549 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.549 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.288058+0000", 2024-09-11T06:51:38.549 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.4376000000000007e-05 2024-09-11T06:51:38.549 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.550 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:38.550 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.288164+0000", 2024-09-11T06:51:38.550 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.00010584100000000001 2024-09-11T06:51:38.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.550 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:38.550 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.288301+0000", 2024-09-11T06:51:38.550 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000137107 2024-09-11T06:51:38.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.550 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:38.550 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.290499+0000", 2024-09-11T06:51:38.550 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0021976779999999998 2024-09-11T06:51:38.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.551 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.551 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:38.551 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298176+0000", 2024-09-11T06:51:38.551 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0076773259999999999 2024-09-11T06:51:38.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.551 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.551 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:38.551 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298221+0000", 2024-09-11T06:51:38.551 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.4152e-05 2024-09-11T06:51:38.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.551 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.551 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:38.551 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298250+0000", 2024-09-11T06:51:38.552 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 2.9050000000000001e-05 2024-09-11T06:51:38.552 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.552 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:38.552 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.552 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.552 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.552 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5723.0:91 2.2 2:45bb0ce9:ns::rbd_data.11adc335555f.000000000000000a:head [delete] snapc 4=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e111)", 2024-09-11T06:51:38.552 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:34.291968+0000", 2024-09-11T06:51:38.552 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.1467670810000001, 2024-09-11T06:51:38.552 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0066095399999999997, 2024-09-11T06:51:38.552 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:38.552 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:38.552 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:38.553 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:38.553 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5723", 2024-09-11T06:51:38.553 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3725019196", 2024-09-11T06:51:38.553 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 91 2024-09-11T06:51:38.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.553 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:38.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.553 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:38.553 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.291968+0000", 2024-09-11T06:51:38.553 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.553 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:38.554 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.291968+0000", 2024-09-11T06:51:38.554 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.554 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.554 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:38.554 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.291970+0000", 2024-09-11T06:51:38.554 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.668e-06 2024-09-11T06:51:38.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.554 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.554 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:38.554 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.291977+0000", 2024-09-11T06:51:38.554 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7270000000000001e-06 2024-09-11T06:51:38.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.554 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.555 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:38.555 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.291991+0000", 2024-09-11T06:51:38.555 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4521999999999999e-05 2024-09-11T06:51:38.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.555 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.555 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:38.555 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.292000+0000", 2024-09-11T06:51:38.555 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.6640000000000004e-06 2024-09-11T06:51:38.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.555 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.555 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.555 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.292080+0000", 2024-09-11T06:51:38.555 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.0092999999999994e-05 2024-09-11T06:51:38.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.556 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.556 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:38.556 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.292162+0000", 2024-09-11T06:51:38.556 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 8.2210000000000001e-05 2024-09-11T06:51:38.556 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.556 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.556 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 0", 2024-09-11T06:51:38.556 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.292278+0000", 2024-09-11T06:51:38.556 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.000115555 2024-09-11T06:51:38.556 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.556 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.556 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:38.556 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.293847+0000", 2024-09-11T06:51:38.557 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0015695010000000001 2024-09-11T06:51:38.557 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.557 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.557 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:38.557 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298503+0000", 2024-09-11T06:51:38.557 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0046562440000000004 2024-09-11T06:51:38.557 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.557 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.557 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:38.557 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298546+0000", 2024-09-11T06:51:38.557 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.2749e-05 2024-09-11T06:51:38.557 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.557 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.557 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:38.558 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298578+0000", 2024-09-11T06:51:38.558 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 3.1606999999999997e-05 2024-09-11T06:51:38.558 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.558 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:38.558 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.558 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.558 INFO:teuthology.orchestra.run.smithi089.stdout: "description": "osd_op(client.5723.0:95 2.3 2:d0a3a233:ns::rbd_data.11adc335555f.000000000000000e:head [delete] snapc 4=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e111)", 2024-09-11T06:51:38.558 INFO:teuthology.orchestra.run.smithi089.stdout: "initiated_at": "2024-09-11T06:51:34.298831+0000", 2024-09-11T06:51:38.558 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 4.1399045120000002, 2024-09-11T06:51:38.558 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.003269582, 2024-09-11T06:51:38.558 INFO:teuthology.orchestra.run.smithi089.stdout: "continuous": false, 2024-09-11T06:51:38.558 INFO:teuthology.orchestra.run.smithi089.stdout: "type_data": { 2024-09-11T06:51:38.558 INFO:teuthology.orchestra.run.smithi089.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T06:51:38.559 INFO:teuthology.orchestra.run.smithi089.stdout: "client_info": { 2024-09-11T06:51:38.559 INFO:teuthology.orchestra.run.smithi089.stdout: "client": "client.5723", 2024-09-11T06:51:38.559 INFO:teuthology.orchestra.run.smithi089.stdout: "client_addr": "172.21.15.176:0/3725019196", 2024-09-11T06:51:38.559 INFO:teuthology.orchestra.run.smithi089.stdout: "tid": 95 2024-09-11T06:51:38.559 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.559 INFO:teuthology.orchestra.run.smithi089.stdout: "events": [ 2024-09-11T06:51:38.559 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.559 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "initiated", 2024-09-11T06:51:38.559 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298831+0000", 2024-09-11T06:51:38.559 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.559 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.559 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.559 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "header_read", 2024-09-11T06:51:38.559 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298831+0000", 2024-09-11T06:51:38.560 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0 2024-09-11T06:51:38.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.560 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "throttled", 2024-09-11T06:51:38.560 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298832+0000", 2024-09-11T06:51:38.560 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.145e-06 2024-09-11T06:51:38.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.560 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "all_read", 2024-09-11T06:51:38.560 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298837+0000", 2024-09-11T06:51:38.560 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.0839999999999999e-06 2024-09-11T06:51:38.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.561 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "dispatched", 2024-09-11T06:51:38.561 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298851+0000", 2024-09-11T06:51:38.561 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.4166e-05 2024-09-11T06:51:38.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.561 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.561 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "queued_for_pg", 2024-09-11T06:51:38.561 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298861+0000", 2024-09-11T06:51:38.561 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 1.0169e-05 2024-09-11T06:51:38.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.561 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.561 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "reached_pg", 2024-09-11T06:51:38.561 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298917+0000", 2024-09-11T06:51:38.561 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.5195999999999998e-05 2024-09-11T06:51:38.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.562 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "started", 2024-09-11T06:51:38.562 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.298969+0000", 2024-09-11T06:51:38.562 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.2550999999999999e-05 2024-09-11T06:51:38.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.562 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "waiting for subops from 2", 2024-09-11T06:51:38.562 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.299036+0000", 2024-09-11T06:51:38.562 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 6.7070000000000007e-05 2024-09-11T06:51:38.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.563 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "op_commit", 2024-09-11T06:51:38.563 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.300825+0000", 2024-09-11T06:51:38.563 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.001788913 2024-09-11T06:51:38.563 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.563 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "sub_op_commit_rec", 2024-09-11T06:51:38.563 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.301997+0000", 2024-09-11T06:51:38.564 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 0.0011715180000000001 2024-09-11T06:51:38.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.564 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "commit_sent", 2024-09-11T06:51:38.564 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.302042+0000", 2024-09-11T06:51:38.564 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 4.5306000000000003e-05 2024-09-11T06:51:38.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-11T06:51:38.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-11T06:51:38.564 INFO:teuthology.orchestra.run.smithi089.stdout: "event": "done", 2024-09-11T06:51:38.564 INFO:teuthology.orchestra.run.smithi089.stdout: "time": "2024-09-11T06:51:34.302100+0000", 2024-09-11T06:51:38.564 INFO:teuthology.orchestra.run.smithi089.stdout: "duration": 5.8464000000000001e-05 2024-09-11T06:51:38.564 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.564 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:38.565 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.565 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-11T06:51:38.565 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-11T06:51:38.565 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-11T06:51:38.949 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is unset 2024-09-11T06:51:39.500 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T06:51:39.954 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is unset 2024-09-11T06:51:41.991 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-11T06:51:42.135 INFO:teuthology.orchestra.run.smithi071.stdout:mon.a: {} 2024-09-11T06:51:42.135 INFO:teuthology.orchestra.run.smithi071.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T06:51:42.148 INFO:teuthology.orchestra.run.smithi071.stdout:mon.b: {} 2024-09-11T06:51:42.148 INFO:teuthology.orchestra.run.smithi071.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T06:51:42.158 INFO:teuthology.orchestra.run.smithi071.stdout:mon.c: {} 2024-09-11T06:51:42.158 INFO:teuthology.orchestra.run.smithi071.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T06:51:42.175 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-11T06:51:42.957 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.0 to 1 (10000) 2024-09-11T06:51:42.984 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-11T06:51:43.980 INFO:teuthology.orchestra.run.smithi071.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-11T06:51:44.002 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-11T06:51:44.985 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.1 to 1 (10000) 2024-09-11T06:51:45.012 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-11T06:51:45.994 INFO:teuthology.orchestra.run.smithi071.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-11T06:51:46.018 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-11T06:51:46.946 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.2 to 1 (10000) 2024-09-11T06:51:46.968 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-11T06:51:48.017 INFO:teuthology.orchestra.run.smithi071.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-11T06:51:48.040 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-11T06:51:49.017 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.3 to 1 (10000) 2024-09-11T06:51:49.044 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-11T06:51:50.039 INFO:teuthology.orchestra.run.smithi071.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-11T06:51:50.062 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-11T06:51:51.035 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.4 to 1 (10000) 2024-09-11T06:51:51.057 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-11T06:51:52.055 INFO:teuthology.orchestra.run.smithi071.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-11T06:51:52.080 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-11T06:51:53.057 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.5 to 1 (10000) 2024-09-11T06:51:53.082 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-11T06:51:54.079 INFO:teuthology.orchestra.run.smithi071.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-11T06:51:54.103 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-11T06:51:54.103 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-11T06:51:54.103 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-11T06:51:54.103 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-11T06:51:54.104 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-11T06:51:54.104 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T06:51:54.388 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:51:54.388 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":130,"fsid":"bd5dcb33-e9f7-4a5f-a26c-a451452e8aa2","created":"2024-09-11T06:48:52.191706+0000","modified":"2024-09-11T06:51:54.066047+0000","last_up_change":"2024-09-11T06:49:04.007203+0000","last_in_change":"2024-09-11T06:51:36.934758+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T06:49:05.084515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-11T06:49:09.530167+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":7,"snap_epoch":111,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c9e35bf-d3db-49db-b10c-d2cf38c7eb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1158147777},{"type":"v1","addr":"172.21.15.71:6801","nonce":1158147777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1158147777},{"type":"v1","addr":"172.21.15.71:6803","nonce":1158147777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1158147777},{"type":"v1","addr":"172.21.15.71:6807","nonce":1158147777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1158147777},{"type":"v1","addr":"172.21.15.71:6805","nonce":1158147777}]},"public_addr":"172.21.15.71:6801/1158147777","cluster_addr":"172.21.15.71:6803/1158147777","heartbeat_back_addr":"172.21.15.71:6807/1158147777","heartbeat_front_addr":"172.21.15.71:6805/1158147777","state":["exists","up"]},{"osd":1,"uuid":"693649dc-db7c-43cf-bdbe-87f6128ef311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":620364628},{"type":"v1","addr":"172.21.15.71:6809","nonce":620364628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":620364628},{"type":"v1","addr":"172.21.15.71:6811","nonce":620364628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":620364628},{"type":"v1","addr":"172.21.15.71:6815","nonce":620364628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":620364628},{"type":"v1","addr":"172.21.15.71:6813","nonce":620364628}]},"public_addr":"172.21.15.71:6809/620364628","cluster_addr":"172.21.15.71:6811/620364628","heartbeat_back_addr":"172.21.15.71:6815/620364628","heartbeat_front_addr":"172.21.15.71:6813/620364628","state":["exists","up"]},{"osd":2,"uuid":"89421ae7-fead-409c-83f7-ede424db6004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2926882521},{"type":"v1","addr":"172.21.15.71:6817","nonce":2926882521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2926882521},{"type":"v1","addr":"172.21.15.71:6819","nonce":2926882521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2926882521},{"type":"v1","addr":"172.21.15.71:6823","nonce":2926882521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2926882521},{"type":"v1","addr":"172.21.15.71:6821","nonce":2926882521}]},"public_addr":"172.21.15.71:6817/2926882521","cluster_addr":"172.21.15.71:6819/2926882521","heartbeat_back_addr":"172.21.15.71:6823/2926882521","heartbeat_front_addr":"172.21.15.71:6821/2926882521","state":["exists","up"]},{"osd":3,"uuid":"f4512e48-63f1-42b6-81be-594bd82e3210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":558942315},{"type":"v1","addr":"172.21.15.89:6809","nonce":558942315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":558942315},{"type":"v1","addr":"172.21.15.89:6811","nonce":558942315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":558942315},{"type":"v1","addr":"172.21.15.89:6815","nonce":558942315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":558942315},{"type":"v1","addr":"172.21.15.89:6813","nonce":558942315}]},"public_addr":"172.21.15.89:6809/558942315","cluster_addr":"172.21.15.89:6811/558942315","heartbeat_back_addr":"172.21.15.89:6815/558942315","heartbeat_front_addr":"172.21.15.89:6813/558942315","state":["exists","up"]},{"osd":4,"uuid":"52d47b6f-af30-455e-8326-e53c598911ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3878488003},{"type":"v1","addr":"172.21.15.89:6817","nonce":3878488003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3878488003},{"type":"v1","addr":"172.21.15.89:6819","nonce":3878488003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":3878488003},{"type":"v1","addr":"172.21.15.89:6823","nonce":3878488003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":3878488003},{"type":"v1","addr":"172.21.15.89:6821","nonce":3878488003}]},"public_addr":"172.21.15.89:6817/3878488003","cluster_addr":"172.21.15.89:6819/3878488003","heartbeat_back_addr":"172.21.15.89:6823/3878488003","heartbeat_front_addr":"172.21.15.89:6821/3878488003","state":["exists","up"]},{"osd":5,"uuid":"6b4f9416-0ba0-4c58-883e-8cbdb362adcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2840204107},{"type":"v1","addr":"172.21.15.89:6801","nonce":2840204107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2840204107},{"type":"v1","addr":"172.21.15.89:6803","nonce":2840204107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2840204107},{"type":"v1","addr":"172.21.15.89:6807","nonce":2840204107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2840204107},{"type":"v1","addr":"172.21.15.89:6805","nonce":2840204107}]},"public_addr":"172.21.15.89:6801/2840204107","cluster_addr":"172.21.15.89:6803/2840204107","heartbeat_back_addr":"172.21.15.89:6807/2840204107","heartbeat_front_addr":"172.21.15.89:6805/2840204107","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.282785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.381259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.986977+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.732329+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.844988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.639989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T06:51:54.408 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-11T06:51:54.408 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T06:51:54.694 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:51:54.695 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":130,"fsid":"bd5dcb33-e9f7-4a5f-a26c-a451452e8aa2","created":"2024-09-11T06:48:52.191706+0000","modified":"2024-09-11T06:51:54.066047+0000","last_up_change":"2024-09-11T06:49:04.007203+0000","last_in_change":"2024-09-11T06:51:36.934758+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T06:49:05.084515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-11T06:49:09.530167+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":7,"snap_epoch":111,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c9e35bf-d3db-49db-b10c-d2cf38c7eb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1158147777},{"type":"v1","addr":"172.21.15.71:6801","nonce":1158147777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1158147777},{"type":"v1","addr":"172.21.15.71:6803","nonce":1158147777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1158147777},{"type":"v1","addr":"172.21.15.71:6807","nonce":1158147777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1158147777},{"type":"v1","addr":"172.21.15.71:6805","nonce":1158147777}]},"public_addr":"172.21.15.71:6801/1158147777","cluster_addr":"172.21.15.71:6803/1158147777","heartbeat_back_addr":"172.21.15.71:6807/1158147777","heartbeat_front_addr":"172.21.15.71:6805/1158147777","state":["exists","up"]},{"osd":1,"uuid":"693649dc-db7c-43cf-bdbe-87f6128ef311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":620364628},{"type":"v1","addr":"172.21.15.71:6809","nonce":620364628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":620364628},{"type":"v1","addr":"172.21.15.71:6811","nonce":620364628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":620364628},{"type":"v1","addr":"172.21.15.71:6815","nonce":620364628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":620364628},{"type":"v1","addr":"172.21.15.71:6813","nonce":620364628}]},"public_addr":"172.21.15.71:6809/620364628","cluster_addr":"172.21.15.71:6811/620364628","heartbeat_back_addr":"172.21.15.71:6815/620364628","heartbeat_front_addr":"172.21.15.71:6813/620364628","state":["exists","up"]},{"osd":2,"uuid":"89421ae7-fead-409c-83f7-ede424db6004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2926882521},{"type":"v1","addr":"172.21.15.71:6817","nonce":2926882521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2926882521},{"type":"v1","addr":"172.21.15.71:6819","nonce":2926882521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2926882521},{"type":"v1","addr":"172.21.15.71:6823","nonce":2926882521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2926882521},{"type":"v1","addr":"172.21.15.71:6821","nonce":2926882521}]},"public_addr":"172.21.15.71:6817/2926882521","cluster_addr":"172.21.15.71:6819/2926882521","heartbeat_back_addr":"172.21.15.71:6823/2926882521","heartbeat_front_addr":"172.21.15.71:6821/2926882521","state":["exists","up"]},{"osd":3,"uuid":"f4512e48-63f1-42b6-81be-594bd82e3210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":558942315},{"type":"v1","addr":"172.21.15.89:6809","nonce":558942315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":558942315},{"type":"v1","addr":"172.21.15.89:6811","nonce":558942315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":558942315},{"type":"v1","addr":"172.21.15.89:6815","nonce":558942315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":558942315},{"type":"v1","addr":"172.21.15.89:6813","nonce":558942315}]},"public_addr":"172.21.15.89:6809/558942315","cluster_addr":"172.21.15.89:6811/558942315","heartbeat_back_addr":"172.21.15.89:6815/558942315","heartbeat_front_addr":"172.21.15.89:6813/558942315","state":["exists","up"]},{"osd":4,"uuid":"52d47b6f-af30-455e-8326-e53c598911ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3878488003},{"type":"v1","addr":"172.21.15.89:6817","nonce":3878488003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3878488003},{"type":"v1","addr":"172.21.15.89:6819","nonce":3878488003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":3878488003},{"type":"v1","addr":"172.21.15.89:6823","nonce":3878488003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":3878488003},{"type":"v1","addr":"172.21.15.89:6821","nonce":3878488003}]},"public_addr":"172.21.15.89:6817/3878488003","cluster_addr":"172.21.15.89:6819/3878488003","heartbeat_back_addr":"172.21.15.89:6823/3878488003","heartbeat_front_addr":"172.21.15.89:6821/3878488003","state":["exists","up"]},{"osd":5,"uuid":"6b4f9416-0ba0-4c58-883e-8cbdb362adcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2840204107},{"type":"v1","addr":"172.21.15.89:6801","nonce":2840204107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2840204107},{"type":"v1","addr":"172.21.15.89:6803","nonce":2840204107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2840204107},{"type":"v1","addr":"172.21.15.89:6807","nonce":2840204107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2840204107},{"type":"v1","addr":"172.21.15.89:6805","nonce":2840204107}]},"public_addr":"172.21.15.89:6801/2840204107","cluster_addr":"172.21.15.89:6803/2840204107","heartbeat_back_addr":"172.21.15.89:6807/2840204107","heartbeat_front_addr":"172.21.15.89:6805/2840204107","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.282785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.381259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.986977+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.732329+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.844988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.639989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T06:51:54.714 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-11T06:51:54.714 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-11T06:51:54.714 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-11T06:51:54.715 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-11T06:51:54.715 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-11T06:51:54.716 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-11T06:51:54.877 INFO:teuthology.orchestra.run.smithi071.stdout:47244640294 2024-09-11T06:51:54.877 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-11T06:51:54.882 INFO:teuthology.orchestra.run.smithi071.stdout:47244640294 2024-09-11T06:51:54.882 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-11T06:51:54.932 INFO:teuthology.orchestra.run.smithi071.stdout:47244640295 2024-09-11T06:51:54.933 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T06:51:54.948 INFO:teuthology.orchestra.run.smithi071.stdout:47244640293 2024-09-11T06:51:54.949 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-11T06:51:54.964 INFO:teuthology.orchestra.run.smithi071.stdout:47244640295 2024-09-11T06:51:54.964 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T06:51:54.969 INFO:teuthology.orchestra.run.smithi071.stdout:47244640296 2024-09-11T06:51:54.969 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T06:51:55.254 INFO:teuthology.orchestra.run.smithi071.stdout:47244640294 2024-09-11T06:51:55.272 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640294 got 47244640294 for osd.4 2024-09-11T06:51:55.272 DEBUG:teuthology.parallel:result is None 2024-09-11T06:51:55.321 INFO:teuthology.orchestra.run.smithi071.stdout:47244640294 2024-09-11T06:51:55.328 INFO:teuthology.orchestra.run.smithi071.stdout:47244640295 2024-09-11T06:51:55.347 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640295 got 47244640295 for osd.1 2024-09-11T06:51:55.348 DEBUG:teuthology.parallel:result is None 2024-09-11T06:51:55.349 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640294 got 47244640294 for osd.3 2024-09-11T06:51:55.349 DEBUG:teuthology.parallel:result is None 2024-09-11T06:51:55.392 INFO:teuthology.orchestra.run.smithi071.stdout:47244640295 2024-09-11T06:51:55.396 INFO:teuthology.orchestra.run.smithi071.stdout:47244640293 2024-09-11T06:51:55.411 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640295 got 47244640295 for osd.2 2024-09-11T06:51:55.411 DEBUG:teuthology.parallel:result is None 2024-09-11T06:51:55.414 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640293 got 47244640293 for osd.5 2024-09-11T06:51:55.415 DEBUG:teuthology.parallel:result is None 2024-09-11T06:51:55.441 INFO:teuthology.orchestra.run.smithi071.stdout:47244640296 2024-09-11T06:51:55.456 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640296 got 47244640296 for osd.0 2024-09-11T06:51:55.456 DEBUG:teuthology.parallel:result is None 2024-09-11T06:51:55.456 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-11T06:51:55.456 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:51:55.765 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:51:55.766 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:51:55.782 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":211,"stamp":"2024-09-11T06:51:55.051230+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7061,"num_read_kb":120429,"num_write":1148,"num_write_kb":275375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":262474456,"num_keys_recovered":41,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1243,"ondisk_log_size":1243,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":197392,"kb_used_data":10252,"kb_used_omap":9,"kb_used_meta":186998,"kb_avail":566033648,"statfs":{"total":579820584960,"available":579618455552,"internally_reserved":0,"allocated":10498048,"data_stored":4782466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9565,"internal_metadata":191486627},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.012941"},"pg_stats":[{"pgid":"1.0","version":"14'117","reported_seq":398,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124649+0000","last_change":"2024-09-11T06:50:37.941212+0000","last_active":"2024-09-11T06:51:54.124649+0000","last_peered":"2024-09-11T06:51:54.124649+0000","last_clean":"2024-09-11T06:51:54.124649+0000","last_became_active":"2024-09-11T06:50:37.940448+0000","last_became_peered":"2024-09-11T06:50:37.940448+0000","last_unstale":"2024-09-11T06:51:54.124649+0000","last_undegraded":"2024-09-11T06:51:54.124649+0000","last_fullsized":"2024-09-11T06:51:54.124649+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_clean_scrub_stamp":"2024-09-11T06:49:05.088839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:48:59.833858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"111'128","reported_seq":438,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124929+0000","last_change":"2024-09-11T06:51:34.211062+0000","last_active":"2024-09-11T06:51:54.124929+0000","last_peered":"2024-09-11T06:51:54.124929+0000","last_clean":"2024-09-11T06:51:54.124929+0000","last_became_active":"2024-09-11T06:50:47.306211+0000","last_became_peered":"2024-09-11T06:50:47.306211+0000","last_unstale":"2024-09-11T06:51:54.124929+0000","last_undegraded":"2024-09-11T06:51:54.124929+0000","last_fullsized":"2024-09-11T06:51:54.124929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:30:28.938965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000473933,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":383,"num_read_kb":8504,"num_write":98,"num_write_kb":28714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":25165862,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"111'89","reported_seq":400,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124735+0000","last_change":"2024-09-11T06:51:37.987266+0000","last_active":"2024-09-11T06:51:54.124735+0000","last_peered":"2024-09-11T06:51:54.124735+0000","last_clean":"2024-09-11T06:51:54.124735+0000","last_became_active":"2024-09-11T06:51:37.955297+0000","last_became_peered":"2024-09-11T06:51:37.955297+0000","last_unstale":"2024-09-11T06:51:54.124735+0000","last_undegraded":"2024-09-11T06:51:54.124735+0000","last_fullsized":"2024-09-11T06:51:54.124735+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:25.093793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098377899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":157,"num_read_kb":23067,"num_write":66,"num_write_kb":45059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":62914560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"111'255","reported_seq":2292,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.077049+0000","last_change":"2024-09-11T06:51:40.956789+0000","last_active":"2024-09-11T06:51:54.077049+0000","last_peered":"2024-09-11T06:51:54.077049+0000","last_clean":"2024-09-11T06:51:54.077049+0000","last_became_active":"2024-09-11T06:51:40.955516+0000","last_became_peered":"2024-09-11T06:51:40.955516+0000","last_unstale":"2024-09-11T06:51:54.077049+0000","last_undegraded":"2024-09-11T06:51:54.077049+0000","last_fullsized":"2024-09-11T06:51:54.077049+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:32:24.031714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017208530000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4173,"num_read_kb":27885,"num_write":144,"num_write_kb":41060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":25231476,"num_keys_recovered":13,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"111'12","reported_seq":273,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124950+0000","last_change":"2024-09-11T06:51:37.965118+0000","last_active":"2024-09-11T06:51:54.124950+0000","last_peered":"2024-09-11T06:51:54.124950+0000","last_clean":"2024-09-11T06:51:54.124950+0000","last_became_active":"2024-09-11T06:51:37.952071+0000","last_became_peered":"2024-09-11T06:51:37.952071+0000","last_unstale":"2024-09-11T06:51:54.124950+0000","last_undegraded":"2024-09-11T06:51:54.124950+0000","last_fullsized":"2024-09-11T06:51:54.124950+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:36:40.530470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011519830000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":93,"num_read_kb":90,"num_write":10,"num_write_kb":8196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":16777446,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"111'201","reported_seq":544,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.077858+0000","last_change":"2024-09-11T06:51:40.961258+0000","last_active":"2024-09-11T06:51:54.077858+0000","last_peered":"2024-09-11T06:51:54.077858+0000","last_clean":"2024-09-11T06:51:54.077858+0000","last_became_active":"2024-09-11T06:51:40.959678+0000","last_became_peered":"2024-09-11T06:51:40.959678+0000","last_unstale":"2024-09-11T06:51:54.077858+0000","last_undegraded":"2024-09-11T06:51:54.077858+0000","last_fullsized":"2024-09-11T06:51:54.077858+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:17.319791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012221140000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":855,"num_read_kb":8871,"num_write":301,"num_write_kb":20705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":12582950,"num_keys_recovered":3,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"111'170","reported_seq":684,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124770+0000","last_change":"2024-09-11T06:51:34.143092+0000","last_active":"2024-09-11T06:51:54.124770+0000","last_peered":"2024-09-11T06:51:54.124770+0000","last_clean":"2024-09-11T06:51:54.124770+0000","last_became_active":"2024-09-11T06:50:35.236333+0000","last_became_peered":"2024-09-11T06:50:35.236333+0000","last_unstale":"2024-09-11T06:51:54.124770+0000","last_undegraded":"2024-09-11T06:51:54.124770+0000","last_fullsized":"2024-09-11T06:51:54.124770+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:18:39.890958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003008405,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":619,"num_read_kb":21604,"num_write":149,"num_write_kb":45181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":41943104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"111'68","reported_seq":50,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.077818+0000","last_change":"2024-09-11T06:51:37.965289+0000","last_active":"2024-09-11T06:51:54.077818+0000","last_peered":"2024-09-11T06:51:54.077818+0000","last_clean":"2024-09-11T06:51:54.077818+0000","last_became_active":"2024-09-11T06:51:37.950286+0000","last_became_peered":"2024-09-11T06:51:37.950286+0000","last_unstale":"2024-09-11T06:51:54.077818+0000","last_undegraded":"2024-09-11T06:51:54.077818+0000","last_fullsized":"2024-09-11T06:51:54.077818+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:36:32.689054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064404600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":160,"num_read_kb":8320,"num_write":79,"num_write_kb":16426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":25165938,"num_keys_recovered":9,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"111'203","reported_seq":477,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124866+0000","last_change":"2024-09-11T06:51:34.142678+0000","last_active":"2024-09-11T06:51:54.124866+0000","last_peered":"2024-09-11T06:51:54.124866+0000","last_clean":"2024-09-11T06:51:54.124866+0000","last_became_active":"2024-09-11T06:50:35.238011+0000","last_became_peered":"2024-09-11T06:50:35.238011+0000","last_unstale":"2024-09-11T06:51:54.124866+0000","last_undegraded":"2024-09-11T06:51:54.124866+0000","last_fullsized":"2024-09-11T06:51:54.124866+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:05:59.612752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027097340000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":505,"num_read_kb":21988,"num_write":142,"num_write_kb":67618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":50331648,"num_keys_recovered":16,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6945,"num_read_kb":120329,"num_write":989,"num_write_kb":272959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":260112984,"num_keys_recovered":41,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1126,"ondisk_log_size":1126,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640293,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1556480,"data_stored":601064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640294,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1552384,"data_stored":601045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640294,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33288,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2146304,"data_stored":1191413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640295,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1552384,"data_stored":601045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640295,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1556480,"data_stored":601064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640296,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2134016,"data_stored":1186835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:51:55.782 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:51:56.052 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:51:56.053 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:51:56.068 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":211,"stamp":"2024-09-11T06:51:55.051230+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7061,"num_read_kb":120429,"num_write":1148,"num_write_kb":275375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":262474456,"num_keys_recovered":41,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1243,"ondisk_log_size":1243,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":197392,"kb_used_data":10252,"kb_used_omap":9,"kb_used_meta":186998,"kb_avail":566033648,"statfs":{"total":579820584960,"available":579618455552,"internally_reserved":0,"allocated":10498048,"data_stored":4782466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9565,"internal_metadata":191486627},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.012941"},"pg_stats":[{"pgid":"1.0","version":"14'117","reported_seq":398,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124649+0000","last_change":"2024-09-11T06:50:37.941212+0000","last_active":"2024-09-11T06:51:54.124649+0000","last_peered":"2024-09-11T06:51:54.124649+0000","last_clean":"2024-09-11T06:51:54.124649+0000","last_became_active":"2024-09-11T06:50:37.940448+0000","last_became_peered":"2024-09-11T06:50:37.940448+0000","last_unstale":"2024-09-11T06:51:54.124649+0000","last_undegraded":"2024-09-11T06:51:54.124649+0000","last_fullsized":"2024-09-11T06:51:54.124649+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_clean_scrub_stamp":"2024-09-11T06:49:05.088839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:48:59.833858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"111'128","reported_seq":438,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124929+0000","last_change":"2024-09-11T06:51:34.211062+0000","last_active":"2024-09-11T06:51:54.124929+0000","last_peered":"2024-09-11T06:51:54.124929+0000","last_clean":"2024-09-11T06:51:54.124929+0000","last_became_active":"2024-09-11T06:50:47.306211+0000","last_became_peered":"2024-09-11T06:50:47.306211+0000","last_unstale":"2024-09-11T06:51:54.124929+0000","last_undegraded":"2024-09-11T06:51:54.124929+0000","last_fullsized":"2024-09-11T06:51:54.124929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:30:28.938965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000473933,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":383,"num_read_kb":8504,"num_write":98,"num_write_kb":28714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":25165862,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"111'89","reported_seq":400,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124735+0000","last_change":"2024-09-11T06:51:37.987266+0000","last_active":"2024-09-11T06:51:54.124735+0000","last_peered":"2024-09-11T06:51:54.124735+0000","last_clean":"2024-09-11T06:51:54.124735+0000","last_became_active":"2024-09-11T06:51:37.955297+0000","last_became_peered":"2024-09-11T06:51:37.955297+0000","last_unstale":"2024-09-11T06:51:54.124735+0000","last_undegraded":"2024-09-11T06:51:54.124735+0000","last_fullsized":"2024-09-11T06:51:54.124735+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:25.093793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098377899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":157,"num_read_kb":23067,"num_write":66,"num_write_kb":45059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":62914560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"111'255","reported_seq":2292,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.077049+0000","last_change":"2024-09-11T06:51:40.956789+0000","last_active":"2024-09-11T06:51:54.077049+0000","last_peered":"2024-09-11T06:51:54.077049+0000","last_clean":"2024-09-11T06:51:54.077049+0000","last_became_active":"2024-09-11T06:51:40.955516+0000","last_became_peered":"2024-09-11T06:51:40.955516+0000","last_unstale":"2024-09-11T06:51:54.077049+0000","last_undegraded":"2024-09-11T06:51:54.077049+0000","last_fullsized":"2024-09-11T06:51:54.077049+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:32:24.031714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017208530000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4173,"num_read_kb":27885,"num_write":144,"num_write_kb":41060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":25231476,"num_keys_recovered":13,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"111'12","reported_seq":273,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124950+0000","last_change":"2024-09-11T06:51:37.965118+0000","last_active":"2024-09-11T06:51:54.124950+0000","last_peered":"2024-09-11T06:51:54.124950+0000","last_clean":"2024-09-11T06:51:54.124950+0000","last_became_active":"2024-09-11T06:51:37.952071+0000","last_became_peered":"2024-09-11T06:51:37.952071+0000","last_unstale":"2024-09-11T06:51:54.124950+0000","last_undegraded":"2024-09-11T06:51:54.124950+0000","last_fullsized":"2024-09-11T06:51:54.124950+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:36:40.530470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011519830000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":93,"num_read_kb":90,"num_write":10,"num_write_kb":8196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":16777446,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"111'201","reported_seq":544,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.077858+0000","last_change":"2024-09-11T06:51:40.961258+0000","last_active":"2024-09-11T06:51:54.077858+0000","last_peered":"2024-09-11T06:51:54.077858+0000","last_clean":"2024-09-11T06:51:54.077858+0000","last_became_active":"2024-09-11T06:51:40.959678+0000","last_became_peered":"2024-09-11T06:51:40.959678+0000","last_unstale":"2024-09-11T06:51:54.077858+0000","last_undegraded":"2024-09-11T06:51:54.077858+0000","last_fullsized":"2024-09-11T06:51:54.077858+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:17.319791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012221140000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":855,"num_read_kb":8871,"num_write":301,"num_write_kb":20705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":12582950,"num_keys_recovered":3,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"111'170","reported_seq":684,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124770+0000","last_change":"2024-09-11T06:51:34.143092+0000","last_active":"2024-09-11T06:51:54.124770+0000","last_peered":"2024-09-11T06:51:54.124770+0000","last_clean":"2024-09-11T06:51:54.124770+0000","last_became_active":"2024-09-11T06:50:35.236333+0000","last_became_peered":"2024-09-11T06:50:35.236333+0000","last_unstale":"2024-09-11T06:51:54.124770+0000","last_undegraded":"2024-09-11T06:51:54.124770+0000","last_fullsized":"2024-09-11T06:51:54.124770+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:18:39.890958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003008405,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":619,"num_read_kb":21604,"num_write":149,"num_write_kb":45181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":41943104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"111'68","reported_seq":50,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.077818+0000","last_change":"2024-09-11T06:51:37.965289+0000","last_active":"2024-09-11T06:51:54.077818+0000","last_peered":"2024-09-11T06:51:54.077818+0000","last_clean":"2024-09-11T06:51:54.077818+0000","last_became_active":"2024-09-11T06:51:37.950286+0000","last_became_peered":"2024-09-11T06:51:37.950286+0000","last_unstale":"2024-09-11T06:51:54.077818+0000","last_undegraded":"2024-09-11T06:51:54.077818+0000","last_fullsized":"2024-09-11T06:51:54.077818+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:36:32.689054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064404600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":160,"num_read_kb":8320,"num_write":79,"num_write_kb":16426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":25165938,"num_keys_recovered":9,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"111'203","reported_seq":477,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124866+0000","last_change":"2024-09-11T06:51:34.142678+0000","last_active":"2024-09-11T06:51:54.124866+0000","last_peered":"2024-09-11T06:51:54.124866+0000","last_clean":"2024-09-11T06:51:54.124866+0000","last_became_active":"2024-09-11T06:50:35.238011+0000","last_became_peered":"2024-09-11T06:50:35.238011+0000","last_unstale":"2024-09-11T06:51:54.124866+0000","last_undegraded":"2024-09-11T06:51:54.124866+0000","last_fullsized":"2024-09-11T06:51:54.124866+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:05:59.612752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027097340000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":505,"num_read_kb":21988,"num_write":142,"num_write_kb":67618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":50331648,"num_keys_recovered":16,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6945,"num_read_kb":120329,"num_write":989,"num_write_kb":272959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":260112984,"num_keys_recovered":41,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1126,"ondisk_log_size":1126,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640293,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1556480,"data_stored":601064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640294,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1552384,"data_stored":601045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640294,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33288,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2146304,"data_stored":1191413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640295,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1552384,"data_stored":601045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640295,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1556480,"data_stored":601064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640296,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2134016,"data_stored":1186835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:51:56.069 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-11T06:51:56.069 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-11T06:51:56.364 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-11T06:51:56.377 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T06:51:56.377 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:51:56.653 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:51:56.653 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:51:56.669 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":211,"stamp":"2024-09-11T06:51:55.051230+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7061,"num_read_kb":120429,"num_write":1148,"num_write_kb":275375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":262474456,"num_keys_recovered":41,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1243,"ondisk_log_size":1243,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":197392,"kb_used_data":10252,"kb_used_omap":9,"kb_used_meta":186998,"kb_avail":566033648,"statfs":{"total":579820584960,"available":579618455552,"internally_reserved":0,"allocated":10498048,"data_stored":4782466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9565,"internal_metadata":191486627},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.012941"},"pg_stats":[{"pgid":"1.0","version":"14'117","reported_seq":398,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124649+0000","last_change":"2024-09-11T06:50:37.941212+0000","last_active":"2024-09-11T06:51:54.124649+0000","last_peered":"2024-09-11T06:51:54.124649+0000","last_clean":"2024-09-11T06:51:54.124649+0000","last_became_active":"2024-09-11T06:50:37.940448+0000","last_became_peered":"2024-09-11T06:50:37.940448+0000","last_unstale":"2024-09-11T06:51:54.124649+0000","last_undegraded":"2024-09-11T06:51:54.124649+0000","last_fullsized":"2024-09-11T06:51:54.124649+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_clean_scrub_stamp":"2024-09-11T06:49:05.088839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:48:59.833858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"111'128","reported_seq":438,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124929+0000","last_change":"2024-09-11T06:51:34.211062+0000","last_active":"2024-09-11T06:51:54.124929+0000","last_peered":"2024-09-11T06:51:54.124929+0000","last_clean":"2024-09-11T06:51:54.124929+0000","last_became_active":"2024-09-11T06:50:47.306211+0000","last_became_peered":"2024-09-11T06:50:47.306211+0000","last_unstale":"2024-09-11T06:51:54.124929+0000","last_undegraded":"2024-09-11T06:51:54.124929+0000","last_fullsized":"2024-09-11T06:51:54.124929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:30:28.938965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000473933,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":383,"num_read_kb":8504,"num_write":98,"num_write_kb":28714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":25165862,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"111'89","reported_seq":400,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124735+0000","last_change":"2024-09-11T06:51:37.987266+0000","last_active":"2024-09-11T06:51:54.124735+0000","last_peered":"2024-09-11T06:51:54.124735+0000","last_clean":"2024-09-11T06:51:54.124735+0000","last_became_active":"2024-09-11T06:51:37.955297+0000","last_became_peered":"2024-09-11T06:51:37.955297+0000","last_unstale":"2024-09-11T06:51:54.124735+0000","last_undegraded":"2024-09-11T06:51:54.124735+0000","last_fullsized":"2024-09-11T06:51:54.124735+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:25.093793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098377899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":157,"num_read_kb":23067,"num_write":66,"num_write_kb":45059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":62914560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"111'255","reported_seq":2292,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.077049+0000","last_change":"2024-09-11T06:51:40.956789+0000","last_active":"2024-09-11T06:51:54.077049+0000","last_peered":"2024-09-11T06:51:54.077049+0000","last_clean":"2024-09-11T06:51:54.077049+0000","last_became_active":"2024-09-11T06:51:40.955516+0000","last_became_peered":"2024-09-11T06:51:40.955516+0000","last_unstale":"2024-09-11T06:51:54.077049+0000","last_undegraded":"2024-09-11T06:51:54.077049+0000","last_fullsized":"2024-09-11T06:51:54.077049+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:32:24.031714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017208530000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4173,"num_read_kb":27885,"num_write":144,"num_write_kb":41060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":25231476,"num_keys_recovered":13,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"111'12","reported_seq":273,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124950+0000","last_change":"2024-09-11T06:51:37.965118+0000","last_active":"2024-09-11T06:51:54.124950+0000","last_peered":"2024-09-11T06:51:54.124950+0000","last_clean":"2024-09-11T06:51:54.124950+0000","last_became_active":"2024-09-11T06:51:37.952071+0000","last_became_peered":"2024-09-11T06:51:37.952071+0000","last_unstale":"2024-09-11T06:51:54.124950+0000","last_undegraded":"2024-09-11T06:51:54.124950+0000","last_fullsized":"2024-09-11T06:51:54.124950+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:36:40.530470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011519830000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":93,"num_read_kb":90,"num_write":10,"num_write_kb":8196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":16777446,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"111'201","reported_seq":544,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.077858+0000","last_change":"2024-09-11T06:51:40.961258+0000","last_active":"2024-09-11T06:51:54.077858+0000","last_peered":"2024-09-11T06:51:54.077858+0000","last_clean":"2024-09-11T06:51:54.077858+0000","last_became_active":"2024-09-11T06:51:40.959678+0000","last_became_peered":"2024-09-11T06:51:40.959678+0000","last_unstale":"2024-09-11T06:51:54.077858+0000","last_undegraded":"2024-09-11T06:51:54.077858+0000","last_fullsized":"2024-09-11T06:51:54.077858+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:17.319791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012221140000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":855,"num_read_kb":8871,"num_write":301,"num_write_kb":20705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":12582950,"num_keys_recovered":3,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"111'170","reported_seq":684,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124770+0000","last_change":"2024-09-11T06:51:34.143092+0000","last_active":"2024-09-11T06:51:54.124770+0000","last_peered":"2024-09-11T06:51:54.124770+0000","last_clean":"2024-09-11T06:51:54.124770+0000","last_became_active":"2024-09-11T06:50:35.236333+0000","last_became_peered":"2024-09-11T06:50:35.236333+0000","last_unstale":"2024-09-11T06:51:54.124770+0000","last_undegraded":"2024-09-11T06:51:54.124770+0000","last_fullsized":"2024-09-11T06:51:54.124770+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:18:39.890958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003008405,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":619,"num_read_kb":21604,"num_write":149,"num_write_kb":45181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":41943104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"111'68","reported_seq":50,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.077818+0000","last_change":"2024-09-11T06:51:37.965289+0000","last_active":"2024-09-11T06:51:54.077818+0000","last_peered":"2024-09-11T06:51:54.077818+0000","last_clean":"2024-09-11T06:51:54.077818+0000","last_became_active":"2024-09-11T06:51:37.950286+0000","last_became_peered":"2024-09-11T06:51:37.950286+0000","last_unstale":"2024-09-11T06:51:54.077818+0000","last_undegraded":"2024-09-11T06:51:54.077818+0000","last_fullsized":"2024-09-11T06:51:54.077818+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:36:32.689054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064404600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":160,"num_read_kb":8320,"num_write":79,"num_write_kb":16426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":25165938,"num_keys_recovered":9,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"111'203","reported_seq":477,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124866+0000","last_change":"2024-09-11T06:51:34.142678+0000","last_active":"2024-09-11T06:51:54.124866+0000","last_peered":"2024-09-11T06:51:54.124866+0000","last_clean":"2024-09-11T06:51:54.124866+0000","last_became_active":"2024-09-11T06:50:35.238011+0000","last_became_peered":"2024-09-11T06:50:35.238011+0000","last_unstale":"2024-09-11T06:51:54.124866+0000","last_undegraded":"2024-09-11T06:51:54.124866+0000","last_fullsized":"2024-09-11T06:51:54.124866+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:05:59.612752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027097340000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":505,"num_read_kb":21988,"num_write":142,"num_write_kb":67618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":50331648,"num_keys_recovered":16,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6945,"num_read_kb":120329,"num_write":989,"num_write_kb":272959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":260112984,"num_keys_recovered":41,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1126,"ondisk_log_size":1126,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640293,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1556480,"data_stored":601064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640294,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1552384,"data_stored":601045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640294,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33288,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2146304,"data_stored":1191413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640295,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1552384,"data_stored":601045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640295,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1556480,"data_stored":601064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640296,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2134016,"data_stored":1186835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:51:56.670 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:51:56.939 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:51:56.940 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:51:56.956 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":211,"stamp":"2024-09-11T06:51:55.051230+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7061,"num_read_kb":120429,"num_write":1148,"num_write_kb":275375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":262474456,"num_keys_recovered":41,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1243,"ondisk_log_size":1243,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":197392,"kb_used_data":10252,"kb_used_omap":9,"kb_used_meta":186998,"kb_avail":566033648,"statfs":{"total":579820584960,"available":579618455552,"internally_reserved":0,"allocated":10498048,"data_stored":4782466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9565,"internal_metadata":191486627},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.012941"},"pg_stats":[{"pgid":"1.0","version":"14'117","reported_seq":398,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124649+0000","last_change":"2024-09-11T06:50:37.941212+0000","last_active":"2024-09-11T06:51:54.124649+0000","last_peered":"2024-09-11T06:51:54.124649+0000","last_clean":"2024-09-11T06:51:54.124649+0000","last_became_active":"2024-09-11T06:50:37.940448+0000","last_became_peered":"2024-09-11T06:50:37.940448+0000","last_unstale":"2024-09-11T06:51:54.124649+0000","last_undegraded":"2024-09-11T06:51:54.124649+0000","last_fullsized":"2024-09-11T06:51:54.124649+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_clean_scrub_stamp":"2024-09-11T06:49:05.088839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:48:59.833858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"111'128","reported_seq":438,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124929+0000","last_change":"2024-09-11T06:51:34.211062+0000","last_active":"2024-09-11T06:51:54.124929+0000","last_peered":"2024-09-11T06:51:54.124929+0000","last_clean":"2024-09-11T06:51:54.124929+0000","last_became_active":"2024-09-11T06:50:47.306211+0000","last_became_peered":"2024-09-11T06:50:47.306211+0000","last_unstale":"2024-09-11T06:51:54.124929+0000","last_undegraded":"2024-09-11T06:51:54.124929+0000","last_fullsized":"2024-09-11T06:51:54.124929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:30:28.938965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000473933,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":383,"num_read_kb":8504,"num_write":98,"num_write_kb":28714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":25165862,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"111'89","reported_seq":400,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124735+0000","last_change":"2024-09-11T06:51:37.987266+0000","last_active":"2024-09-11T06:51:54.124735+0000","last_peered":"2024-09-11T06:51:54.124735+0000","last_clean":"2024-09-11T06:51:54.124735+0000","last_became_active":"2024-09-11T06:51:37.955297+0000","last_became_peered":"2024-09-11T06:51:37.955297+0000","last_unstale":"2024-09-11T06:51:54.124735+0000","last_undegraded":"2024-09-11T06:51:54.124735+0000","last_fullsized":"2024-09-11T06:51:54.124735+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:25.093793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098377899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":157,"num_read_kb":23067,"num_write":66,"num_write_kb":45059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":62914560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"111'255","reported_seq":2292,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.077049+0000","last_change":"2024-09-11T06:51:40.956789+0000","last_active":"2024-09-11T06:51:54.077049+0000","last_peered":"2024-09-11T06:51:54.077049+0000","last_clean":"2024-09-11T06:51:54.077049+0000","last_became_active":"2024-09-11T06:51:40.955516+0000","last_became_peered":"2024-09-11T06:51:40.955516+0000","last_unstale":"2024-09-11T06:51:54.077049+0000","last_undegraded":"2024-09-11T06:51:54.077049+0000","last_fullsized":"2024-09-11T06:51:54.077049+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:32:24.031714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017208530000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4173,"num_read_kb":27885,"num_write":144,"num_write_kb":41060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":25231476,"num_keys_recovered":13,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"111'12","reported_seq":273,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124950+0000","last_change":"2024-09-11T06:51:37.965118+0000","last_active":"2024-09-11T06:51:54.124950+0000","last_peered":"2024-09-11T06:51:54.124950+0000","last_clean":"2024-09-11T06:51:54.124950+0000","last_became_active":"2024-09-11T06:51:37.952071+0000","last_became_peered":"2024-09-11T06:51:37.952071+0000","last_unstale":"2024-09-11T06:51:54.124950+0000","last_undegraded":"2024-09-11T06:51:54.124950+0000","last_fullsized":"2024-09-11T06:51:54.124950+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:36:40.530470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011519830000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":93,"num_read_kb":90,"num_write":10,"num_write_kb":8196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":16777446,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"111'201","reported_seq":544,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.077858+0000","last_change":"2024-09-11T06:51:40.961258+0000","last_active":"2024-09-11T06:51:54.077858+0000","last_peered":"2024-09-11T06:51:54.077858+0000","last_clean":"2024-09-11T06:51:54.077858+0000","last_became_active":"2024-09-11T06:51:40.959678+0000","last_became_peered":"2024-09-11T06:51:40.959678+0000","last_unstale":"2024-09-11T06:51:54.077858+0000","last_undegraded":"2024-09-11T06:51:54.077858+0000","last_fullsized":"2024-09-11T06:51:54.077858+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:17.319791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012221140000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":855,"num_read_kb":8871,"num_write":301,"num_write_kb":20705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":12582950,"num_keys_recovered":3,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"111'170","reported_seq":684,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124770+0000","last_change":"2024-09-11T06:51:34.143092+0000","last_active":"2024-09-11T06:51:54.124770+0000","last_peered":"2024-09-11T06:51:54.124770+0000","last_clean":"2024-09-11T06:51:54.124770+0000","last_became_active":"2024-09-11T06:50:35.236333+0000","last_became_peered":"2024-09-11T06:50:35.236333+0000","last_unstale":"2024-09-11T06:51:54.124770+0000","last_undegraded":"2024-09-11T06:51:54.124770+0000","last_fullsized":"2024-09-11T06:51:54.124770+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:18:39.890958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003008405,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":619,"num_read_kb":21604,"num_write":149,"num_write_kb":45181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":41943104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"111'68","reported_seq":50,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.077818+0000","last_change":"2024-09-11T06:51:37.965289+0000","last_active":"2024-09-11T06:51:54.077818+0000","last_peered":"2024-09-11T06:51:54.077818+0000","last_clean":"2024-09-11T06:51:54.077818+0000","last_became_active":"2024-09-11T06:51:37.950286+0000","last_became_peered":"2024-09-11T06:51:37.950286+0000","last_unstale":"2024-09-11T06:51:54.077818+0000","last_undegraded":"2024-09-11T06:51:54.077818+0000","last_fullsized":"2024-09-11T06:51:54.077818+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:36:32.689054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064404600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":160,"num_read_kb":8320,"num_write":79,"num_write_kb":16426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":25165938,"num_keys_recovered":9,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"111'203","reported_seq":477,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124866+0000","last_change":"2024-09-11T06:51:34.142678+0000","last_active":"2024-09-11T06:51:54.124866+0000","last_peered":"2024-09-11T06:51:54.124866+0000","last_clean":"2024-09-11T06:51:54.124866+0000","last_became_active":"2024-09-11T06:50:35.238011+0000","last_became_peered":"2024-09-11T06:50:35.238011+0000","last_unstale":"2024-09-11T06:51:54.124866+0000","last_undegraded":"2024-09-11T06:51:54.124866+0000","last_fullsized":"2024-09-11T06:51:54.124866+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:05:59.612752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027097340000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":505,"num_read_kb":21988,"num_write":142,"num_write_kb":67618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":50331648,"num_keys_recovered":16,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6945,"num_read_kb":120329,"num_write":989,"num_write_kb":272959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":260112984,"num_keys_recovered":41,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1126,"ondisk_log_size":1126,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640293,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1556480,"data_stored":601064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640294,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1552384,"data_stored":601045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640294,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33288,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2146304,"data_stored":1191413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640295,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1552384,"data_stored":601045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640295,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1556480,"data_stored":601064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640296,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2134016,"data_stored":1186835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:51:56.957 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T06:51:56.960 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:51:57.253 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:51:57.253 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:51:57.271 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-09-11T06:51:57.051761+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7061,"num_read_kb":120429,"num_write":1148,"num_write_kb":275375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":262474456,"num_keys_recovered":41,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1243,"ondisk_log_size":1243,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":197392,"kb_used_data":10252,"kb_used_omap":9,"kb_used_meta":186998,"kb_avail":566033648,"statfs":{"total":579820584960,"available":579618455552,"internally_reserved":0,"allocated":10498048,"data_stored":4782466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9565,"internal_metadata":191486627},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001450"},"pg_stats":[{"pgid":"1.0","version":"14'117","reported_seq":398,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124649+0000","last_change":"2024-09-11T06:50:37.941212+0000","last_active":"2024-09-11T06:51:54.124649+0000","last_peered":"2024-09-11T06:51:54.124649+0000","last_clean":"2024-09-11T06:51:54.124649+0000","last_became_active":"2024-09-11T06:50:37.940448+0000","last_became_peered":"2024-09-11T06:50:37.940448+0000","last_unstale":"2024-09-11T06:51:54.124649+0000","last_undegraded":"2024-09-11T06:51:54.124649+0000","last_fullsized":"2024-09-11T06:51:54.124649+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_clean_scrub_stamp":"2024-09-11T06:49:05.088839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:48:59.833858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"111'128","reported_seq":438,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124929+0000","last_change":"2024-09-11T06:51:34.211062+0000","last_active":"2024-09-11T06:51:54.124929+0000","last_peered":"2024-09-11T06:51:54.124929+0000","last_clean":"2024-09-11T06:51:54.124929+0000","last_became_active":"2024-09-11T06:50:47.306211+0000","last_became_peered":"2024-09-11T06:50:47.306211+0000","last_unstale":"2024-09-11T06:51:54.124929+0000","last_undegraded":"2024-09-11T06:51:54.124929+0000","last_fullsized":"2024-09-11T06:51:54.124929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:30:28.938965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000473933,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":383,"num_read_kb":8504,"num_write":98,"num_write_kb":28714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":25165862,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"111'89","reported_seq":400,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124735+0000","last_change":"2024-09-11T06:51:37.987266+0000","last_active":"2024-09-11T06:51:54.124735+0000","last_peered":"2024-09-11T06:51:54.124735+0000","last_clean":"2024-09-11T06:51:54.124735+0000","last_became_active":"2024-09-11T06:51:37.955297+0000","last_became_peered":"2024-09-11T06:51:37.955297+0000","last_unstale":"2024-09-11T06:51:54.124735+0000","last_undegraded":"2024-09-11T06:51:54.124735+0000","last_fullsized":"2024-09-11T06:51:54.124735+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:25.093793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098377899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":157,"num_read_kb":23067,"num_write":66,"num_write_kb":45059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":62914560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"111'255","reported_seq":2292,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.077049+0000","last_change":"2024-09-11T06:51:40.956789+0000","last_active":"2024-09-11T06:51:54.077049+0000","last_peered":"2024-09-11T06:51:54.077049+0000","last_clean":"2024-09-11T06:51:54.077049+0000","last_became_active":"2024-09-11T06:51:40.955516+0000","last_became_peered":"2024-09-11T06:51:40.955516+0000","last_unstale":"2024-09-11T06:51:54.077049+0000","last_undegraded":"2024-09-11T06:51:54.077049+0000","last_fullsized":"2024-09-11T06:51:54.077049+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:32:24.031714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017208530000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4173,"num_read_kb":27885,"num_write":144,"num_write_kb":41060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":25231476,"num_keys_recovered":13,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"111'12","reported_seq":273,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124950+0000","last_change":"2024-09-11T06:51:37.965118+0000","last_active":"2024-09-11T06:51:54.124950+0000","last_peered":"2024-09-11T06:51:54.124950+0000","last_clean":"2024-09-11T06:51:54.124950+0000","last_became_active":"2024-09-11T06:51:37.952071+0000","last_became_peered":"2024-09-11T06:51:37.952071+0000","last_unstale":"2024-09-11T06:51:54.124950+0000","last_undegraded":"2024-09-11T06:51:54.124950+0000","last_fullsized":"2024-09-11T06:51:54.124950+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:36:40.530470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011519830000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":93,"num_read_kb":90,"num_write":10,"num_write_kb":8196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":16777446,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"111'201","reported_seq":544,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.077858+0000","last_change":"2024-09-11T06:51:40.961258+0000","last_active":"2024-09-11T06:51:54.077858+0000","last_peered":"2024-09-11T06:51:54.077858+0000","last_clean":"2024-09-11T06:51:54.077858+0000","last_became_active":"2024-09-11T06:51:40.959678+0000","last_became_peered":"2024-09-11T06:51:40.959678+0000","last_unstale":"2024-09-11T06:51:54.077858+0000","last_undegraded":"2024-09-11T06:51:54.077858+0000","last_fullsized":"2024-09-11T06:51:54.077858+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:17.319791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012221140000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":855,"num_read_kb":8871,"num_write":301,"num_write_kb":20705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":12582950,"num_keys_recovered":3,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"111'170","reported_seq":684,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124770+0000","last_change":"2024-09-11T06:51:34.143092+0000","last_active":"2024-09-11T06:51:54.124770+0000","last_peered":"2024-09-11T06:51:54.124770+0000","last_clean":"2024-09-11T06:51:54.124770+0000","last_became_active":"2024-09-11T06:50:35.236333+0000","last_became_peered":"2024-09-11T06:50:35.236333+0000","last_unstale":"2024-09-11T06:51:54.124770+0000","last_undegraded":"2024-09-11T06:51:54.124770+0000","last_fullsized":"2024-09-11T06:51:54.124770+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:18:39.890958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003008405,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":619,"num_read_kb":21604,"num_write":149,"num_write_kb":45181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":41943104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"111'68","reported_seq":50,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.077818+0000","last_change":"2024-09-11T06:51:37.965289+0000","last_active":"2024-09-11T06:51:54.077818+0000","last_peered":"2024-09-11T06:51:54.077818+0000","last_clean":"2024-09-11T06:51:54.077818+0000","last_became_active":"2024-09-11T06:51:37.950286+0000","last_became_peered":"2024-09-11T06:51:37.950286+0000","last_unstale":"2024-09-11T06:51:54.077818+0000","last_undegraded":"2024-09-11T06:51:54.077818+0000","last_fullsized":"2024-09-11T06:51:54.077818+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:36:32.689054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064404600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":160,"num_read_kb":8320,"num_write":79,"num_write_kb":16426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":25165938,"num_keys_recovered":9,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"111'203","reported_seq":477,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124866+0000","last_change":"2024-09-11T06:51:34.142678+0000","last_active":"2024-09-11T06:51:54.124866+0000","last_peered":"2024-09-11T06:51:54.124866+0000","last_clean":"2024-09-11T06:51:54.124866+0000","last_became_active":"2024-09-11T06:50:35.238011+0000","last_became_peered":"2024-09-11T06:50:35.238011+0000","last_unstale":"2024-09-11T06:51:54.124866+0000","last_undegraded":"2024-09-11T06:51:54.124866+0000","last_fullsized":"2024-09-11T06:51:54.124866+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:05:59.612752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027097340000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":505,"num_read_kb":21988,"num_write":142,"num_write_kb":67618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":50331648,"num_keys_recovered":16,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6945,"num_read_kb":120329,"num_write":989,"num_write_kb":272959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":260112984,"num_keys_recovered":41,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1126,"ondisk_log_size":1126,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640293,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1556480,"data_stored":601064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640294,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1552384,"data_stored":601045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640294,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33288,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2146304,"data_stored":1191413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640295,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1552384,"data_stored":601045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640295,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1556480,"data_stored":601064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640296,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2134016,"data_stored":1186835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:51:57.272 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T06:51:57.548 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:51:57.549 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":130,"fsid":"bd5dcb33-e9f7-4a5f-a26c-a451452e8aa2","created":"2024-09-11T06:48:52.191706+0000","modified":"2024-09-11T06:51:54.066047+0000","last_up_change":"2024-09-11T06:49:04.007203+0000","last_in_change":"2024-09-11T06:51:36.934758+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T06:49:05.084515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-11T06:49:09.530167+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":7,"snap_epoch":111,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c9e35bf-d3db-49db-b10c-d2cf38c7eb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1158147777},{"type":"v1","addr":"172.21.15.71:6801","nonce":1158147777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1158147777},{"type":"v1","addr":"172.21.15.71:6803","nonce":1158147777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1158147777},{"type":"v1","addr":"172.21.15.71:6807","nonce":1158147777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1158147777},{"type":"v1","addr":"172.21.15.71:6805","nonce":1158147777}]},"public_addr":"172.21.15.71:6801/1158147777","cluster_addr":"172.21.15.71:6803/1158147777","heartbeat_back_addr":"172.21.15.71:6807/1158147777","heartbeat_front_addr":"172.21.15.71:6805/1158147777","state":["exists","up"]},{"osd":1,"uuid":"693649dc-db7c-43cf-bdbe-87f6128ef311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":620364628},{"type":"v1","addr":"172.21.15.71:6809","nonce":620364628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":620364628},{"type":"v1","addr":"172.21.15.71:6811","nonce":620364628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":620364628},{"type":"v1","addr":"172.21.15.71:6815","nonce":620364628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":620364628},{"type":"v1","addr":"172.21.15.71:6813","nonce":620364628}]},"public_addr":"172.21.15.71:6809/620364628","cluster_addr":"172.21.15.71:6811/620364628","heartbeat_back_addr":"172.21.15.71:6815/620364628","heartbeat_front_addr":"172.21.15.71:6813/620364628","state":["exists","up"]},{"osd":2,"uuid":"89421ae7-fead-409c-83f7-ede424db6004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2926882521},{"type":"v1","addr":"172.21.15.71:6817","nonce":2926882521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2926882521},{"type":"v1","addr":"172.21.15.71:6819","nonce":2926882521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2926882521},{"type":"v1","addr":"172.21.15.71:6823","nonce":2926882521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2926882521},{"type":"v1","addr":"172.21.15.71:6821","nonce":2926882521}]},"public_addr":"172.21.15.71:6817/2926882521","cluster_addr":"172.21.15.71:6819/2926882521","heartbeat_back_addr":"172.21.15.71:6823/2926882521","heartbeat_front_addr":"172.21.15.71:6821/2926882521","state":["exists","up"]},{"osd":3,"uuid":"f4512e48-63f1-42b6-81be-594bd82e3210","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":558942315},{"type":"v1","addr":"172.21.15.89:6809","nonce":558942315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":558942315},{"type":"v1","addr":"172.21.15.89:6811","nonce":558942315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":558942315},{"type":"v1","addr":"172.21.15.89:6815","nonce":558942315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":558942315},{"type":"v1","addr":"172.21.15.89:6813","nonce":558942315}]},"public_addr":"172.21.15.89:6809/558942315","cluster_addr":"172.21.15.89:6811/558942315","heartbeat_back_addr":"172.21.15.89:6815/558942315","heartbeat_front_addr":"172.21.15.89:6813/558942315","state":["exists","up"]},{"osd":4,"uuid":"52d47b6f-af30-455e-8326-e53c598911ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":3878488003},{"type":"v1","addr":"172.21.15.89:6817","nonce":3878488003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":3878488003},{"type":"v1","addr":"172.21.15.89:6819","nonce":3878488003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":3878488003},{"type":"v1","addr":"172.21.15.89:6823","nonce":3878488003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":3878488003},{"type":"v1","addr":"172.21.15.89:6821","nonce":3878488003}]},"public_addr":"172.21.15.89:6817/3878488003","cluster_addr":"172.21.15.89:6819/3878488003","heartbeat_back_addr":"172.21.15.89:6823/3878488003","heartbeat_front_addr":"172.21.15.89:6821/3878488003","state":["exists","up"]},{"osd":5,"uuid":"6b4f9416-0ba0-4c58-883e-8cbdb362adcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2840204107},{"type":"v1","addr":"172.21.15.89:6801","nonce":2840204107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2840204107},{"type":"v1","addr":"172.21.15.89:6803","nonce":2840204107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2840204107},{"type":"v1","addr":"172.21.15.89:6807","nonce":2840204107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2840204107},{"type":"v1","addr":"172.21.15.89:6805","nonce":2840204107}]},"public_addr":"172.21.15.89:6801/2840204107","cluster_addr":"172.21.15.89:6803/2840204107","heartbeat_back_addr":"172.21.15.89:6807/2840204107","heartbeat_front_addr":"172.21.15.89:6805/2840204107","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.282785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.381259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.986977+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.732329+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.844988+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T06:49:00.639989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T06:51:58.566 INFO:tasks.ceph:Scrubbing osd.0 2024-09-11T06:51:58.567 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-11T06:51:58.721 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:58.721 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T06:51:58.721 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:58.737 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-11T06:51:59.013 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 0 to deep-scrub 2024-09-11T06:51:59.030 INFO:tasks.ceph:Scrubbing osd.1 2024-09-11T06:51:59.031 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-11T06:51:59.183 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:59.183 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T06:51:59.183 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:59.199 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-11T06:51:59.485 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 1 to deep-scrub 2024-09-11T06:51:59.502 INFO:tasks.ceph:Scrubbing osd.2 2024-09-11T06:51:59.503 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-11T06:51:59.639 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:51:59.640 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T06:51:59.640 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:51:59.654 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-11T06:51:59.918 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 2 to deep-scrub 2024-09-11T06:51:59.935 INFO:tasks.ceph:Scrubbing osd.3 2024-09-11T06:51:59.935 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-11T06:52:00.086 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:52:00.086 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T06:52:00.086 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:52:00.103 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-11T06:52:00.387 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 3 to deep-scrub 2024-09-11T06:52:00.405 INFO:tasks.ceph:Scrubbing osd.4 2024-09-11T06:52:00.405 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-11T06:52:00.551 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:52:00.551 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T06:52:00.551 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:52:00.566 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-11T06:52:00.832 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 4 to deep-scrub 2024-09-11T06:52:00.850 INFO:tasks.ceph:Scrubbing osd.5 2024-09-11T06:52:00.850 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-11T06:52:00.983 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T06:52:00.983 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T06:52:00.983 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T06:52:00.998 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-11T06:52:01.273 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 5 to deep-scrub 2024-09-11T06:52:01.291 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:52:01.572 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:52:01.572 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:52:01.589 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2024-09-11T06:52:01.052740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7061,"num_read_kb":120429,"num_write":1148,"num_write_kb":275375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":262474456,"num_keys_recovered":41,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1243,"ondisk_log_size":1243,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":197400,"kb_used_data":10264,"kb_used_omap":9,"kb_used_meta":186998,"kb_avail":566033640,"statfs":{"total":579820584960,"available":579618447360,"internally_reserved":0,"allocated":10510336,"data_stored":4787044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9565,"internal_metadata":191486627},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001963"},"pg_stats":[{"pgid":"1.0","version":"14'117","reported_seq":398,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124649+0000","last_change":"2024-09-11T06:50:37.941212+0000","last_active":"2024-09-11T06:51:54.124649+0000","last_peered":"2024-09-11T06:51:54.124649+0000","last_clean":"2024-09-11T06:51:54.124649+0000","last_became_active":"2024-09-11T06:50:37.940448+0000","last_became_peered":"2024-09-11T06:50:37.940448+0000","last_unstale":"2024-09-11T06:51:54.124649+0000","last_undegraded":"2024-09-11T06:51:54.124649+0000","last_fullsized":"2024-09-11T06:51:54.124649+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:05.088839+0000","last_clean_scrub_stamp":"2024-09-11T06:49:05.088839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:48:59.833858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"111'128","reported_seq":438,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124929+0000","last_change":"2024-09-11T06:51:34.211062+0000","last_active":"2024-09-11T06:51:54.124929+0000","last_peered":"2024-09-11T06:51:54.124929+0000","last_clean":"2024-09-11T06:51:54.124929+0000","last_became_active":"2024-09-11T06:50:47.306211+0000","last_became_peered":"2024-09-11T06:50:47.306211+0000","last_unstale":"2024-09-11T06:51:54.124929+0000","last_undegraded":"2024-09-11T06:51:54.124929+0000","last_fullsized":"2024-09-11T06:51:54.124929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:30:28.938965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000473933,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":383,"num_read_kb":8504,"num_write":98,"num_write_kb":28714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":25165862,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"111'89","reported_seq":400,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124735+0000","last_change":"2024-09-11T06:51:37.987266+0000","last_active":"2024-09-11T06:51:54.124735+0000","last_peered":"2024-09-11T06:51:54.124735+0000","last_clean":"2024-09-11T06:51:54.124735+0000","last_became_active":"2024-09-11T06:51:37.955297+0000","last_became_peered":"2024-09-11T06:51:37.955297+0000","last_unstale":"2024-09-11T06:51:54.124735+0000","last_undegraded":"2024-09-11T06:51:54.124735+0000","last_fullsized":"2024-09-11T06:51:54.124735+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:25.093793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098377899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":157,"num_read_kb":23067,"num_write":66,"num_write_kb":45059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":62914560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"111'255","reported_seq":2292,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.077049+0000","last_change":"2024-09-11T06:51:40.956789+0000","last_active":"2024-09-11T06:51:54.077049+0000","last_peered":"2024-09-11T06:51:54.077049+0000","last_clean":"2024-09-11T06:51:54.077049+0000","last_became_active":"2024-09-11T06:51:40.955516+0000","last_became_peered":"2024-09-11T06:51:40.955516+0000","last_unstale":"2024-09-11T06:51:54.077049+0000","last_undegraded":"2024-09-11T06:51:54.077049+0000","last_fullsized":"2024-09-11T06:51:54.077049+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:32:24.031714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017208530000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4173,"num_read_kb":27885,"num_write":144,"num_write_kb":41060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":25231476,"num_keys_recovered":13,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"111'12","reported_seq":273,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124950+0000","last_change":"2024-09-11T06:51:37.965118+0000","last_active":"2024-09-11T06:51:54.124950+0000","last_peered":"2024-09-11T06:51:54.124950+0000","last_clean":"2024-09-11T06:51:54.124950+0000","last_became_active":"2024-09-11T06:51:37.952071+0000","last_became_peered":"2024-09-11T06:51:37.952071+0000","last_unstale":"2024-09-11T06:51:54.124950+0000","last_undegraded":"2024-09-11T06:51:54.124950+0000","last_fullsized":"2024-09-11T06:51:54.124950+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:36:40.530470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011519830000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":93,"num_read_kb":90,"num_write":10,"num_write_kb":8196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":16777446,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"111'201","reported_seq":544,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.077858+0000","last_change":"2024-09-11T06:51:40.961258+0000","last_active":"2024-09-11T06:51:54.077858+0000","last_peered":"2024-09-11T06:51:54.077858+0000","last_clean":"2024-09-11T06:51:54.077858+0000","last_became_active":"2024-09-11T06:51:40.959678+0000","last_became_peered":"2024-09-11T06:51:40.959678+0000","last_unstale":"2024-09-11T06:51:54.077858+0000","last_undegraded":"2024-09-11T06:51:54.077858+0000","last_fullsized":"2024-09-11T06:51:54.077858+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:17.319791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012221140000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":855,"num_read_kb":8871,"num_write":301,"num_write_kb":20705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":12582950,"num_keys_recovered":3,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"111'170","reported_seq":684,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124770+0000","last_change":"2024-09-11T06:51:34.143092+0000","last_active":"2024-09-11T06:51:54.124770+0000","last_peered":"2024-09-11T06:51:54.124770+0000","last_clean":"2024-09-11T06:51:54.124770+0000","last_became_active":"2024-09-11T06:50:35.236333+0000","last_became_peered":"2024-09-11T06:50:35.236333+0000","last_unstale":"2024-09-11T06:51:54.124770+0000","last_undegraded":"2024-09-11T06:51:54.124770+0000","last_fullsized":"2024-09-11T06:51:54.124770+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:18:39.890958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003008405,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":619,"num_read_kb":21604,"num_write":149,"num_write_kb":45181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":41943104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"111'68","reported_seq":50,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.077818+0000","last_change":"2024-09-11T06:51:37.965289+0000","last_active":"2024-09-11T06:51:54.077818+0000","last_peered":"2024-09-11T06:51:54.077818+0000","last_clean":"2024-09-11T06:51:54.077818+0000","last_became_active":"2024-09-11T06:51:37.950286+0000","last_became_peered":"2024-09-11T06:51:37.950286+0000","last_unstale":"2024-09-11T06:51:54.077818+0000","last_undegraded":"2024-09-11T06:51:54.077818+0000","last_fullsized":"2024-09-11T06:51:54.077818+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:36:32.689054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064404600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":160,"num_read_kb":8320,"num_write":79,"num_write_kb":16426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":25165938,"num_keys_recovered":9,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"111'203","reported_seq":477,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:51:54.124866+0000","last_change":"2024-09-11T06:51:34.142678+0000","last_active":"2024-09-11T06:51:54.124866+0000","last_peered":"2024-09-11T06:51:54.124866+0000","last_clean":"2024-09-11T06:51:54.124866+0000","last_became_active":"2024-09-11T06:50:35.238011+0000","last_became_peered":"2024-09-11T06:50:35.238011+0000","last_unstale":"2024-09-11T06:51:54.124866+0000","last_undegraded":"2024-09-11T06:51:54.124866+0000","last_fullsized":"2024-09-11T06:51:54.124866+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T06:49:10.202937+0000","last_clean_scrub_stamp":"2024-09-11T06:49:10.202937+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:05:59.612752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027097340000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":505,"num_read_kb":21988,"num_write":142,"num_write_kb":67618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":50331648,"num_keys_recovered":16,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6945,"num_read_kb":120329,"num_write":989,"num_write_kb":272959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":260112984,"num_keys_recovered":41,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1126,"ondisk_log_size":1126,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640294,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1556480,"data_stored":601064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640295,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1552384,"data_stored":601045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640295,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33288,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2146304,"data_stored":1191413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640296,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1552384,"data_stored":601045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640296,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1556480,"data_stored":601064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640297,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33284,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338556,"statfs":{"total":96636764160,"available":96602681344,"internally_reserved":0,"allocated":2146304,"data_stored":1191413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:52:01.592 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-11T06:49:05.088839+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=6, tm_min=49, tm_sec=5, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=6, tm_min=51, tm_sec=57, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T06:52:01.593 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-11T06:49:10.202937+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=6, tm_min=49, tm_sec=10, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=6, tm_min=51, tm_sec=57, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T06:52:01.593 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-11T06:49:10.202937+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=6, tm_min=49, tm_sec=10, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=6, tm_min=51, tm_sec=57, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T06:52:01.593 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-11T06:49:10.202937+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=6, tm_min=49, tm_sec=10, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=6, tm_min=51, tm_sec=57, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T06:52:01.593 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-11T06:49:10.202937+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=6, tm_min=49, tm_sec=10, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=6, tm_min=51, tm_sec=57, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T06:52:01.593 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-11T06:49:10.202937+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=6, tm_min=49, tm_sec=10, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=6, tm_min=51, tm_sec=57, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T06:52:01.593 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-11T06:49:10.202937+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=6, tm_min=49, tm_sec=10, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=6, tm_min=51, tm_sec=57, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T06:52:01.593 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-11T06:49:10.202937+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=6, tm_min=49, tm_sec=10, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=6, tm_min=51, tm_sec=57, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T06:52:01.594 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-11T06:49:10.202937+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=6, tm_min=49, tm_sec=10, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=6, tm_min=51, tm_sec=57, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T06:52:01.594 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-11T06:52:21.595 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T06:52:21.902 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T06:52:21.902 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T06:52:21.920 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":224,"stamp":"2024-09-11T06:52:21.057639+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7061,"num_read_kb":120429,"num_write":1148,"num_write_kb":275375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":262474456,"num_keys_recovered":41,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1246,"ondisk_log_size":1246,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":197372,"kb_used_data":10264,"kb_used_omap":9,"kb_used_meta":186998,"kb_avail":566033668,"statfs":{"total":579820584960,"available":579618476032,"internally_reserved":0,"allocated":10510336,"data_stored":4787044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9565,"internal_metadata":191486627},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002864"},"pg_stats":[{"pgid":"1.0","version":"130'119","reported_seq":409,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:52:02.576576+0000","last_change":"2024-09-11T06:52:02.576745+0000","last_active":"2024-09-11T06:52:02.576576+0000","last_peered":"2024-09-11T06:52:02.576576+0000","last_clean":"2024-09-11T06:52:02.576576+0000","last_became_active":"2024-09-11T06:50:37.940448+0000","last_became_peered":"2024-09-11T06:50:37.940448+0000","last_unstale":"2024-09-11T06:52:02.576576+0000","last_undegraded":"2024-09-11T06:52:02.576576+0000","last_fullsized":"2024-09-11T06:52:02.576576+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"130'119","last_scrub_stamp":"2024-09-11T06:52:02.576347+0000","last_deep_scrub":"130'119","last_deep_scrub_stamp":"2024-09-11T06:52:02.576347+0000","last_clean_scrub_stamp":"2024-09-11T06:52:02.576347+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:39:10.405050+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"111'128","reported_seq":447,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:52:01.800627+0000","last_change":"2024-09-11T06:52:01.800691+0000","last_active":"2024-09-11T06:52:01.800627+0000","last_peered":"2024-09-11T06:52:01.800627+0000","last_clean":"2024-09-11T06:52:01.800627+0000","last_became_active":"2024-09-11T06:50:47.306211+0000","last_became_peered":"2024-09-11T06:50:47.306211+0000","last_unstale":"2024-09-11T06:52:01.800627+0000","last_undegraded":"2024-09-11T06:52:01.800627+0000","last_fullsized":"2024-09-11T06:52:01.800627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'128","last_scrub_stamp":"2024-09-11T06:52:01.800433+0000","last_deep_scrub":"111'128","last_deep_scrub_stamp":"2024-09-11T06:52:01.800433+0000","last_clean_scrub_stamp":"2024-09-11T06:52:01.800433+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:30.347606+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.000473933,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":383,"num_read_kb":8504,"num_write":98,"num_write_kb":28714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":25165862,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"111'89","reported_seq":409,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:52:03.551019+0000","last_change":"2024-09-11T06:52:03.551084+0000","last_active":"2024-09-11T06:52:03.551019+0000","last_peered":"2024-09-11T06:52:03.551019+0000","last_clean":"2024-09-11T06:52:03.551019+0000","last_became_active":"2024-09-11T06:51:37.955297+0000","last_became_peered":"2024-09-11T06:51:37.955297+0000","last_unstale":"2024-09-11T06:52:03.551019+0000","last_undegraded":"2024-09-11T06:52:03.551019+0000","last_fullsized":"2024-09-11T06:52:03.551019+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'89","last_scrub_stamp":"2024-09-11T06:52:03.550813+0000","last_deep_scrub":"111'89","last_deep_scrub_stamp":"2024-09-11T06:52:03.550813+0000","last_clean_scrub_stamp":"2024-09-11T06:52:03.550813+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:59.250667+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00098377899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":157,"num_read_kb":23067,"num_write":66,"num_write_kb":45059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":62914560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"111'255","reported_seq":2301,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:52:00.375177+0000","last_change":"2024-09-11T06:52:00.375247+0000","last_active":"2024-09-11T06:52:00.375177+0000","last_peered":"2024-09-11T06:52:00.375177+0000","last_clean":"2024-09-11T06:52:00.375177+0000","last_became_active":"2024-09-11T06:51:40.955516+0000","last_became_peered":"2024-09-11T06:51:40.955516+0000","last_unstale":"2024-09-11T06:52:00.375177+0000","last_undegraded":"2024-09-11T06:52:00.375177+0000","last_fullsized":"2024-09-11T06:52:00.375177+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'255","last_scrub_stamp":"2024-09-11T06:52:00.374972+0000","last_deep_scrub":"111'255","last_deep_scrub_stamp":"2024-09-11T06:52:00.374972+0000","last_clean_scrub_stamp":"2024-09-11T06:52:00.374972+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:51:43.517610+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0017208530000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4173,"num_read_kb":27885,"num_write":144,"num_write_kb":41060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":25231476,"num_keys_recovered":13,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"130'13","reported_seq":283,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:52:02.844863+0000","last_change":"2024-09-11T06:52:02.844937+0000","last_active":"2024-09-11T06:52:02.844863+0000","last_peered":"2024-09-11T06:52:02.844863+0000","last_clean":"2024-09-11T06:52:02.844863+0000","last_became_active":"2024-09-11T06:51:37.952071+0000","last_became_peered":"2024-09-11T06:51:37.952071+0000","last_unstale":"2024-09-11T06:52:02.844863+0000","last_undegraded":"2024-09-11T06:52:02.844863+0000","last_fullsized":"2024-09-11T06:52:02.844863+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"130'13","last_scrub_stamp":"2024-09-11T06:52:02.844662+0000","last_deep_scrub":"130'13","last_deep_scrub_stamp":"2024-09-11T06:52:02.844662+0000","last_clean_scrub_stamp":"2024-09-11T06:52:02.844662+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:24:41.222231+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0011519830000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":93,"num_read_kb":90,"num_write":10,"num_write_kb":8196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":16777446,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"111'201","reported_seq":553,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:52:01.244950+0000","last_change":"2024-09-11T06:52:01.244991+0000","last_active":"2024-09-11T06:52:01.244950+0000","last_peered":"2024-09-11T06:52:01.244950+0000","last_clean":"2024-09-11T06:52:01.244950+0000","last_became_active":"2024-09-11T06:51:40.959678+0000","last_became_peered":"2024-09-11T06:51:40.959678+0000","last_unstale":"2024-09-11T06:52:01.244950+0000","last_undegraded":"2024-09-11T06:52:01.244950+0000","last_fullsized":"2024-09-11T06:52:01.244950+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'201","last_scrub_stamp":"2024-09-11T06:52:01.244839+0000","last_deep_scrub":"111'201","last_deep_scrub_stamp":"2024-09-11T06:52:01.244839+0000","last_clean_scrub_stamp":"2024-09-11T06:52:01.244839+0000","objects_scrubbed":2,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:25:44.951334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012221140000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":855,"num_read_kb":8871,"num_write":301,"num_write_kb":20705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":12582950,"num_keys_recovered":3,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"111'170","reported_seq":693,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:52:00.579212+0000","last_change":"2024-09-11T06:52:00.579278+0000","last_active":"2024-09-11T06:52:00.579212+0000","last_peered":"2024-09-11T06:52:00.579212+0000","last_clean":"2024-09-11T06:52:00.579212+0000","last_became_active":"2024-09-11T06:50:35.236333+0000","last_became_peered":"2024-09-11T06:50:35.236333+0000","last_unstale":"2024-09-11T06:52:00.579212+0000","last_undegraded":"2024-09-11T06:52:00.579212+0000","last_fullsized":"2024-09-11T06:52:00.579212+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'170","last_scrub_stamp":"2024-09-11T06:52:00.579016+0000","last_deep_scrub":"111'170","last_deep_scrub_stamp":"2024-09-11T06:52:00.579016+0000","last_clean_scrub_stamp":"2024-09-11T06:52:00.579016+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:18:35.915597+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.003008405,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":619,"num_read_kb":21604,"num_write":149,"num_write_kb":45181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":41943104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"111'68","reported_seq":59,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:52:00.285604+0000","last_change":"2024-09-11T06:52:00.285686+0000","last_active":"2024-09-11T06:52:00.285604+0000","last_peered":"2024-09-11T06:52:00.285604+0000","last_clean":"2024-09-11T06:52:00.285604+0000","last_became_active":"2024-09-11T06:51:37.950286+0000","last_became_peered":"2024-09-11T06:51:37.950286+0000","last_unstale":"2024-09-11T06:52:00.285604+0000","last_undegraded":"2024-09-11T06:52:00.285604+0000","last_fullsized":"2024-09-11T06:52:00.285604+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'68","last_scrub_stamp":"2024-09-11T06:52:00.285397+0000","last_deep_scrub":"111'68","last_deep_scrub_stamp":"2024-09-11T06:52:00.285397+0000","last_clean_scrub_stamp":"2024-09-11T06:52:00.285397+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:04:19.353549+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00064404600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":160,"num_read_kb":8320,"num_write":79,"num_write_kb":16426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":25165938,"num_keys_recovered":9,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"111'203","reported_seq":486,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-11T06:52:01.607167+0000","last_change":"2024-09-11T06:52:01.607288+0000","last_active":"2024-09-11T06:52:01.607167+0000","last_peered":"2024-09-11T06:52:01.607167+0000","last_clean":"2024-09-11T06:52:01.607167+0000","last_became_active":"2024-09-11T06:50:35.238011+0000","last_became_peered":"2024-09-11T06:50:35.238011+0000","last_unstale":"2024-09-11T06:52:01.607167+0000","last_undegraded":"2024-09-11T06:52:01.607167+0000","last_fullsized":"2024-09-11T06:52:01.607167+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'203","last_scrub_stamp":"2024-09-11T06:52:01.606908+0000","last_deep_scrub":"111'203","last_deep_scrub_stamp":"2024-09-11T06:52:01.606908+0000","last_clean_scrub_stamp":"2024-09-11T06:52:01.606908+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:16:13.588222+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0027097340000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":505,"num_read_kb":21988,"num_write":142,"num_write_kb":67618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":50331648,"num_keys_recovered":16,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6945,"num_read_kb":120329,"num_write":989,"num_write_kb":272959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":260112984,"num_keys_recovered":41,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1127,"ondisk_log_size":1127,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640298,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1556480,"data_stored":601064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640299,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1552384,"data_stored":601045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640299,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33280,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2146304,"data_stored":1191413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640300,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1552384,"data_stored":601045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640300,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1556480,"data_stored":601064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640301,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33280,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2146304,"data_stored":1191413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T06:52:21.922 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-11T06:52:22.249 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-11T06:52:22.250 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-11T06:52:22.250 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-11T06:52:22.250 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T06:52:22.328 INFO:tasks.ceph.osd.0:Stopped 2024-09-11T06:52:22.328 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-11T06:52:22.328 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T06:52:22.426 INFO:tasks.ceph.osd.1:Stopped 2024-09-11T06:52:22.426 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-11T06:52:22.427 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T06:52:27.870 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-11T06:52:28.529 INFO:tasks.ceph.osd.2:Stopped 2024-09-11T06:52:28.529 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-11T06:52:28.529 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T06:52:28.600 INFO:tasks.ceph.osd.3:Stopped 2024-09-11T06:52:28.600 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-11T06:52:28.600 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T06:52:28.690 INFO:tasks.ceph.osd.4:Stopped 2024-09-11T06:52:28.690 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-11T06:52:28.691 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T06:52:28.754 INFO:tasks.ceph.osd.5:Stopped 2024-09-11T06:52:28.754 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-11T06:52:28.754 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-11T06:52:28.755 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T06:52:28.784 INFO:tasks.ceph.mgr.x:Stopped 2024-09-11T06:52:28.785 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-11T06:52:28.785 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-11T06:52:28.785 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T06:52:28.809 INFO:tasks.ceph.mon.a:Stopped 2024-09-11T06:52:28.809 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-11T06:52:28.809 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T06:52:28.874 INFO:tasks.ceph.mon.c:Stopped 2024-09-11T06:52:28.875 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-11T06:52:28.875 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T06:52:28.890 INFO:tasks.ceph.mon.b:Stopped 2024-09-11T06:52:28.890 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-11T06:52:28.890 DEBUG:teuthology.orchestra.run.smithi071:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-11T06:52:28.934 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi071.front.sepia.ceph.com 2024-09-11T06:52:28.934 DEBUG:teuthology.orchestra.run.smithi071:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-11T06:52:29.264 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi071.front.sepia.ceph.com 2024-09-11T06:52:29.264 DEBUG:teuthology.orchestra.run.smithi071:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-11T06:52:29.457 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi071.front.sepia.ceph.com 2024-09-11T06:52:29.457 DEBUG:teuthology.orchestra.run.smithi071:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-11T06:52:29.650 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi089.front.sepia.ceph.com 2024-09-11T06:52:29.651 DEBUG:teuthology.orchestra.run.smithi089:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-11T06:52:29.903 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi089.front.sepia.ceph.com 2024-09-11T06:52:29.903 DEBUG:teuthology.orchestra.run.smithi089:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-11T06:52:30.098 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi089.front.sepia.ceph.com 2024-09-11T06:52:30.098 DEBUG:teuthology.orchestra.run.smithi089:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-11T06:52:30.254 INFO:tasks.ceph:Archiving mon data... 2024-09-11T06:52:30.255 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900631/data/mon.a.tgz 2024-09-11T06:52:30.256 DEBUG:teuthology.orchestra.run.smithi071:> mktemp 2024-09-11T06:52:30.262 INFO:teuthology.orchestra.run.smithi071.stdout:/tmp/tmp.n33KVqf9B0 2024-09-11T06:52:30.262 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.n33KVqf9B0 2024-09-11T06:52:30.385 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0666 /tmp/tmp.n33KVqf9B0 2024-09-11T06:52:30.459 DEBUG:teuthology.orchestra.remote:smithi071:/tmp/tmp.n33KVqf9B0 is 287KB 2024-09-11T06:52:30.619 DEBUG:teuthology.orchestra.run.smithi071:> rm -fr /tmp/tmp.n33KVqf9B0 2024-09-11T06:52:30.625 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900631/data/mon.c.tgz 2024-09-11T06:52:30.625 DEBUG:teuthology.orchestra.run.smithi071:> mktemp 2024-09-11T06:52:30.677 INFO:teuthology.orchestra.run.smithi071.stdout:/tmp/tmp.uimIITXDp5 2024-09-11T06:52:30.677 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.uimIITXDp5 2024-09-11T06:52:30.829 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0666 /tmp/tmp.uimIITXDp5 2024-09-11T06:52:30.896 DEBUG:teuthology.orchestra.remote:smithi071:/tmp/tmp.uimIITXDp5 is 394KB 2024-09-11T06:52:30.955 DEBUG:teuthology.orchestra.run.smithi071:> rm -fr /tmp/tmp.uimIITXDp5 2024-09-11T06:52:30.959 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900631/data/mon.b.tgz 2024-09-11T06:52:30.960 DEBUG:teuthology.orchestra.run.smithi089:> mktemp 2024-09-11T06:52:30.965 INFO:teuthology.orchestra.run.smithi089.stdout:/tmp/tmp.lKlSZKv2fo 2024-09-11T06:52:30.966 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.lKlSZKv2fo 2024-09-11T06:52:31.109 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0666 /tmp/tmp.lKlSZKv2fo 2024-09-11T06:52:31.174 DEBUG:teuthology.orchestra.remote:smithi089:/tmp/tmp.lKlSZKv2fo is 394KB 2024-09-11T06:52:31.191 DEBUG:teuthology.orchestra.run.smithi089:> rm -fr /tmp/tmp.lKlSZKv2fo 2024-09-11T06:52:31.196 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-11T06:52:31.196 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-11T06:52:31.199 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-11T06:52:31.239 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-11T06:52:31.259 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-11T06:52:31.297 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-11T06:52:31.303 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-11T06:52:31.303 INFO:tasks.ceph:Archiving crash dumps... 2024-09-11T06:52:31.305 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900631/remote/smithi071/crash 2024-09-11T06:52:31.305 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-11T06:52:31.319 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900631/remote/smithi089/crash 2024-09-11T06:52:31.320 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-11T06:52:31.354 DEBUG:teuthology.misc:Transferring archived files from smithi176:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900631/remote/smithi176/crash 2024-09-11T06:52:31.355 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-11T06:52:31.367 INFO:tasks.ceph:Compressing logs... 2024-09-11T06:52:31.367 DEBUG:teuthology.orchestra.run.smithi071:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T06:52:31.370 DEBUG:teuthology.orchestra.run.smithi089:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T06:52:31.399 DEBUG:teuthology.orchestra.run.smithi176:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T06:52:31.408 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36882.log 2024-09-11T06:52:31.409 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31282.log 2024-09-11T06:52:31.409 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36882.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36882.log.gz 2024-09-11T06:52:31.409 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35939.log 2024-09-11T06:52:31.412 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37740.log 2024-09-11T06:52:31.412 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.31282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29721.log 2024-09-11T06:52:31.412 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31282.log.gz 2024-09-11T06:52:31.412 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35939.log: /var/log/ceph/ceph-client.admin.37740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31460.log 2024-09-11T06:52:31.412 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35939.log.gz 2024-09-11T06:52:31.412 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37740.log.gz 2024-09-11T06:52:31.413 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.29721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29650.log 2024-09-11T06:52:31.413 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29721.log.gz 2024-09-11T06:52:31.413 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.31460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36517.log 2024-09-11T06:52:31.413 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31460.log.gz 2024-09-11T06:52:31.413 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.29650.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29650.log.gz 2024-09-11T06:52:31.413 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37324.log 2024-09-11T06:52:31.413 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33133.log 2024-09-11T06:52:31.413 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36517.log.gz 2024-09-11T06:52:31.419 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34099.log 2024-09-11T06:52:31.419 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37324.log.gz 2024-09-11T06:52:31.419 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37272.log 2024-09-11T06:52:31.419 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33133.log.gz 2024-09-11T06:52:31.419 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34099.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34099.log.gz 2024-09-11T06:52:31.419 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30649.log 2024-09-11T06:52:31.419 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36287.log 2024-09-11T06:52:31.420 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37272.log.gz 2024-09-11T06:52:31.420 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.30649.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.30649.log.gz -- 2024-09-11T06:52:31.420 INFO:teuthology.orchestra.run.smithi071.stderr: /var/log/ceph/ceph-client.admin.31075.log 2024-09-11T06:52:31.420 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38000.log 2024-09-11T06:52:31.420 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36287.log.gz 2024-09-11T06:52:31.420 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25263.log 2024-09-11T06:52:31.420 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.31075.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31075.log.gz 2024-09-11T06:52:31.420 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30793.log 2024-09-11T06:52:31.420 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38000.log.gz 2024-09-11T06:52:31.420 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.25263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36385.log 2024-09-11T06:52:31.420 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25263.log.gz 2024-09-11T06:52:31.420 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.30793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33402.log 2024-09-11T06:52:31.420 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30793.log.gz 2024-09-11T06:52:31.421 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30059.log 2024-09-11T06:52:31.421 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36385.log.gz 2024-09-11T06:52:31.421 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32893.log 2024-09-11T06:52:31.421 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33402.log.gz 2024-09-11T06:52:31.421 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.30059.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30059.log.gz 2024-09-11T06:52:31.421 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33047.log 2024-09-11T06:52:31.421 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.32893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30258.log 2024-09-11T06:52:31.421 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32893.log.gz 2024-09-11T06:52:31.421 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29572.log 2024-09-11T06:52:31.421 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33047.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33047.log.gz 2024-09-11T06:52:31.421 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.30258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37383.log 2024-09-11T06:52:31.421 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30258.log.gz 2024-09-11T06:52:31.421 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.29572.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29572.log.gz 2024-09-11T06:52:31.422 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32663.log 2024-09-11T06:52:31.427 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25190.log 2024-09-11T06:52:31.428 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25070.log 2024-09-11T06:52:31.428 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.25190.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-11T06:52:31.428 INFO:teuthology.orchestra.run.smithi089.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25190.log.gz 2024-09-11T06:52:31.428 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.25070.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25070.log.gz 2024-09-11T06:52:31.428 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-11T06:52:31.428 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25118.log 2024-09-11T06:52:31.428 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25094.log 2024-09-11T06:52:31.428 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.25118.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-11T06:52:31.428 INFO:teuthology.orchestra.run.smithi089.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25118.log.gz 2024-09-11T06:52:31.428 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.25094.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-11T06:52:31.428 INFO:teuthology.orchestra.run.smithi089.stderr: 25.6% -- replaced with /var/log/ceph/ceph-client.admin.25094.log.gz 2024-09-11T06:52:31.428 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25166.log 2024-09-11T06:52:31.428 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25142.log 2024-09-11T06:52:31.429 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.25166.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.25166.log.gz --verbose 2024-09-11T06:52:31.429 INFO:teuthology.orchestra.run.smithi089.stderr: -- /var/log/ceph/ceph.audit.log 2024-09-11T06:52:31.429 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.25142.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25142.log.gz 2024-09-11T06:52:31.429 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-11T06:52:31.429 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-11T06:52:31.429 INFO:teuthology.orchestra.run.smithi089.stderr: 85.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-11T06:52:31.429 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-osd.4.log: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-11T06:52:31.482 INFO:teuthology.orchestra.run.smithi089.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-11T06:52:31.561 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36549.log 2024-09-11T06:52:31.561 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37383.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37383.log.gz 2024-09-11T06:52:31.561 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.32663.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-11T06:52:31.561 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32663.log.gz 2024-09-11T06:52:31.561 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36549.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36549.log.gz 2024-09-11T06:52:31.562 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36223.log 2024-09-11T06:52:31.562 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35244.log 2024-09-11T06:52:31.562 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30111.log 2024-09-11T06:52:31.562 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36223.log.gz 2024-09-11T06:52:31.562 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35244.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35244.log.gz 2024-09-11T06:52:31.562 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35975.log 2024-09-11T06:52:31.562 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.30111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33974.log 2024-09-11T06:52:31.562 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30111.log.gz 2024-09-11T06:52:31.562 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35975.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35975.log.gz 2024-09-11T06:52:31.562 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37090.log 2024-09-11T06:52:31.562 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34040.log 2024-09-11T06:52:31.562 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33974.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33974.log.gz 2024-09-11T06:52:31.562 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30137.log 2024-09-11T06:52:31.562 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37090.log.gz 2024-09-11T06:52:31.562 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29585.log 2024-09-11T06:52:31.563 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34040.log.gz 2024-09-11T06:52:31.563 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.30137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33347.log 2024-09-11T06:52:31.563 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30137.log.gz 2024-09-11T06:52:31.563 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.29585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37168.log 2024-09-11T06:52:31.563 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29585.log.gz 2024-09-11T06:52:31.563 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29958.log 2024-09-11T06:52:31.563 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33347.log.gz 2024-09-11T06:52:31.563 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33560.log 2024-09-11T06:52:31.563 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37168.log.gz 2024-09-11T06:52:31.563 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.29958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31903.log 2024-09-11T06:52:31.563 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29958.log.gz 2024-09-11T06:52:31.563 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37194.log 2024-09-11T06:52:31.563 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33560.log.gz 2024-09-11T06:52:31.563 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.31903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34192.log 2024-09-11T06:52:31.564 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31903.log.gz 2024-09-11T06:52:31.564 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30260.log 2024-09-11T06:52:31.564 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37194.log.gz 2024-09-11T06:52:31.564 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36341.log 2024-09-11T06:52:31.564 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34192.log.gz 2024-09-11T06:52:31.564 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5/var/log/ceph/ceph-client.admin.30260.log: --verbose -- /var/log/ceph/ceph-client.admin.37766.log 2024-09-11T06:52:31.564 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30260.log.gz 2024-09-11T06:52:31.564 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29513.log 2024-09-11T06:52:31.564 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36341.log.gz 2024-09-11T06:52:31.564 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37818.log 2024-09-11T06:52:31.564 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37766.log.gz 2024-09-11T06:52:31.564 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.29513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35506.log 2024-09-11T06:52:31.564 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29513.log.gz 2024-09-11T06:52:31.565 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37142.log 2024-09-11T06:52:31.565 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37818.log.gz 2024-09-11T06:52:31.565 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35139.log 2024-09-11T06:52:31.565 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35506.log.gz 2024-09-11T06:52:31.565 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34420.log 2024-09-11T06:52:31.565 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37142.log.gz 2024-09-11T06:52:31.565 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36574.log 2024-09-11T06:52:31.565 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35139.log.gz 2024-09-11T06:52:31.565 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37544.log 2024-09-11T06:52:31.565 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34420.log.gz 2024-09-11T06:52:31.565 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33916.log 2024-09-11T06:52:31.565 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36574.log.gz 2024-09-11T06:52:31.565 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34319.log 2024-09-11T06:52:31.565 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37544.log.gz 2024-09-11T06:52:31.566 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33916.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-11T06:52:31.566 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33916.log.gz 2024-09-11T06:52:31.566 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37116.log 2024-09-11T06:52:31.566 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34319.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34319.log.gz 2024-09-11T06:52:31.566 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30970.log 2024-09-11T06:52:31.566 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32032.log 2024-09-11T06:52:31.566 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37116.log.gz 2024-09-11T06:52:31.566 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.30970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35843.log 2024-09-11T06:52:31.566 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30970.log.gz 2024-09-11T06:52:31.566 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.32032.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32032.log.gz 2024-09-11T06:52:31.566 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35173.log 2024-09-11T06:52:31.566 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34969.log 2024-09-11T06:52:31.566 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35843.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35843.log.gz 2024-09-11T06:52:31.566 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35772.log 2024-09-11T06:52:31.566 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35173.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35173.log.gz 2024-09-11T06:52:31.567 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34969.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.34969.log.gz 2024-09-11T06:52:31.567 INFO:teuthology.orchestra.run.smithi071.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36725.log 2024-09-11T06:52:31.567 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30085.log 2024-09-11T06:52:31.567 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35772.log.gz 2024-09-11T06:52:31.567 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31862.log 2024-09-11T06:52:31.567 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36725.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36725.log.gz 2024-09-11T06:52:31.567 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.30085.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30085.log.gz 2024-09-11T06:52:31.567 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34387.log 2024-09-11T06:52:31.567 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.31862.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-11T06:52:31.567 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31862.log.gz 2024-09-11T06:52:31.567 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34387.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34387.log.gz 2024-09-11T06:52:31.567 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37572.log 2024-09-11T06:52:31.567 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37844.log 2024-09-11T06:52:31.567 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.37572.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37572.log.gz 2024-09-11T06:52:31.567 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36619.log 2024-09-11T06:52:31.567 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34222.log 2024-09-11T06:52:31.568 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37844.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37844.log.gz 2024-09-11T06:52:31.568 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36619.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36619.log.gz 2024-09-11T06:52:31.568 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34218.log 2024-09-11T06:52:31.568 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29251.log 2024-09-11T06:52:31.568 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34222.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34222.log.gz 2024-09-11T06:52:31.568 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34218.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34218.log.gz 2024-09-11T06:52:31.568 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29877.log 2024-09-11T06:52:31.568 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34557.log 2024-09-11T06:52:31.568 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.29251.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29251.log.gz 2024-09-11T06:52:31.568 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.29877.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29877.log.gz 2024-09-11T06:52:31.568 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36797.log 2024-09-11T06:52:31.568 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32485.log 2024-09-11T06:52:31.568 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34557.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34557.log.gz 2024-09-11T06:52:31.568 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37394.log 2024-09-11T06:52:31.568 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36797.log.gz 2024-09-11T06:52:31.569 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.32485.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32485.log.gz 2024-09-11T06:52:31.569 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32930.log 2024-09-11T06:52:31.569 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38026.log 2024-09-11T06:52:31.569 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37394.log: /var/log/ceph/ceph-client.admin.32930.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37394.log.gz 2024-09-11T06:52:31.569 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32930.log.gz 2024-09-11T06:52:31.569 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38052.log 2024-09-11T06:52:31.569 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35381.log 2024-09-11T06:52:31.569 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38026.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38026.log.gz 2024-09-11T06:52:31.569 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37393.log 2024-09-11T06:52:31.569 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38052.log.gz 2024-09-11T06:52:31.569 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35381.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35381.log.gz 2024-09-11T06:52:31.569 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37396.log 2024-09-11T06:52:31.569 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32299.log 2024-09-11T06:52:31.569 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37393.log: 0.0%/var/log/ceph/ceph-client.admin.37396.log: -- replaced with /var/log/ceph/ceph-client.admin.37393.log.gz 2024-09-11T06:52:31.569 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37396.log.gz 2024-09-11T06:52:31.570 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37569.log 2024-09-11T06:52:31.570 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35797.log 2024-09-11T06:52:31.570 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.32299.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32299.log.gz 2024-09-11T06:52:31.570 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37569.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37569.log.gz 2024-09-11T06:52:31.570 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35323.log 2024-09-11T06:52:31.570 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34994.log 2024-09-11T06:52:31.570 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35797.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35797.log.gz 2024-09-11T06:52:31.570 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35323.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.35323.log.gz -5 2024-09-11T06:52:31.570 INFO:teuthology.orchestra.run.smithi071.stderr: --verbose -- /var/log/ceph/ceph-client.admin.36464.log 2024-09-11T06:52:31.570 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34015.log 2024-09-11T06:52:31.570 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34994.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34994.log.gz 2024-09-11T06:52:31.570 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36955.log 2024-09-11T06:52:31.570 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36464.log.gz 2024-09-11T06:52:31.570 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34908.log 2024-09-11T06:52:31.571 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34015.log.gz 2024-09-11T06:52:31.571 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36955.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36955.log.gz 2024-09-11T06:52:31.571 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31101.log 2024-09-11T06:52:31.571 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30263.log 2024-09-11T06:52:31.571 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34908.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34908.log.gz 2024-09-11T06:52:31.571 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.31101.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31101.log.gz 2024-09-11T06:52:31.571 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36257.log 2024-09-11T06:52:31.571 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30924.log 2024-09-11T06:52:31.571 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.30263.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30263.log.gz 2024-09-11T06:52:31.571 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36257.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36257.log.gz 2024-09-11T06:52:31.571 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30974.log 2024-09-11T06:52:31.571 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.30924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29582.log 2024-09-11T06:52:31.571 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30924.log.gz 2024-09-11T06:52:31.571 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.30974.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30974.log.gz 2024-09-11T06:52:31.572 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37038.log 2024-09-11T06:52:31.572 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32589.log 2024-09-11T06:52:31.572 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.29582.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29582.log.gz 2024-09-11T06:52:31.572 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37038.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37038.log.gz 2024-09-11T06:52:31.572 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31763.log 2024-09-11T06:52:31.572 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30189.log 2024-09-11T06:52:31.572 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.32589.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32589.log.gz 2024-09-11T06:52:31.572 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.31763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36064.log 2024-09-11T06:52:31.572 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31763.log.gz 2024-09-11T06:52:31.572 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.30189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31197.log 2024-09-11T06:52:31.572 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30189.log.gz 2024-09-11T06:52:31.572 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36064.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36064.log.gz 2024-09-11T06:52:31.572 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33659.log 2024-09-11T06:52:31.572 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30033.log 2024-09-11T06:52:31.573 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.31197.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31197.log.gz 2024-09-11T06:52:31.573 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33659.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33659.log.gz 2024-09-11T06:52:31.573 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29539.log 2024-09-11T06:52:31.573 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33808.log 2024-09-11T06:52:31.573 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.30033.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30033.log.gz 2024-09-11T06:52:31.573 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.29539.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.29539.log.gz 2024-09-11T06:52:31.573 INFO:teuthology.orchestra.run.smithi071.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.29881.log 2024-09-11T06:52:31.573 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33186.log 2024-09-11T06:52:31.573 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33808.log.gz 2024-09-11T06:52:31.573 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.29881.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29881.log.gz 2024-09-11T06:52:31.573 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32412.log 2024-09-11T06:52:31.573 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38156.log 2024-09-11T06:52:31.573 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33186.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33186.log.gz 2024-09-11T06:52:31.574 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.32412.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32412.log.gz 2024-09-11T06:52:31.574 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32791.log 2024-09-11T06:52:31.574 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.21471.log 2024-09-11T06:52:31.574 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38156.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38156.log.gz 2024-09-11T06:52:31.574 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.32791.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32791.log.gz 2024-09-11T06:52:31.574 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35406.log 2024-09-11T06:52:31.574 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36612.log 2024-09-11T06:52:31.574 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph.tmp-client.admin.21471.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.21471.log.gz 2024-09-11T06:52:31.574 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35406.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35406.log.gz 2024-09-11T06:52:31.574 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37974.log 2024-09-11T06:52:31.574 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37064.log 2024-09-11T06:52:31.574 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36612.log.gz 2024-09-11T06:52:31.574 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36436.log 2024-09-11T06:52:31.574 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37974.log.gz 2024-09-11T06:52:31.574 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30736.log 2024-09-11T06:52:31.575 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37064.log.gz 2024-09-11T06:52:31.575 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36436.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36436.log.gz 2024-09-11T06:52:31.575 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35095.log 2024-09-11T06:52:31.575 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.30736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32250.log 2024-09-11T06:52:31.575 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30736.log.gz 2024-09-11T06:52:31.575 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35095.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35095.log.gz 2024-09-11T06:52:31.575 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35207.log 2024-09-11T06:52:31.575 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25069.log 2024-09-11T06:52:31.575 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.32250.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32250.log.gz 2024-09-11T06:52:31.575 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35207.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35207.log.gz 2024-09-11T06:52:31.575 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37350.log 2024-09-11T06:52:31.575 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30219.log 2024-09-11T06:52:31.575 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.25069.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25069.log.gz 2024-09-11T06:52:31.576 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36167.log 2024-09-11T06:52:31.576 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37350.log.gz 2024-09-11T06:52:31.576 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.30219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36763.log 2024-09-11T06:52:31.576 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30219.log.gz 2024-09-11T06:52:31.576 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36167.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-11T06:52:31.576 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36167.log.gz 2024-09-11T06:52:31.576 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37473.log 2024-09-11T06:52:31.576 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36763.log.gz 2024-09-11T06:52:31.576 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34585.log 2024-09-11T06:52:31.576 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36850.log 2024-09-11T06:52:31.576 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37473.log.gz 2024-09-11T06:52:31.576 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37395.log 2024-09-11T06:52:31.576 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34585.log.gz 2024-09-11T06:52:31.576 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29583.log 2024-09-11T06:52:31.577 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36850.log.gz 2024-09-11T06:52:31.577 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35068.log 2024-09-11T06:52:31.577 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37395.log.gz 2024-09-11T06:52:31.577 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.29583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34821.log 2024-09-11T06:52:31.577 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29583.log.gz 2024-09-11T06:52:31.577 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32332.log 2024-09-11T06:52:31.577 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35068.log.gz 2024-09-11T06:52:31.577 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25186.log 2024-09-11T06:52:31.577 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34821.log.gz 2024-09-11T06:52:31.577 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.32332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31729.log 2024-09-11T06:52:31.577 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32332.log.gz 2024-09-11T06:52:31.577 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.25186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31666.log 2024-09-11T06:52:31.577 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25186.log.gz 2024-09-11T06:52:31.577 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.31729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38104.log 2024-09-11T06:52:31.577 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31729.log.gz 2024-09-11T06:52:31.577 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.31666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32559.log 2024-09-11T06:52:31.578 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31666.log.gz 2024-09-11T06:52:31.578 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37792.log 2024-09-11T06:52:31.578 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38104.log.gz 2024-09-11T06:52:31.578 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.32559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36036.log 2024-09-11T06:52:31.578 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32559.log.gz 2024-09-11T06:52:31.578 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34755.log 2024-09-11T06:52:31.578 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37792.log.gz 2024-09-11T06:52:31.578 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30590.log 2024-09-11T06:52:31.578 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36036.log.gz 2024-09-11T06:52:31.578 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35625.log 2024-09-11T06:52:31.578 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34755.log.gz 2024-09-11T06:52:31.578 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.30590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31828.log 2024-09-11T06:52:31.578 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30590.log.gz 2024-09-11T06:52:31.578 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31036.log 2024-09-11T06:52:31.578 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35625.log.gz 2024-09-11T06:52:31.579 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.31828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35456.log 2024-09-11T06:52:31.579 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31828.log.gz 2024-09-11T06:52:31.579 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.31036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31169.log 2024-09-11T06:52:31.579 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31036.log.gz 2024-09-11T06:52:31.579 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35514.log 2024-09-11T06:52:31.579 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35456.log.gz 2024-09-11T06:52:31.579 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.31169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34109.log 2024-09-11T06:52:31.579 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31169.log.gz 2024-09-11T06:52:31.579 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30163.log 2024-09-11T06:52:31.579 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35514.log.gz 2024-09-11T06:52:31.579 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34109.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-11T06:52:31.579 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34109.log.gz 2024-09-11T06:52:31.579 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.30163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32664.log 2024-09-11T06:52:31.579 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30163.log.gz 2024-09-11T06:52:31.579 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29573.log 2024-09-11T06:52:31.579 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.32664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32982.log 2024-09-11T06:52:31.580 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32664.log.gz 2024-09-11T06:52:31.580 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.29573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36929.log 2024-09-11T06:52:31.580 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29573.log.gz 2024-09-11T06:52:31.580 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.32982.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32982.log.gz 2024-09-11T06:52:31.580 INFO:teuthology.orchestra.run.smithi071.stderr: 92.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-11T06:52:31.580 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37714.log 2024-09-11T06:52:31.580 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36692.log 2024-09-11T06:52:31.580 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36929.log.gz 2024-09-11T06:52:31.580 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34266.log 2024-09-11T06:52:31.580 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37714.log.gz 2024-09-11T06:52:31.580 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30527.log 2024-09-11T06:52:31.580 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36692.log.gz 2024-09-11T06:52:31.580 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30475.log 2024-09-11T06:52:31.581 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34266.log.gz 2024-09-11T06:52:31.581 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.30527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34860.log 2024-09-11T06:52:31.581 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30527.log.gz 2024-09-11T06:52:31.581 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.30475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34353.log 2024-09-11T06:52:31.581 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30475.log.gz 2024-09-11T06:52:31.581 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29885.log 2024-09-11T06:52:31.581 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34860.log.gz 2024-09-11T06:52:31.581 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33879.log 2024-09-11T06:52:31.581 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34353.log.gz 2024-09-11T06:52:31.581 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.29885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33048.log 2024-09-11T06:52:31.581 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29885.log.gz 2024-09-11T06:52:31.581 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30861.log 2024-09-11T06:52:31.582 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33879.log.gz 2024-09-11T06:52:31.582 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37530.log 2024-09-11T06:52:31.582 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33048.log.gz 2024-09-11T06:52:31.582 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.30861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37540.log 2024-09-11T06:52:31.582 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30861.log.gz 2024-09-11T06:52:31.582 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32386.log 2024-09-11T06:52:31.582 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37530.log.gz 2024-09-11T06:52:31.582 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38182.log 2024-09-11T06:52:31.582 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37540.log.gz 2024-09-11T06:52:31.582 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.32386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32473.log 2024-09-11T06:52:31.582 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32386.log.gz 2024-09-11T06:52:31.582 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37870.log 2024-09-11T06:52:31.582 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38182.log.gz 2024-09-11T06:52:31.582 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.32473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34458.log 2024-09-11T06:52:31.582 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32473.log.gz 2024-09-11T06:52:31.583 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37870.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-11T06:52:31.583 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37870.log.gz 2024-09-11T06:52:31.583 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32198.log 2024-09-11T06:52:31.583 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34458.log.gz 2024-09-11T06:52:31.583 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33012.log 2024-09-11T06:52:31.583 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.32198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35716.log 2024-09-11T06:52:31.583 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32198.log.gz 2024-09-11T06:52:31.583 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31135.log 2024-09-11T06:52:31.583 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33012.log.gz 2024-09-11T06:52:31.583 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36115.log 2024-09-11T06:52:31.583 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35716.log.gz 2024-09-11T06:52:31.583 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.31135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35877.log 2024-09-11T06:52:31.583 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31135.log.gz 2024-09-11T06:52:31.583 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38078.log 2024-09-11T06:52:31.583 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36115.log.gz 2024-09-11T06:52:31.584 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37948.log 2024-09-11T06:52:31.584 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35877.log.gz 2024-09-11T06:52:31.584 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31638.log 2024-09-11T06:52:31.584 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38078.log.gz 2024-09-11T06:52:31.584 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34066.log 2024-09-11T06:52:31.584 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37948.log.gz 2024-09-11T06:52:31.584 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.31638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34792.log 0.0% 2024-09-11T06:52:31.584 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31638.log.gz 2024-09-11T06:52:31.584 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34066.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34066.log.gz -5 2024-09-11T06:52:31.584 INFO:teuthology.orchestra.run.smithi071.stderr: --verbose -- /var/log/ceph/ceph-client.admin.33700.log 2024-09-11T06:52:31.584 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29729.log 2024-09-11T06:52:31.584 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34792.log.gz 2024-09-11T06:52:31.584 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33700.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33700.log.gz 2024-09-11T06:52:31.584 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30653.log 2024-09-11T06:52:31.584 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.29729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31573.log 2024-09-11T06:52:31.585 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29729.log.gz 2024-09-11T06:52:31.585 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.30653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31335.log 2024-09-11T06:52:31.585 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30653.log.gz 2024-09-11T06:52:31.585 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.31573.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31573.log.gz 2024-09-11T06:52:31.585 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31237.log 2024-09-11T06:52:31.585 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.31335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35911.log 2024-09-11T06:52:31.585 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31335.log.gz 2024-09-11T06:52:31.585 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.31237.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-11T06:52:31.585 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31237.log.gz 2024-09-11T06:52:31.585 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38130.log 2024-09-11T06:52:31.585 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35911.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35911.log.gz 2024-09-11T06:52:31.585 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29754.log 2024-09-11T06:52:31.585 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37012.log 2024-09-11T06:52:31.585 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38130.log: /var/log/ceph/ceph-client.admin.29754.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38130.log.gz 2024-09-11T06:52:31.586 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29754.log.gz 2024-09-11T06:52:31.586 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34888.log 2024-09-11T06:52:31.586 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30376.log 2024-09-11T06:52:31.586 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37012.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37012.log.gz 2024-09-11T06:52:31.586 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34888.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34888.log.gz 2024-09-11T06:52:31.586 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32077.log 2024-09-11T06:52:31.586 INFO:teuthology.orchestra.run.smithi071.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-11T06:52:31.586 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.30376.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30376.log.gz 2024-09-11T06:52:31.586 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34508.log 2024-09-11T06:52:31.586 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.32077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34660.log 2024-09-11T06:52:31.586 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32077.log.gz 2024-09-11T06:52:31.586 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35510.log 2024-09-11T06:52:31.586 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34508.log.gz 2024-09-11T06:52:31.586 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34660.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34660.log.gz 2024-09-11T06:52:31.586 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29933.log 2024-09-11T06:52:31.587 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37922.log 2024-09-11T06:52:31.587 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35510.log.gz 2024-09-11T06:52:31.587 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.29933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33421.log 2024-09-11T06:52:31.587 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29933.log.gz 2024-09-11T06:52:31.587 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38208.log 2024-09-11T06:52:31.587 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37922.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37922.log.gz 2024-09-11T06:52:31.587 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37384.log 2024-09-11T06:52:31.587 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33421.log.gz 2024-09-11T06:52:31.587 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38208.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38208.log.gz 2024-09-11T06:52:31.587 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33232.log 2024-09-11T06:52:31.587 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37384.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33493.log 2024-09-11T06:52:31.587 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37384.log.gz 2024-09-11T06:52:31.587 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33780.log 2024-09-11T06:52:31.587 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33232.log.gz 2024-09-11T06:52:31.588 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35282.log 2024-09-11T06:52:31.588 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33493.log.gz 2024-09-11T06:52:31.588 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32127.log 2024-09-11T06:52:31.588 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33780.log: /var/log/ceph/ceph-client.admin.35282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35712.log 2024-09-11T06:52:31.588 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33780.log.gz 2024-09-11T06:52:31.588 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35282.log.gz 2024-09-11T06:52:31.588 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.32127.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32127.log.gz 2024-09-11T06:52:31.588 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30886.log 2024-09-11T06:52:31.588 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31555.log 2024-09-11T06:52:31.588 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35712.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35712.log.gz 2024-09-11T06:52:31.588 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.30886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29757.log 2024-09-11T06:52:31.588 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30886.log.gz 2024-09-11T06:52:31.588 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35595.log 2024-09-11T06:52:31.589 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.31555.log: /var/log/ceph/ceph-client.admin.29757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37688.log 2024-09-11T06:52:31.589 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31555.log.gz 2024-09-11T06:52:31.589 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29757.log.gz 2024-09-11T06:52:31.589 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35595.log: 0.0%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.35595.log.gz --verbose 2024-09-11T06:52:31.589 INFO:teuthology.orchestra.run.smithi071.stderr: -- /var/log/ceph/ceph-client.admin.30428.log 2024-09-11T06:52:31.589 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30528.log 2024-09-11T06:52:31.589 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37688.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37688.log.gz 2024-09-11T06:52:31.589 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.30428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29783.log 2024-09-11T06:52:31.589 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30428.log.gz 2024-09-11T06:52:31.589 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32740.log 2024-09-11T06:52:31.589 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.30528.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30528.log.gz 2024-09-11T06:52:31.589 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.29783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36789.log 2024-09-11T06:52:31.589 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29783.log.gz 2024-09-11T06:52:31.589 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.32740.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32740.log.gz 2024-09-11T06:52:31.589 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29467.log 2024-09-11T06:52:31.589 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36981.log 2024-09-11T06:52:31.590 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36789.log.gz 2024-09-11T06:52:31.590 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.29467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29487.log 2024-09-11T06:52:31.590 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29467.log.gz 2024-09-11T06:52:31.590 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36981.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36981.log.gz 2024-09-11T06:52:31.590 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33307.log 2024-09-11T06:52:31.590 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.29487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33517.log 2024-09-11T06:52:31.590 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29487.log.gz 2024-09-11T06:52:31.590 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29442.log 2024-09-11T06:52:31.590 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33307.log.gz 2024-09-11T06:52:31.590 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32441.log 2024-09-11T06:52:31.590 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33517.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33517.log.gz 2024-09-11T06:52:31.590 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.29442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31937.log 2024-09-11T06:52:31.590 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29442.log.gz 2024-09-11T06:52:31.590 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.32441.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32441.log.gz 2024-09-11T06:52:31.591 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32593.log 2024-09-11T06:52:31.591 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.31937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33754.log 2024-09-11T06:52:31.591 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31937.log.gz 2024-09-11T06:52:31.591 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.32593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37220.log 2024-09-11T06:52:31.591 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32593.log.gz 2024-09-11T06:52:31.591 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37896.log 2024-09-11T06:52:31.591 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33754.log: 0.0%/var/log/ceph/ceph-client.admin.37220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32855.log 2024-09-11T06:52:31.591 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33754.log.gz 0.0% 2024-09-11T06:52:31.591 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37220.log.gz 2024-09-11T06:52:31.591 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37896.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37896.log.gz 2024-09-11T06:52:31.591 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38232.log 2024-09-11T06:52:31.591 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.32855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35661.log 2024-09-11T06:52:31.591 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32855.log.gz 2024-09-11T06:52:31.591 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29584.log 2024-09-11T06:52:31.591 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38232.log.gz 2024-09-11T06:52:31.591 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31501.log 2024-09-11T06:52:31.592 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35661.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35661.log.gz 2024-09-11T06:52:31.592 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.29584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31970.log 2024-09-11T06:52:31.592 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29584.log.gz 2024-09-11T06:52:31.592 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34656.log 2024-09-11T06:52:31.592 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.31501.log: /var/log/ceph/ceph-client.admin.31970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32764.log 2024-09-11T06:52:31.592 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31970.log.gz 2024-09-11T06:52:31.592 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31501.log.gz 2024-09-11T06:52:31.592 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34656.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34656.log.gz 2024-09-11T06:52:31.592 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32829.log 2024-09-11T06:52:31.592 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.32764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29909.log 2024-09-11T06:52:31.592 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32764.log.gz 2024-09-11T06:52:31.592 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.32829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30788.log 0.0% 2024-09-11T06:52:31.592 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32829.log.gz 2024-09-11T06:52:31.592 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33618.log 2024-09-11T06:52:31.593 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.29909.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29909.log.gz 2024-09-11T06:52:31.593 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.30788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36168.log 2024-09-11T06:52:31.593 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30788.log.gz 2024-09-11T06:52:31.593 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33618.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33618.log.gz 2024-09-11T06:52:31.593 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37246.log 2024-09-11T06:52:31.593 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29417.log 2024-09-11T06:52:31.593 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36168.log.gz 2024-09-11T06:52:31.593 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31414.log 2024-09-11T06:52:31.593 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37246.log.gz 2024-09-11T06:52:31.593 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37298.log 2024-09-11T06:52:31.593 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.29417.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29417.log.gz 2024-09-11T06:52:31.593 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.31414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31377.log 2024-09-11T06:52:31.593 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31414.log.gz 2024-09-11T06:52:31.593 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37298.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37298.log.gz 2024-09-11T06:52:31.594 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33269.log 2024-09-11T06:52:31.594 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32123.log 2024-09-11T06:52:31.594 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.31377.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31377.log.gz 2024-09-11T06:52:31.594 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33269.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33269.log.gz 2024-09-11T06:52:31.594 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.32123.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32123.log.gz 2024-09-11T06:52:31.596 INFO:teuthology.orchestra.run.smithi089.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-11T06:52:31.596 INFO:teuthology.orchestra.run.smithi089.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-11T06:52:31.607 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25064.log 2024-09-11T06:52:31.607 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22699.log 2024-09-11T06:52:31.607 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.25064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25144.log 2024-09-11T06:52:31.607 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25064.log.gz 2024-09-11T06:52:31.608 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22699.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22699.log.gz 2024-09-11T06:52:31.608 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24921.log 2024-09-11T06:52:31.608 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.25144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24366.log 2024-09-11T06:52:31.608 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25144.log.gz 2024-09-11T06:52:31.608 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23564.log 2024-09-11T06:52:31.608 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24921.log.gz 2024-09-11T06:52:31.608 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25360.log 2024-09-11T06:52:31.608 INFO:teuthology.orchestra.run.smithi176.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.24366.log.gz 2024-09-11T06:52:31.608 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23823.log 2024-09-11T06:52:31.608 INFO:teuthology.orchestra.run.smithi176.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23564.log.gz 2024-09-11T06:52:31.608 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.25360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24944.log 2024-09-11T06:52:31.608 INFO:teuthology.orchestra.run.smithi176.stderr: 50.3% -- replaced with /var/log/ceph/ceph-client.admin.25360.log.gz 2024-09-11T06:52:31.608 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25155.log 2024-09-11T06:52:31.608 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23823.log.gz 2024-09-11T06:52:31.609 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24944.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24944.log.gz 2024-09-11T06:52:31.609 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22966.log 2024-09-11T06:52:31.609 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22876.log 2024-09-11T06:52:31.609 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.25155.log: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.25155.log.gz 2024-09-11T06:52:31.609 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22559.log 2024-09-11T06:52:31.609 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22966.log.gz 2024-09-11T06:52:31.609 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22876.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22876.log.gz 2024-09-11T06:52:31.609 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24302.log 2024-09-11T06:52:31.609 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22993.log 2024-09-11T06:52:31.609 INFO:teuthology.orchestra.run.smithi176.stderr: 53.6% -- replaced with /var/log/ceph/ceph-client.admin.22559.log.gz 2024-09-11T06:52:31.609 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22830.log 2024-09-11T06:52:31.609 INFO:teuthology.orchestra.run.smithi176.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.24302.log.gz 2024-09-11T06:52:31.609 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22993.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22993.log.gz 2024-09-11T06:52:31.609 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24358.log 2024-09-11T06:52:31.609 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24262.log 2024-09-11T06:52:31.610 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22830.log.gz 2024-09-11T06:52:31.610 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23063.log 2024-09-11T06:52:31.610 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24358.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24358.log.gz 2024-09-11T06:52:31.610 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24262.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24262.log.gz 2024-09-11T06:52:31.610 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24095.log 2024-09-11T06:52:31.610 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24134.log 2024-09-11T06:52:31.610 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23063.log.gz 2024-09-11T06:52:31.610 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24016.log 2024-09-11T06:52:31.610 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24095.log: 45.5% -- replaced with /var/log/ceph/ceph-client.admin.24095.log.gz 2024-09-11T06:52:31.610 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24134.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24134.log.gz 2024-09-11T06:52:31.610 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22466.log 2024-09-11T06:52:31.610 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24016.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.24016.log.gz 2024-09-11T06:52:31.610 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23656.log 2024-09-11T06:52:31.610 INFO:teuthology.orchestra.run.smithi176.stderr:gzip/var/log/ceph/ceph-client.admin.22466.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.23883.log 2024-09-11T06:52:31.610 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22466.log.gz 2024-09-11T06:52:31.611 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23656.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23656.log.gz 2024-09-11T06:52:31.611 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24805.log 2024-09-11T06:52:31.611 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22841.log 2024-09-11T06:52:31.611 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23883.log.gz 2024-09-11T06:52:31.611 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23718.log 2024-09-11T06:52:31.611 INFO:teuthology.orchestra.run.smithi176.stderr: 5.5% -- replaced with /var/log/ceph/ceph-client.admin.24805.log.gz 2024-09-11T06:52:31.611 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22904.log 2024-09-11T06:52:31.611 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22841.log: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.22841.log.gz 2024-09-11T06:52:31.611 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24066.log 2024-09-11T06:52:31.611 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23718.log.gz 2024-09-11T06:52:31.611 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22904.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22904.log.gz 2024-09-11T06:52:31.611 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22733.log 2024-09-11T06:52:31.611 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24410.log 2024-09-11T06:52:31.611 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24066.log.gz 2024-09-11T06:52:31.612 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22532.log 2024-09-11T06:52:31.612 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22733.log.gz 2024-09-11T06:52:31.612 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24410.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24410.log.gz 2024-09-11T06:52:31.612 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24401.log 2024-09-11T06:52:31.612 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22532.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22532.log.gz 2024-09-11T06:52:31.612 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23183.log 2024-09-11T06:52:31.612 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25010.log/var/log/ceph/ceph-client.admin.24401.log: 2024-09-11T06:52:31.612 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24401.log.gz 2024-09-11T06:52:31.612 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22253.log 2024-09-11T06:52:31.612 INFO:teuthology.orchestra.run.smithi176.stderr: 35.0% -- replaced with /var/log/ceph/ceph-client.admin.23183.log.gz 2024-09-11T06:52:31.612 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.25010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23147.log 2024-09-11T06:52:31.612 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25010.log.gz 2024-09-11T06:52:31.612 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5/var/log/ceph/ceph-client.admin.22253.log: --verbose -- /var/log/ceph/ceph-client.admin.23392.log 2024-09-11T06:52:31.613 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22253.log.gz 2024-09-11T06:52:31.613 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23147.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23147.log.gz 2024-09-11T06:52:31.613 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22550.log 2024-09-11T06:52:31.613 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24078.log 2024-09-11T06:52:31.613 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23392.log.gz 2024-09-11T06:52:31.613 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22550.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22550.log.gz 2024-09-11T06:52:31.613 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21947.log 2024-09-11T06:52:31.613 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25257.log 2024-09-11T06:52:31.613 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24078.log.gz 2024-09-11T06:52:31.613 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.21947.log: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.21947.log.gz 2024-09-11T06:52:31.613 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25131.log 2024-09-11T06:52:31.613 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.25257.log: 23.8% -- replaced with /var/log/ceph/ceph-client.admin.25257.log.gz 2024-09-11T06:52:31.613 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21891.log 2024-09-11T06:52:31.614 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24179.log 2024-09-11T06:52:31.614 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.25131.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25131.log.gz 2024-09-11T06:52:31.614 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.21891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23172.log 2024-09-11T06:52:31.614 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21891.log.gz 2024-09-11T06:52:31.614 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22507.log 2024-09-11T06:52:31.614 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24179.log.gz 2024-09-11T06:52:31.614 INFO:teuthology.orchestra.run.smithi176.stderr:gzip/var/log/ceph/ceph-client.admin.23172.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.24275.log 2024-09-11T06:52:31.614 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23172.log.gz 2024-09-11T06:52:31.614 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22507.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22507.log.gz -5 2024-09-11T06:52:31.614 INFO:teuthology.orchestra.run.smithi176.stderr: --verbose -- /var/log/ceph/ceph-client.admin.22735.log 2024-09-11T06:52:31.614 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24275.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24275.log.gz 2024-09-11T06:52:31.614 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23999.log 2024-09-11T06:52:31.614 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22147.log 2024-09-11T06:52:31.615 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22735.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22735.log.gz 2024-09-11T06:52:31.615 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23850.log 2024-09-11T06:52:31.615 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23999.log.gz 2024-09-11T06:52:31.615 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22147.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22147.log.gz 2024-09-11T06:52:31.615 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25128.log 2024-09-11T06:52:31.615 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24570.log 2024-09-11T06:52:31.615 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23850.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23850.log.gz 2024-09-11T06:52:31.615 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.25128.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.23283.log 2024-09-11T06:52:31.615 INFO:teuthology.orchestra.run.smithi176.stderr: -- replaced with /var/log/ceph/ceph-client.admin.25128.log.gz 2024-09-11T06:52:31.615 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23037.log 2024-09-11T06:52:31.615 INFO:teuthology.orchestra.run.smithi176.stderr: 5.5% -- replaced with /var/log/ceph/ceph-client.admin.24570.log.gz 2024-09-11T06:52:31.615 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23221.log 2024-09-11T06:52:31.615 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23283.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23283.log.gz 2024-09-11T06:52:31.615 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23218.log 2024-09-11T06:52:31.616 INFO:teuthology.orchestra.run.smithi176.stderr: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.23037.log.gz 2024-09-11T06:52:31.616 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23221.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.23221.log.gz /var/log/ceph/ceph-client.admin.21956.log 2024-09-11T06:52:31.616 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-09-11T06:52:31.616 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21748.log 2024-09-11T06:52:31.616 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23218.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23218.log.gz 2024-09-11T06:52:31.616 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.21956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22657.log 2024-09-11T06:52:31.616 INFO:teuthology.orchestra.run.smithi176.stderr: 26.3% -- replaced with /var/log/ceph/ceph-client.admin.21956.log.gz 2024-09-11T06:52:31.616 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.21748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21894.log 2024-09-11T06:52:31.616 INFO:teuthology.orchestra.run.smithi176.stderr: 26.3% -- replaced with /var/log/ceph/ceph-client.admin.21748.log.gz 2024-09-11T06:52:31.616 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23447.log 2024-09-11T06:52:31.616 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22657.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22657.log.gz 2024-09-11T06:52:31.616 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.21894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25215.log 2024-09-11T06:52:31.616 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21894.log.gz 2024-09-11T06:52:31.617 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22631.log 2024-09-11T06:52:31.617 INFO:teuthology.orchestra.run.smithi176.stderr: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.23447.log.gz 2024-09-11T06:52:31.617 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.25215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25381.log 2024-09-11T06:52:31.617 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25215.log.gz 2024-09-11T06:52:31.617 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23756.log 2024-09-11T06:52:31.617 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22631.log: 55.0% -- replaced with /var/log/ceph/ceph-client.admin.22631.log.gz 2024-09-11T06:52:31.617 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.25381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22696.log 2024-09-11T06:52:31.617 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25381.log.gz 2024-09-11T06:52:31.617 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23626.log 2024-09-11T06:52:31.617 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23756.log.gz 2024-09-11T06:52:31.617 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22620.log 2024-09-11T06:52:31.617 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22696.log.gz 2024-09-11T06:52:31.617 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23002.log 2024-09-11T06:52:31.618 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23626.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23626.log.gz 2024-09-11T06:52:31.618 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25005.log 2024-09-11T06:52:31.618 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22620.log.gz 2024-09-11T06:52:31.618 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23002.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.23002.log.gz 2024-09-11T06:52:31.618 INFO:teuthology.orchestra.run.smithi176.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.24300.log 2024-09-11T06:52:31.618 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.25005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23380.log 2024-09-11T06:52:31.618 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25005.log.gz 2024-09-11T06:52:31.618 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25277.log 2024-09-11T06:52:31.618 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24300.log.gz 2024-09-11T06:52:31.618 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23380.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23380.log.gz 2024-09-11T06:52:31.618 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23236.log 2024-09-11T06:52:31.618 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24463.log 2024-09-11T06:52:31.618 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.25277.log: 52.5% -- replaced with /var/log/ceph/ceph-client.admin.25277.log.gz 2024-09-11T06:52:31.618 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22797.log 2024-09-11T06:52:31.619 INFO:teuthology.orchestra.run.smithi176.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23236.log.gz 2024-09-11T06:52:31.619 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24463.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24463.log.gz -5 2024-09-11T06:52:31.619 INFO:teuthology.orchestra.run.smithi176.stderr: --verbose -- /var/log/ceph/ceph-client.admin.22931.log 2024-09-11T06:52:31.619 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24346.log 2024-09-11T06:52:31.619 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22797.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22797.log.gz 2024-09-11T06:52:31.619 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21828.log 2024-09-11T06:52:31.619 INFO:teuthology.orchestra.run.smithi176.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22931.log.gz 2024-09-11T06:52:31.619 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24346.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24346.log.gz 2024-09-11T06:52:31.619 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23667.log 2024-09-11T06:52:31.619 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23635.log 2024-09-11T06:52:31.619 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.21828.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.21828.log.gz 2024-09-11T06:52:31.619 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22266.log 2024-09-11T06:52:31.619 INFO:teuthology.orchestra.run.smithi176.stderr: 46.2% -- replaced with /var/log/ceph/ceph-client.admin.23667.log.gz 2024-09-11T06:52:31.619 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24153.log 2024-09-11T06:52:31.619 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23635.log.gz 2024-09-11T06:52:31.620 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23709.log 2024-09-11T06:52:31.620 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22266.log.gz 2024-09-11T06:52:31.620 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22309.log 2024-09-11T06:52:31.620 INFO:teuthology.orchestra.run.smithi176.stderr: 24.1% -- replaced with /var/log/ceph/ceph-client.admin.24153.log.gz 2024-09-11T06:52:31.620 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23709.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.23709.log.gz /var/log/ceph/ceph-client.admin.23438.log 2024-09-11T06:52:31.620 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-09-11T06:52:31.620 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22770.log 2024-09-11T06:52:31.620 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22309.log.gz 2024-09-11T06:52:31.620 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23209.log 2024-09-11T06:52:31.620 INFO:teuthology.orchestra.run.smithi176.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23438.log.gz 2024-09-11T06:52:31.620 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22770.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22770.log.gz 2024-09-11T06:52:31.620 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23342.log 2024-09-11T06:52:31.620 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23573.log 2024-09-11T06:52:31.621 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23209.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23209.log.gz 2024-09-11T06:52:31.621 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22779.log 2024-09-11T06:52:31.621 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23342.log.gz 2024-09-11T06:52:31.621 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22033.log 2024-09-11T06:52:31.621 INFO:teuthology.orchestra.run.smithi176.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.23573.log.gz 2024-09-11T06:52:31.621 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25116.log 2024-09-11T06:52:31.621 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22779.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22779.log.gz 2024-09-11T06:52:31.621 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22033.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22033.log.gz 2024-09-11T06:52:31.621 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23473.log 2024-09-11T06:52:31.621 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.25116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25037.log 2024-09-11T06:52:31.621 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25116.log.gz 2024-09-11T06:52:31.621 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21938.log 2024-09-11T06:52:31.621 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23473.log.gz 2024-09-11T06:52:31.621 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.25037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24909.log 2024-09-11T06:52:31.622 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25037.log.gz 2024-09-11T06:52:31.622 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.21938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24979.log 2024-09-11T06:52:31.622 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21938.log.gz 2024-09-11T06:52:31.622 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25061.log 2024-09-11T06:52:31.622 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24909.log.gz 2024-09-11T06:52:31.622 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25398.log 2024-09-11T06:52:31.622 INFO:teuthology.orchestra.run.smithi176.stderr: 53.3% -- replaced with /var/log/ceph/ceph-client.admin.24979.log.gz 2024-09-11T06:52:31.622 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.25061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22975.log 2024-09-11T06:52:31.622 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25061.log.gz 2024-09-11T06:52:31.622 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.25398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24970.log 2024-09-11T06:52:31.622 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25398.log.gz 2024-09-11T06:52:31.622 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23333.log 2024-09-11T06:52:31.622 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22975.log.gz 2024-09-11T06:52:31.622 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21901.log 2024-09-11T06:52:31.622 INFO:teuthology.orchestra.run.smithi176.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.24970.log.gz 2024-09-11T06:52:31.622 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22359.log 2024-09-11T06:52:31.623 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23333.log.gz 2024-09-11T06:52:31.623 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.21901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22121.log 2024-09-11T06:52:31.623 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21901.log.gz 2024-09-11T06:52:31.623 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24025.log 2024-09-11T06:52:31.623 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22359.log.gz 2024-09-11T06:52:31.623 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21999.log 2024-09-11T06:52:31.623 INFO:teuthology.orchestra.run.smithi176.stderr: 53.6% -- replaced with /var/log/ceph/ceph-client.admin.22121.log.gz 2024-09-11T06:52:31.623 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24216.log 2024-09-11T06:52:31.623 INFO:teuthology.orchestra.run.smithi176.stderr: 43.6% -- replaced with /var/log/ceph/ceph-client.admin.24025.log.gz 2024-09-11T06:52:31.623 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.21999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24438.log 2024-09-11T06:52:31.623 INFO:teuthology.orchestra.run.smithi176.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.21999.log.gz 2024-09-11T06:52:31.623 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21702.log 2024-09-11T06:52:31.623 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24216.log.gz 2024-09-11T06:52:31.623 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23389.log 0.0% 2024-09-11T06:52:31.623 INFO:teuthology.orchestra.run.smithi176.stderr: -- replaced with /var/log/ceph/ceph-client.admin.24438.log.gz 2024-09-11T06:52:31.623 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.21702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23298.log 2024-09-11T06:52:31.624 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21702.log.gz 2024-09-11T06:52:31.624 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24604.log 2024-09-11T06:52:31.624 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23389.log.gz 2024-09-11T06:52:31.624 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22867.log 2024-09-11T06:52:31.624 INFO:teuthology.orchestra.run.smithi176.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.23298.log.gz 2024-09-11T06:52:31.624 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23729.log 2024-09-11T06:52:31.624 INFO:teuthology.orchestra.run.smithi176.stderr: 45.5% -- replaced with /var/log/ceph/ceph-client.admin.24604.log.gz 2024-09-11T06:52:31.624 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25146.log 2024-09-11T06:52:31.624 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22867.log.gz 2024-09-11T06:52:31.624 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24227.log 2024-09-11T06:52:31.624 INFO:teuthology.orchestra.run.smithi176.stderr: 59.7% -- replaced with /var/log/ceph/ceph-client.admin.23729.log.gz 2024-09-11T06:52:31.624 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.25146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22422.log 2024-09-11T06:52:31.624 INFO:teuthology.orchestra.run.smithi176.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.25146.log.gz 2024-09-11T06:52:31.625 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22940.log 2024-09-11T06:52:31.625 INFO:teuthology.orchestra.run.smithi176.stderr: 24.1% -- replaced with /var/log/ceph/ceph-client.admin.24227.log.gz 2024-09-11T06:52:31.625 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22422.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.22422.log.gz --verbose 2024-09-11T06:52:31.625 INFO:teuthology.orchestra.run.smithi176.stderr: -- /var/log/ceph/ceph-client.admin.23886.log 2024-09-11T06:52:31.625 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22409.log 2024-09-11T06:52:31.625 INFO:teuthology.orchestra.run.smithi176.stderr: 23.5% -- replaced with /var/log/ceph/ceph-client.admin.22940.log.gz 2024-09-11T06:52:31.625 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22894.log 2024-09-11T06:52:31.625 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23886.log.gz 2024-09-11T06:52:31.625 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22409.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.22409.log.gz 2024-09-11T06:52:31.625 INFO:teuthology.orchestra.run.smithi176.stderr: /var/log/ceph/ceph-client.admin.22744.log 2024-09-11T06:52:31.625 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24671.log 2024-09-11T06:52:31.625 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22894.log.gz 2024-09-11T06:52:31.625 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23028.log 2024-09-11T06:52:31.625 INFO:teuthology.orchestra.run.smithi176.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.22744.log.gz 2024-09-11T06:52:31.625 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22110.log 2024-09-11T06:52:31.626 INFO:teuthology.orchestra.run.smithi176.stderr: 4.5% -- replaced with /var/log/ceph/ceph-client.admin.24671.log.gz 2024-09-11T06:52:31.626 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23159.log 2024-09-11T06:52:31.626 INFO:teuthology.orchestra.run.smithi176.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23028.log.gz 2024-09-11T06:52:31.626 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23307.log 2024-09-11T06:52:31.626 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22110.log.gz 2024-09-11T06:52:31.626 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23159.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23159.log.gz 2024-09-11T06:52:31.626 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25319.log 2024-09-11T06:52:31.626 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24265.log 2024-09-11T06:52:31.626 INFO:teuthology.orchestra.run.smithi176.stderr: 35.0% -- replaced with /var/log/ceph/ceph-client.admin.23307.log.gz 2024-09-11T06:52:31.626 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.25319.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25319.log.gz 2024-09-11T06:52:31.626 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22535.log 2024-09-11T06:52:31.626 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24375.log 2024-09-11T06:52:31.626 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24265.log.gz 2024-09-11T06:52:31.626 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23658.log 2024-09-11T06:52:31.627 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22535.log.gz 2024-09-11T06:52:31.627 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24637.log 2024-09-11T06:52:31.627 INFO:teuthology.orchestra.run.smithi176.stderr: 45.5% -- replaced with /var/log/ceph/ceph-client.admin.24375.log.gz 2024-09-11T06:52:31.627 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21986.log 2024-09-11T06:52:31.627 INFO:teuthology.orchestra.run.smithi176.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23658.log.gz 2024-09-11T06:52:31.627 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24465.log 2024-09-11T06:52:31.627 INFO:teuthology.orchestra.run.smithi176.stderr: 5.5% -- replaced with /var/log/ceph/ceph-client.admin.24637.log.gz 2024-09-11T06:52:31.627 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.21986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22832.log 2024-09-11T06:52:31.627 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21986.log.gz 2024-09-11T06:52:31.627 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22671.log 2024-09-11T06:52:31.627 INFO:teuthology.orchestra.run.smithi176.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.24465.log.gz 2024-09-11T06:52:31.627 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23156.log 2024-09-11T06:52:31.627 INFO:teuthology.orchestra.run.smithi176.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22832.log.gz 2024-09-11T06:52:31.627 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24771.log 2024-09-11T06:52:31.628 INFO:teuthology.orchestra.run.smithi176.stderr: 40.9% -- replaced with /var/log/ceph/ceph-client.admin.22671.log.gz 2024-09-11T06:52:31.628 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23156.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.23156.log.gz /var/log/ceph/ceph-client.admin.23075.log 2024-09-11T06:52:31.628 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-09-11T06:52:31.628 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24142.log 2024-09-11T06:52:31.628 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24771.log: 4.5% -- replaced with /var/log/ceph/ceph-client.admin.24771.log.gz 2024-09-11T06:52:31.628 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21795.log 2024-09-11T06:52:31.628 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23075.log.gz 2024-09-11T06:52:31.628 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25188.log 2024-09-11T06:52:31.628 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24142.log.gz 2024-09-11T06:52:31.628 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.21795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23271.log 2024-09-11T06:52:31.628 INFO:teuthology.orchestra.run.smithi176.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.21795.log.gz 2024-09-11T06:52:31.628 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.25188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24218.log 2024-09-11T06:52:31.628 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25188.log.gz 2024-09-11T06:52:31.628 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23271.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.23271.log.gz --verbose 2024-09-11T06:52:31.628 INFO:teuthology.orchestra.run.smithi176.stderr: -- /var/log/ceph/ceph-client.admin.24121.log 2024-09-11T06:52:31.629 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21804.log 2024-09-11T06:52:31.629 INFO:teuthology.orchestra.run.smithi176.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.24218.log.gz 2024-09-11T06:52:31.629 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25090.log 2024-09-11T06:52:31.629 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24121.log.gz 2024-09-11T06:52:31.629 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.21804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23112.log 2024-09-11T06:52:31.629 INFO:teuthology.orchestra.run.smithi176.stderr: 3.6% -- replaced with /var/log/ceph/ceph-client.admin.21804.log.gz 2024-09-11T06:52:31.629 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.25090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22268.log 2024-09-11T06:52:31.629 INFO:teuthology.orchestra.run.smithi176.stderr: 23.5% -- replaced with /var/log/ceph/ceph-client.admin.25090.log.gz 2024-09-11T06:52:31.629 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24912.log 2024-09-11T06:52:31.629 INFO:teuthology.orchestra.run.smithi176.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.23112.log.gz 2024-09-11T06:52:31.629 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23509.log 2024-09-11T06:52:31.629 INFO:teuthology.orchestra.run.smithi176.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22268.log.gz 2024-09-11T06:52:31.629 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23526.log 2024-09-11T06:52:31.629 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24912.log.gz 2024-09-11T06:52:31.630 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23509.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23509.log.gz 2024-09-11T06:52:31.630 INFO:teuthology.orchestra.run.smithi176.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.24537.log 2024-09-11T06:52:31.630 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21722.log 2024-09-11T06:52:31.630 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23526.log.gz 2024-09-11T06:52:31.630 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22069.log 2024-09-11T06:52:31.630 INFO:teuthology.orchestra.run.smithi176.stderr: 6.4% -- replaced with /var/log/ceph/ceph-client.admin.24537.log.gz 2024-09-11T06:52:31.630 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.21722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24144.log 2024-09-11T06:52:31.630 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21722.log.gz 2024-09-11T06:52:31.630 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25081.log 2024-09-11T06:52:31.630 INFO:teuthology.orchestra.run.smithi176.stderr: 53.8% -- replaced with /var/log/ceph/ceph-client.admin.22069.log.gz 2024-09-11T06:52:31.630 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22585.log 2024-09-11T06:52:31.630 INFO:teuthology.orchestra.run.smithi176.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.24144.log.gz 2024-09-11T06:52:31.630 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.25081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24130.log 2024-09-11T06:52:31.630 INFO:teuthology.orchestra.run.smithi176.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.25081.log.gz 2024-09-11T06:52:31.631 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22585.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.22585.log.gz 2024-09-11T06:52:31.631 INFO:teuthology.orchestra.run.smithi176.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.25298.log 2024-09-11T06:52:31.631 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25415.log 2024-09-11T06:52:31.631 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24130.log.gz 2024-09-11T06:52:31.631 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.25298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24364.log 2024-09-11T06:52:31.631 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25298.log.gz 2024-09-11T06:52:31.631 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.25415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23761.log 2024-09-11T06:52:31.631 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25415.log.gz 2024-09-11T06:52:31.631 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23500.log 2024-09-11T06:52:31.631 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24364.log.gz 2024-09-11T06:52:31.631 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21778.log 2024-09-11T06:52:31.631 INFO:teuthology.orchestra.run.smithi176.stderr: 45.1% -- replaced with /var/log/ceph/ceph-client.admin.23761.log.gz 2024-09-11T06:52:31.631 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23500.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.23500.log.gz --verbose 2024-09-11T06:52:31.632 INFO:teuthology.orchestra.run.smithi176.stderr: -- /var/log/ceph/ceph-client.admin.23353.log 2024-09-11T06:52:31.632 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.21778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25340.log 2024-09-11T06:52:31.632 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21778.log.gz 2024-09-11T06:52:31.632 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23026.log 2024-09-11T06:52:31.632 INFO:teuthology.orchestra.run.smithi176.stderr: 24.1% -- replaced with /var/log/ceph/ceph-client.admin.23353.log.gz 2024-09-11T06:52:31.632 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.25340.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25340.log.gz 2024-09-11T06:52:31.632 INFO:teuthology.orchestra.run.smithi176.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.22607.log 2024-09-11T06:52:31.632 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25079.log 2024-09-11T06:52:31.632 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23026.log.gz 2024-09-11T06:52:31.632 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23174.log 2024-09-11T06:52:31.632 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22607.log.gz 2024-09-11T06:52:31.632 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.25079.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.25079.log.gz 2024-09-11T06:52:31.632 INFO:teuthology.orchestra.run.smithi176.stderr: --verbose -- /var/log/ceph/ceph-client.admin.23421.log 2024-09-11T06:52:31.632 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22153.log 2024-09-11T06:52:31.632 INFO:teuthology.orchestra.run.smithi176.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23174.log.gz 2024-09-11T06:52:31.633 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24414.log 2024-09-11T06:52:31.633 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23421.log.gz 2024-09-11T06:52:31.633 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22153.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.22153.log.gz 2024-09-11T06:52:31.633 INFO:teuthology.orchestra.run.smithi176.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.23121.log 2024-09-11T06:52:31.633 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22277.log 2024-09-11T06:52:31.633 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24414.log.gz 2024-09-11T06:52:31.633 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21621.log 2024-09-11T06:52:31.633 INFO:teuthology.orchestra.run.smithi176.stderr: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.23121.log.gz 2024-09-11T06:52:31.633 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23344.log 2024-09-11T06:52:31.633 INFO:teuthology.orchestra.run.smithi176.stderr: 52.5% -- replaced with /var/log/ceph/ceph-client.admin.22277.log.gz 2024-09-11T06:52:31.633 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.21621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22806.log 2024-09-11T06:52:31.633 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21621.log.gz 2024-09-11T06:52:31.633 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23954.log 2024-09-11T06:52:31.633 INFO:teuthology.orchestra.run.smithi176.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23344.log.gz 2024-09-11T06:52:31.634 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22806.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.21889.log -- replaced with /var/log/ceph/ceph-client.admin.22806.log.gz 2024-09-11T06:52:31.634 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-09-11T06:52:31.634 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25200.log 2024-09-11T06:52:31.634 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23954.log.gz 2024-09-11T06:52:31.634 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.21889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23234.log 2024-09-11T06:52:31.634 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21889.log.gz 2024-09-11T06:52:31.634 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.25200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24014.log 2024-09-11T06:52:31.634 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25200.log.gz 2024-09-11T06:52:31.634 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24070.log 2024-09-11T06:52:31.634 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23234.log.gz 2024-09-11T06:52:31.634 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23405.log 2024-09-11T06:52:31.634 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24014.log.gz 2024-09-11T06:52:31.634 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24070.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24070.log.gz 2024-09-11T06:52:31.634 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21983.log 2024-09-11T06:52:31.634 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24253.log 2024-09-11T06:52:31.635 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23405.log.gz 2024-09-11T06:52:31.635 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.21983.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.21983.log.gz --verbose 2024-09-11T06:52:31.635 INFO:teuthology.orchestra.run.smithi176.stderr: -- /var/log/ceph/ceph-client.admin.22604.log 2024-09-11T06:52:31.635 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22459.log 2024-09-11T06:52:31.635 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24253.log.gz 2024-09-11T06:52:31.635 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23280.log 2024-09-11T06:52:31.635 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22604.log.gz 2024-09-11T06:52:31.635 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22459.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22459.log.gz 2024-09-11T06:52:31.635 INFO:teuthology.orchestra.run.smithi176.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.23408.log 2024-09-11T06:52:31.635 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22303.log 2024-09-11T06:52:31.635 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23280.log.gz 2024-09-11T06:52:31.635 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22622.log 2024-09-11T06:52:31.635 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23408.log.gz 2024-09-11T06:52:31.635 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22203.log 2024-09-11T06:52:31.635 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22303.log.gz 2024-09-11T06:52:31.635 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21837.log 2024-09-11T06:52:31.636 INFO:teuthology.orchestra.run.smithi176.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22622.log.gz 2024-09-11T06:52:31.636 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23296.log 2024-09-11T06:52:31.636 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22203.log.gz 2024-09-11T06:52:31.636 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.21837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23072.log 2024-09-11T06:52:31.636 INFO:teuthology.orchestra.run.smithi176.stderr: 26.3% -- replaced with /var/log/ceph/ceph-client.admin.21837.log.gz 2024-09-11T06:52:31.636 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23296.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.22708.log -- replaced with /var/log/ceph/ceph-client.admin.23296.log.gz 2024-09-11T06:52:31.636 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-09-11T06:52:31.636 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23547.log 2024-09-11T06:52:31.636 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23072.log.gz 2024-09-11T06:52:31.636 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24872.log 2024-09-11T06:52:31.636 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22708.log.gz 2024-09-11T06:52:31.636 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24738.log 0.0% 2024-09-11T06:52:31.636 INFO:teuthology.orchestra.run.smithi176.stderr: -- replaced with /var/log/ceph/ceph-client.admin.23547.log.gz 2024-09-11T06:52:31.636 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24501.log 2024-09-11T06:52:31.637 INFO:teuthology.orchestra.run.smithi176.stderr: 5.5% -- replaced with /var/log/ceph/ceph-client.admin.24872.log.gz 2024-09-11T06:52:31.637 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23245.log 2024-09-11T06:52:31.637 INFO:teuthology.orchestra.run.smithi176.stderr: 3.6% -- replaced with /var/log/ceph/ceph-client.admin.24738.log.gz 2024-09-11T06:52:31.637 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23693.log 2024-09-11T06:52:31.637 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24501.log.gz 2024-09-11T06:52:31.637 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24337.log 2024-09-11T06:52:31.637 INFO:teuthology.orchestra.run.smithi176.stderr: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.23245.log.gz 2024-09-11T06:52:31.637 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23704.log 2024-09-11T06:52:31.637 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23693.log.gz 2024-09-11T06:52:31.637 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21912.log 2024-09-11T06:52:31.637 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24337.log.gz 2024-09-11T06:52:31.637 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22225.log 2024-09-11T06:52:31.637 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23704.log.gz 2024-09-11T06:52:31.637 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.21912.log: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.21912.log.gz 2024-09-11T06:52:31.638 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24203.log 2024-09-11T06:52:31.638 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22225.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22225.log.gz 2024-09-11T06:52:31.638 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23599.log 2024-09-11T06:52:31.638 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23937.log 2024-09-11T06:52:31.638 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24203.log.gz 2024-09-11T06:52:31.638 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25236.log 2024-09-11T06:52:31.638 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23599.log.gz 2024-09-11T06:52:31.638 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23937.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23937.log.gz -- 2024-09-11T06:52:31.638 INFO:teuthology.orchestra.run.smithi176.stderr: /var/log/ceph/ceph-client.admin.21739.log 2024-09-11T06:52:31.638 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.25236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24418.log 2024-09-11T06:52:31.638 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25236.log.gz 2024-09-11T06:52:31.638 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.21739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23790.log 2024-09-11T06:52:31.638 INFO:teuthology.orchestra.run.smithi176.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.21739.log.gz 2024-09-11T06:52:31.638 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21638.log 2024-09-11T06:52:31.639 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24418.log.gz 2024-09-11T06:52:31.639 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24086.log 2024-09-11T06:52:31.639 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23790.log.gz 2024-09-11T06:52:31.639 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.21638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21658.log 2024-09-11T06:52:31.639 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21638.log.gz 2024-09-11T06:52:31.639 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24086.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.24086.log.gz 2024-09-11T06:52:31.639 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22424.log 2024-09-11T06:52:31.639 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23482.log 2024-09-11T06:52:31.639 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.21658.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21658.log.gz 2024-09-11T06:52:31.639 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22112.log 2024-09-11T06:52:31.639 INFO:teuthology.orchestra.run.smithi176.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.22424.log.gz 2024-09-11T06:52:31.639 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23482.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23482.log.gz 2024-09-11T06:52:31.639 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22038.log 2024-09-11T06:52:31.639 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24051.log 2024-09-11T06:52:31.639 INFO:teuthology.orchestra.run.smithi176.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.22112.log.gz 2024-09-11T06:52:31.640 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22250.log 2024-09-11T06:52:31.640 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22038.log.gz 2024-09-11T06:52:31.640 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23720.log 0.0% 2024-09-11T06:52:31.640 INFO:teuthology.orchestra.run.smithi176.stderr: -- replaced with /var/log/ceph/ceph-client.admin.24051.log.gz 2024-09-11T06:52:31.640 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22929.log 2024-09-11T06:52:31.640 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22250.log.gz 2024-09-11T06:52:31.640 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22175.log 2024-09-11T06:52:31.640 INFO:teuthology.orchestra.run.smithi176.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.23720.log.gz 2024-09-11T06:52:31.640 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21903.log 2024-09-11T06:52:31.640 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22929.log.gz 2024-09-11T06:52:31.640 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24474.log 2024-09-11T06:52:31.640 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22175.log.gz 2024-09-11T06:52:31.640 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.21903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24084.log 2024-09-11T06:52:31.640 INFO:teuthology.orchestra.run.smithi176.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.21903.log.gz 2024-09-11T06:52:31.641 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22008.log 2024-09-11T06:52:31.641 INFO:teuthology.orchestra.run.smithi176.stderr: 52.6% -- replaced with /var/log/ceph/ceph-client.admin.24474.log.gz 2024-09-11T06:52:31.641 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23110.log 2024-09-11T06:52:31.641 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24084.log.gz 2024-09-11T06:52:31.641 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23996.log 2024-09-11T06:52:31.641 INFO:teuthology.orchestra.run.smithi176.stderr: 55.9% -- replaced with /var/log/ceph/ceph-client.admin.22008.log.gz 2024-09-11T06:52:31.641 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23110.log: gzip -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23110.log.gz /var/log/ceph/ceph-client.admin.23608.log 2024-09-11T06:52:31.641 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-09-11T06:52:31.641 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24200.log 2024-09-11T06:52:31.641 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23996.log.gz 2024-09-11T06:52:31.641 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23085.log 2024-09-11T06:52:31.641 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23608.log.gz 2024-09-11T06:52:31.641 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24200.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.24200.log.gz 2024-09-11T06:52:31.642 INFO:teuthology.orchestra.run.smithi176.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.24838.log 2024-09-11T06:52:31.642 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.23085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21678.log 2024-09-11T06:52:31.642 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23085.log.gz 2024-09-11T06:52:31.642 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22097.log 2024-09-11T06:52:31.642 INFO:teuthology.orchestra.run.smithi176.stderr: 6.4% -- replaced with /var/log/ceph/ceph-client.admin.24838.log.gz 2024-09-11T06:52:31.642 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.21678.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.22548.log -- replaced with /var/log/ceph/ceph-client.admin.21678.log.gz 2024-09-11T06:52:31.642 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-09-11T06:52:31.642 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22331.log 2024-09-11T06:52:31.642 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22097.log.gz 2024-09-11T06:52:31.642 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22486.log 2024-09-11T06:52:31.642 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22548.log.gz 2024-09-11T06:52:31.642 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24350.log 2024-09-11T06:52:31.642 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22331.log.gz 2024-09-11T06:52:31.643 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22486.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22486.log.gz 2024-09-11T06:52:31.643 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22094.log 2024-09-11T06:52:31.643 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24704.log 2024-09-11T06:52:31.643 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24350.log.gz 2024-09-11T06:52:31.643 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22094.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.24311.log -- replaced with /var/log/ceph/ceph-client.admin.22094.log.gz 2024-09-11T06:52:31.643 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-09-11T06:52:31.643 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22381.log 2024-09-11T06:52:31.643 INFO:teuthology.orchestra.run.smithi176.stderr: 3.6% -- replaced with /var/log/ceph/ceph-client.admin.24704.log.gz 2024-09-11T06:52:31.643 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22433.log 2024-09-11T06:52:31.643 INFO:teuthology.orchestra.run.smithi176.stderr: 44.8% -- replaced with /var/log/ceph/ceph-client.admin.24311.log.gz 2024-09-11T06:52:31.643 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24968.log 0.0% 2024-09-11T06:52:31.643 INFO:teuthology.orchestra.run.smithi176.stderr: -- replaced with /var/log/ceph/ceph-client.admin.22381.log.gz 2024-09-11T06:52:31.643 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25203.log 2024-09-11T06:52:31.644 INFO:teuthology.orchestra.run.smithi176.stderr: 53.4% -- replaced with /var/log/ceph/ceph-client.admin.22433.log.gz 2024-09-11T06:52:31.644 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.24968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22406.log 2024-09-11T06:52:31.644 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24968.log.gz 2024-09-11T06:52:31.644 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.25203.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25203.log.gz 2024-09-11T06:52:31.644 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.22406.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22406.log.gz 2024-09-11T06:52:31.644 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-09-11T06:52:31.644 INFO:teuthology.orchestra.run.smithi176.stderr:real 0m0.113s 2024-09-11T06:52:31.644 INFO:teuthology.orchestra.run.smithi176.stderr:user 0m0.170s 2024-09-11T06:52:31.644 INFO:teuthology.orchestra.run.smithi176.stderr:sys 0m0.078s 2024-09-11T06:52:31.645 INFO:teuthology.orchestra.run.smithi089.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-11T06:52:31.645 INFO:teuthology.orchestra.run.smithi071.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-11T06:52:31.648 INFO:teuthology.orchestra.run.smithi071.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-11T06:52:31.687 INFO:teuthology.orchestra.run.smithi071.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-11T06:52:31.746 INFO:teuthology.orchestra.run.smithi089.stderr: 94.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-11T06:52:31.748 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-09-11T06:52:31.748 INFO:teuthology.orchestra.run.smithi089.stderr:real 0m0.347s 2024-09-11T06:52:31.748 INFO:teuthology.orchestra.run.smithi089.stderr:user 0m0.887s 2024-09-11T06:52:31.748 INFO:teuthology.orchestra.run.smithi089.stderr:sys 0m0.057s 2024-09-11T06:52:31.777 INFO:teuthology.orchestra.run.smithi071.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-11T06:52:31.803 INFO:teuthology.orchestra.run.smithi071.stderr: 94.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-11T06:52:31.804 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-09-11T06:52:31.804 INFO:teuthology.orchestra.run.smithi071.stderr:real 0m0.432s 2024-09-11T06:52:31.804 INFO:teuthology.orchestra.run.smithi071.stderr:user 0m1.458s 2024-09-11T06:52:31.804 INFO:teuthology.orchestra.run.smithi071.stderr:sys 0m0.151s 2024-09-11T06:52:31.805 INFO:tasks.ceph:Archiving logs... 2024-09-11T06:52:31.805 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900631/remote/smithi071/log 2024-09-11T06:52:31.806 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T06:52:32.202 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900631/remote/smithi089/log 2024-09-11T06:52:32.203 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T06:52:32.317 DEBUG:teuthology.misc:Transferring archived files from smithi176:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900631/remote/smithi176/log 2024-09-11T06:52:32.318 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T06:52:32.553 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-11T06:52:32.563 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-11T06:52:32.563 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T06:52:32.566 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T06:52:32.568 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T06:52:32.652 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-11T06:52:32.661 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-11T06:52:32.661 DEBUG:teuthology.orchestra.run.smithi071:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T06:52:32.663 DEBUG:teuthology.orchestra.run.smithi089:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T06:52:32.666 DEBUG:teuthology.orchestra.run.smithi176:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T06:52:32.674 INFO:teuthology.orchestra.run.smithi071.stdout: remote refid st t when poll reach delay offset jitter 2024-09-11T06:52:32.674 INFO:teuthology.orchestra.run.smithi071.stdout:============================================================================== 2024-09-11T06:52:32.674 INFO:teuthology.orchestra.run.smithi071.stdout:+hv01.front.sepi 67.205.162.81 3 u 39 64 77 0.084 -4.208 1.463 2024-09-11T06:52:32.674 INFO:teuthology.orchestra.run.smithi071.stdout:+hv02.front.sepi 63.231.80.2 3 u 33 64 77 0.080 -0.205 0.801 2024-09-11T06:52:32.674 INFO:teuthology.orchestra.run.smithi071.stdout:*hv03.front.sepi 74.6.168.72 3 u 32 64 77 0.083 -0.128 0.799 2024-09-11T06:52:32.674 INFO:teuthology.orchestra.run.smithi071.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T06:52:32.676 INFO:teuthology.orchestra.run.smithi089.stdout: remote refid st t when poll reach delay offset jitter 2024-09-11T06:52:32.676 INFO:teuthology.orchestra.run.smithi089.stdout:============================================================================== 2024-09-11T06:52:32.676 INFO:teuthology.orchestra.run.smithi089.stdout:+hv01.front.sepi 67.205.162.81 3 u 30 64 77 0.104 -2.209 1.756 2024-09-11T06:52:32.677 INFO:teuthology.orchestra.run.smithi089.stdout:+hv02.front.sepi 63.231.80.2 3 u 33 64 77 0.069 +0.097 1.703 2024-09-11T06:52:32.677 INFO:teuthology.orchestra.run.smithi089.stdout:*hv03.front.sepi 74.6.168.72 3 u 33 64 77 0.130 +0.093 1.690 2024-09-11T06:52:32.677 INFO:teuthology.orchestra.run.smithi089.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T06:52:32.680 INFO:teuthology.orchestra.run.smithi176.stdout: remote refid st t when poll reach delay offset jitter 2024-09-11T06:52:32.680 INFO:teuthology.orchestra.run.smithi176.stdout:============================================================================== 2024-09-11T06:52:32.680 INFO:teuthology.orchestra.run.smithi176.stdout: hv01.front.sepi 67.205.162.81 3 u 32 64 77 0.069 -2.434 0.201 2024-09-11T06:52:32.680 INFO:teuthology.orchestra.run.smithi176.stdout: hv02.front.sepi 63.231.80.2 3 u 33 64 77 0.052 -0.278 0.145 2024-09-11T06:52:32.680 INFO:teuthology.orchestra.run.smithi176.stdout:*hv03.front.sepi 74.6.168.72 3 u 37 64 77 0.078 -0.004 0.220 2024-09-11T06:52:32.680 INFO:teuthology.orchestra.run.smithi176.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T06:52:32.681 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-11T06:52:32.692 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-11T06:52:32.693 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-11T06:52:32.701 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-11T06:52:32.710 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-11T06:52:32.719 INFO:teuthology.task.internal:Duration was 819.195148 seconds 2024-09-11T06:52:32.719 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-11T06:52:32.728 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-11T06:52:32.728 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T06:52:32.731 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T06:52:32.733 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T06:52:32.769 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-11T06:52:32.769 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi071.front.sepia.ceph.com 2024-09-11T06:52:32.769 DEBUG:teuthology.orchestra.run.smithi071:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T06:52:32.822 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi089.front.sepia.ceph.com 2024-09-11T06:52:32.822 DEBUG:teuthology.orchestra.run.smithi089:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T06:52:32.831 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi176.front.sepia.ceph.com 2024-09-11T06:52:32.832 DEBUG:teuthology.orchestra.run.smithi176:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T06:52:32.841 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-11T06:52:32.842 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T06:52:32.866 DEBUG:teuthology.orchestra.run.smithi089:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T06:52:32.871 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-11T06:52:32.878 DEBUG:teuthology.orchestra.run.smithi176:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T06:52:32.911 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-11T06:52:32.912 DEBUG:teuthology.orchestra.run.smithi071:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T06:52:32.938 DEBUG:teuthology.orchestra.run.smithi089:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T06:52:32.950 DEBUG:teuthology.orchestra.run.smithi176:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T06:52:33.015 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-11T06:52:33.024 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-11T06:52:33.024 DEBUG:teuthology.orchestra.run.smithi071:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T06:52:33.036 DEBUG:teuthology.orchestra.run.smithi089:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T06:52:33.046 DEBUG:teuthology.orchestra.run.smithi176:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T06:52:33.058 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-11T06:52:33.067 DEBUG:teuthology.orchestra.run.smithi071:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T06:52:33.086 DEBUG:teuthology.orchestra.run.smithi089:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T06:52:33.090 DEBUG:teuthology.orchestra.run.smithi176:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T06:52:33.094 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = core 2024-09-11T06:52:33.100 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = core 2024-09-11T06:52:33.112 INFO:teuthology.orchestra.run.smithi176.stdout:kernel.core_pattern = core 2024-09-11T06:52:33.195 DEBUG:teuthology.orchestra.run.smithi071:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T06:52:33.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T06:52:33.224 DEBUG:teuthology.orchestra.run.smithi089:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T06:52:33.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T06:52:33.241 DEBUG:teuthology.orchestra.run.smithi176:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T06:52:33.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T06:52:33.250 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-11T06:52:33.258 INFO:teuthology.task.internal:Transferring archived files... 2024-09-11T06:52:33.259 DEBUG:teuthology.misc:Transferring archived files from smithi071:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900631/remote/smithi071 2024-09-11T06:52:33.260 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T06:52:33.289 DEBUG:teuthology.misc:Transferring archived files from smithi089:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900631/remote/smithi089 2024-09-11T06:52:33.289 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T06:52:33.310 DEBUG:teuthology.misc:Transferring archived files from smithi176:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900631/remote/smithi176 2024-09-11T06:52:33.311 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T06:52:33.331 INFO:teuthology.task.internal:Removing archive directory... 2024-09-11T06:52:33.331 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T06:52:33.333 DEBUG:teuthology.orchestra.run.smithi089:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T06:52:33.351 DEBUG:teuthology.orchestra.run.smithi176:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T06:52:33.378 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-11T06:52:33.390 INFO:teuthology.task.internal:Not uploading archives. 2024-09-11T06:52:33.390 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-11T06:52:33.461 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-11T06:52:33.461 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T06:52:33.463 DEBUG:teuthology.orchestra.run.smithi089:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T06:52:33.466 DEBUG:teuthology.orchestra.run.smithi176:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T06:52:33.467 INFO:teuthology.orchestra.run.smithi071.stdout: 658616 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 06:52 /home/ubuntu/cephtest 2024-09-11T06:52:33.469 INFO:teuthology.orchestra.run.smithi089.stdout: 658639 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 06:52 /home/ubuntu/cephtest 2024-09-11T06:52:33.472 INFO:teuthology.orchestra.run.smithi176.stdout: 658639 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 06:52 /home/ubuntu/cephtest 2024-09-11T06:52:33.473 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-11T06:52:33.483 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-11T06:52:33.504 INFO:teuthology.run:Summary data: description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_nbd} duration: 819.1951479911804 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-11T06:52:33.504 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T06:52:33.595 INFO:teuthology.run:pass